BrainTrust Investment Lab › Band 1 — Extraction (Remote Sources → Local System)
✗ SS2 HALTED — rank 9
fsds step 3 FAILEDp.451-452 -- capture ALL changes, closed both ways — RuntimeError: partition_column '' exists in no declared table of `fsds` -- the reconcile has no axis to run along.
✗ SS2 HALTED — rank 12
sec_name_cik step 4 FAILEDp.451 -- reason codes: corrections vs true updates — MemoryError: …
✗ SS2 HALTED — rank 15
fred step 1Now (2026-10-16 10:02, OWED): `fred` declares CDC mode `provider_stamp` but does not say which of p.452's techniques it is. …
What tripped it (2026-10-16 05:08): p.451-452 -- a change strategy, per source, mapped — OutOfMemory: out of memory DETAIL: Failed on request of size 986 in memory context "Caller tuples".
… and 2 more.
The walk stops here: the next rank is not released until this is answered.
Band 1 — Extraction (Remote Sources → Local System) · book 451-453 · grain: one source per walk sheet
Knowing what changed is the difference between reloading the world and loading the truth. This station pins each source's honest change signal — and where a provider revises history silently, we catch it by measurement, never by trust.
completed
1 of 48
running now
sec_cik
completedrunningawaiting its turn · owedfailedexcluded
sec_cik↻rank 1 · walking · 5 step(s) certified
security_master↻rank 2 · walking · 5 step(s) certified
prices↻rank 3 · walking · 5 step(s) certified
edgar_filings↻rank 4 · walking · 4 certified · 1 owed
companyfacts↻rank 5 · walking · 5 step(s) certified
submissions↻rank 6 · walking · 5 step(s) certified
overview↻rank 7 · walking · 5 step(s) certified
form345↻rank 8 · walking · 5 step(s) certified
fsds↻rank 9 · walking · 2 step(s) certified
ftdrank 10 · owed · 1 step(s) owed
finra_short↻rank 11 · walking · 2 certified · 1 owed
sec_name_cik✓rank 12 · certified · closed · 2 step(s)
notes↻rank 13 · walking · 1 step(s) certified
framesrank 14 · owed · 1 step(s) owed
fredrank 15 · owed · 1 step(s) owed
waybackrank 16 · queued
pbsrank 17 · queued
wikipediarank 18 · queued
edgarrank 19 · queued
av_transcripts↻rank 20 · walking · 5 step(s) certified
shred_8krank 21 · owed · 1 step(s) owed
shred_13dgrank 22 · owed · 1 step(s) owed
aapbrank 23 · queued
bdcrank 24 · owed · 1 step(s) owed
crowdfundingrank 25 · owed · 1 step(s) owed
form13f↻rank 26 · walking · 1 certified · 1 owed
formd↻rank 27 · walking · 1 certified · 1 owed
market_structurerank 28 · owed · 1 step(s) owed
mfp_rr↻rank 29 · walking · 1 certified · 1 owed
ncen↻rank 30 · walking · 1 certified · 1 owed
nmfp↻rank 31 · walking · 1 certified · 1 owed
nportrank 32 · owed · 1 step(s) owed
rega↻rank 33 · walking · 1 certified · 1 owed
sec_enforcementrank 34 · queued
sec_filings_statsrank 35 · owed · 1 step(s) owed
sec_registriesrank 36 · queued
transfer_agent↻rank 37 · walking · 1 certified · 1 owed
vip↻rank 38 · walking · 1 certified · 1 owed
fomc_textrank 39 · queued
lda_lobbyingrank 40 · excluded
av_options↻rank 41 · walking · 5 step(s) certified
wikidatarank 42 · queued
dividends↻rank 43 · walking · 5 step(s) certified
splits↻rank 44 · walking · 5 step(s) certified
gdeltrank 45 · queued
usaspendingrank 46 · queued
finra_regshorank 47 · queued
edgar_logsrank 48 · queued
edgar_docsrank 49 · queued
The contract — what each step must produce
| # | step | page | what the book requires |
|---|---|---|---|
| 3 | the CDC mode is DECLARED by LOOKUP, enum-gated against the six signed modes | p.451-452 | 'You must carefully evaluate your strategy for each data source. Determining the appropriate strategy to identify changed data may take some… |
| 4 | audit columns tested, not trusted | p.452 | 'pay special attention to their integrity, analyzing and testing each column... If you uncover any NULL values, you must find an alternative approach… |
| 5 | restart safety proven by the dup screen | p.452 | 'This process is horribly unreliable. Time-based data selection loads duplicate rows when it is restarted from mid-process failures.' |
| 6 | deletion visibility MEASURED by the mode's own instrument | p.452 | 'Finally, you need to understand what happens when a record is deleted from the source because querying the audit column may not capture this event.' |
| 7 | corrections distinguished from true updates -- candidates ENUMERATED, every one disposed | p.451 | 'Tag changed data with reason codes to distinguish error corrections from true updates.' |
| 9 | no skipped period can hide -- the PROVIDER'S denominator, both directions | p.452 | 'if the nightly load process fails to run and skips a day, there’s a risk that the missed data will never make it into the data warehouse.' |
| 10 | CDC as early as the provider allows -- the shelf digest is the gate | p.452 | 'Perform the CDC step as early as possible, preferably before a bulk data transfer to the data warehouse.' |
| 11 | THE LANDING (blocked until step 9's SIGN) | p.421 (Develop Physical Database Model, under the Physical Design heading on p.420) | "There are several additional sets of tables that need to be designed and deployed as part of the DW/BI system, including staging tables to support… |