BrainTrust Investment Lab › Band 2 — Transform (Cleaning & Conforming)
Band 2 — Transform (Cleaning & Conforming) · book 460-461 · grain: one source per walk sheet
One company, one row — proven, not assumed. Duplicates are hunted with measured keys and merged by rule, and every survivorship decision is written down where an auditor can find it.
completed
0 of 48
running now
—
completedrunningawaiting its turn · owedfailedexcluded
sec_cikrank 1 · owed · 1 step(s) owed
security_masterrank 2 · owed · 6 step(s) owed
pricesrank 3 · owed · 6 step(s) owed
edgar_filingsrank 4 · queued
companyfactsrank 5 · queued
submissionsrank 6 · queued
overviewrank 7 · queued
form345rank 8 · queued
fsdsrank 9 · queued
ftdrank 10 · queued
finra_shortrank 11 · queued
sec_name_cikrank 12 · queued
notesrank 13 · queued
framesrank 14 · queued
fredrank 15 · queued
waybackrank 16 · queued
pbsrank 17 · queued
wikipediarank 18 · queued
edgarrank 19 · queued
av_transcriptsrank 20 · queued
shred_8krank 21 · queued
shred_13dgrank 22 · queued
aapbrank 23 · queued
bdcrank 24 · queued
crowdfundingrank 25 · queued
form13frank 26 · queued
formdrank 27 · queued
market_structurerank 28 · queued
mfp_rrrank 29 · queued
ncenrank 30 · queued
nmfprank 31 · queued
nportrank 32 · queued
regarank 33 · queued
sec_enforcementrank 34 · queued
sec_filings_statsrank 35 · queued
sec_registriesrank 36 · queued
transfer_agentrank 37 · queued
viprank 38 · queued
fomc_textrank 39 · queued
lda_lobbyingrank 40 · excluded
av_optionsrank 41 · queued
wikidatarank 42 · queued
dividendsrank 43 · queued
splitsrank 44 · queued
gdeltrank 45 · queued
usaspendingrank 46 · queued
finra_regshorank 47 · queued
edgar_logsrank 48 · queued
edgar_docsrank 49 · queued
The contract — what each step must produce
| # | step | page | what the book requires |
|---|---|---|---|
| 2 | the sources, named and counted | p.460 | 'Often dimensions are derived from several sources... many customer-facing source systems that create and manage separate customer master tables.' |
| 3 | the match, DECLARED in priority order | p.461 | 'a match may be based on fuzzy criteria, such as names and addresses that may nearly match except for minor spelling differences.' |
| 4 | the priority sequence, RULED PER COLUMN | p.461 | 'clear business rules that define the priority sequence for column values from all possible source systems to enable the creation of a single row… |
| 5 | the source natural keys are KEPT | p.461 | 'you must maintain separate columns with back references, such as natural keys, to all participating source systems used to construct the row.' |
| 7 | every match decision is a ROW | p.460 | 'each quality screen has the responsibility for populating these tables at the time of an error.' |