Jon colorado data (boulder) #2
Conversation
|
Some issues neeed to be addressed before merging this in:
|
|
|
||
|
|
||
| --fields for politician table | ||
| ----Where is the id field coming from? |
There was a problem hiding this comment.
Id's are generated on INSERT for pretty much every table (they are UUIDs which aren't sequential or dependent on one another)
There was a problem hiding this comment.
Got it, that has been confusing me (and is why I had a few fields. commented out) because if the IDs are generated on insert they don't exist in my table yet, I won't be able to add them to the select, right?
There was a problem hiding this comment.
Correct, they aren't needed for these staging SELECTs
populist.code-workspace
Outdated
There was a problem hiding this comment.
add this file to .gitignore
wileymc
force-pushed
the
jon_colorado_data
branch
from
31de5e9
to
88987fd
Compare
|
I created the boulder_updated_filings table from the boulder_updated_filings.csv, that should be the only file needed to run the models. |
| ELSE 'district' | ||
| END AS election_scope, | ||
| CASE | ||
| WHEN office ILIKE '%Mayor%' THEN 'Mayor' |
There was a problem hiding this comment.
I'm not sure we want to set seat as "Mayor" in this case
There was a problem hiding this comment.
You'll also need to join this model to our existing public.politician table so that we can deduplicate politicians and insert the race_candidate records properly. Look at what i did in the mn intermediate model
| LEFT JOIN transformed_filings AS tf ON f.email = tf.email | ||
| LEFT JOIN transformed_filings_1 AS tf1 ON tf.email = tf1.email |
There was a problem hiding this comment.
i dont think these joins are needed as you can select from either of the CTEs in this final select statement to get exactly what you need
I first created an intermediate table where I did most of the transformations, then created tables for boulder races, offices, politicians, and race candidates. I used the existing tables as a guide and they appear to almost match. One area where I see differences is when it comes to the ID fields. I left those fields in models, but commented out.
I was under the impression those uuid fields are generated when we insert the rows into the tables, so maybe they just need to be inserted in a specific order so that the first uuid field is generated, and the rest of the uuid fields are based on that?
I commented out the insert statements.
I used sql fluff to lint the models.