-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Jon colorado data (boulder) #2
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
0b6a468
working on creating boulder city pols and offices
jonbig d9ad0f8
working on creating boulder city pols and offices
jonbig 19ea27f
progress on politicians table
jonbig b523d56
progress on politicians table
jonbig 9fa061c
finishing pols and office table
jonbig ce3de6b
finished races and race/candidate records
jonbig 8a6c100
working on creating boulder city pols and offices
jonbig 03ab048
Lint and clean up
wileymc 88987fd
Rebase to main
wileymc a37dd3b
Fix issues
wileymc 9b82be5
Join politicians, races, and offices to intermediate model
wileymc ad588b9
Add dir structure
wileymc a7a73bb
Clean up seat and move seeds to seeds
wileymc ac153f6
Delete vscode file
wileymc File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
WITH transformed_filings AS ( | ||
SELECT | ||
email AS email, | ||
first_name AS first_name, | ||
last_name AS last_name, | ||
official_website_url AS official_website_url, | ||
home_state AS home_state, | ||
office AS office, | ||
office AS office_title, | ||
'city wide' AS district, | ||
'Boulder' AS county, | ||
CONCAT(first_name, ' ', last_name) AS candidate_full_name, | ||
CASE | ||
WHEN office ILIKE '%City Council%' THEN 'City Council' | ||
END AS political_scope | ||
FROM p6t_state_co.boulder_updated_filings | ||
), | ||
|
||
transformed_filings_1 AS ( | ||
SELECT | ||
email, | ||
first_name, | ||
last_name, | ||
official_website_url, | ||
home_state, | ||
office, | ||
candidate_full_name, | ||
political_scope, | ||
'city' AS district_type, | ||
SLUGIFY(candidate_full_name) AS politician_slug, | ||
CASE | ||
WHEN | ||
office ILIKE '%City Council%' OR office ILIKE '%Mayor%' | ||
THEN 'city' | ||
ELSE 'district' | ||
END AS election_scope, | ||
CASE | ||
WHEN office ILIKE '%Mayor%' THEN NULL | ||
ELSE 'At Large' | ||
END AS seat | ||
|
||
FROM transformed_filings | ||
) | ||
|
||
SELECT | ||
tf.*, | ||
p.id AS politician_id, | ||
r.id AS race_id, | ||
tf1.politician_slug, | ||
tf1.politician_slug AS slug, | ||
tf1.election_scope, | ||
tf1.district_type, | ||
tf1.seat, | ||
SLUGIFY( | ||
tf.home_state | ||
|| ' ' | ||
|| tf.office_title | ||
|| ' ' | ||
|| tf.county | ||
|| ' ' | ||
|| tf.district | ||
|| ' ' | ||
|| tf1.seat | ||
) AS office_slug | ||
FROM | ||
p6t_state_co.boulder_updated_filings AS f | ||
LEFT JOIN transformed_filings AS tf ON f.email = tf.email | ||
LEFT JOIN transformed_filings_1 AS tf1 ON tf.email = tf1.email | ||
LEFT JOIN public.politician AS p ON p.slug = tf1.politician_slug | ||
LEFT JOIN | ||
office AS o | ||
ON | ||
o.slug | ||
= SLUGIFY( | ||
CONCAT( | ||
'MN', | ||
' ', | ||
tf.office_title, | ||
' ', | ||
tf.county, | ||
' ', | ||
tf.district, | ||
' ', | ||
tf1.seat | ||
) | ||
) | ||
LEFT JOIN race AS r ON r.office_id = o.id |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
SELECT {{ dbt_utils.generate_surrogate_key(['email', 'first_name', 'last_name', 'office']) }} as _surrogate_key, * FROM p6t_state_co.boulder_updated_filings --noqa |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
SELECT DISTINCT ON (office) | ||
'general' AS race_type, | ||
home_state AS state, | ||
FALSE AS is_special_election, | ||
CASE | ||
WHEN office ILIKE '%Mayor%' THEN 'Mayor (Boulder)' | ||
ELSE 'Council Member (Boulder)' | ||
END AS title, | ||
slugify(concat(county, ' ', office_title, ' ', '2023')) AS slug, | ||
CASE | ||
WHEN office ILIKE '%Mayor%' THEN NULL | ||
ELSE 4 | ||
END AS num_elect | ||
FROM | ||
{{ ref('co_boulder_city_filings') }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
SELECT DISTINCT ON (office_slug) | ||
office_slug AS ref_key, | ||
office_slug AS slug, | ||
home_state AS state, | ||
office_title AS title, | ||
seat, | ||
district, | ||
political_scope AS political_scope, | ||
election_scope AS election_scope, | ||
district_type AS district_type, | ||
'Boulder' AS municipality, | ||
county AS county | ||
FROM | ||
{{ ref('co_boulder_city_filings') }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
SELECT DISTINCT ON (politician_slug) | ||
politician_slug AS ref_key, | ||
politician_slug AS slug, | ||
first_name, | ||
last_name, | ||
email, | ||
home_state | ||
FROM | ||
{{ ref ('co_boulder_city_filings') }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
SELECT DISTINCT ON (politician_id) | ||
politician_id AS candidate_id, | ||
race_id | ||
FROM | ||
{{ ref('co_boulder_city_filings') }} |
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
SELECT DISTINCT ON (office_slug) | ||
office_id AS id, | ||
office_slug AS ref_key, | ||
office_slug AS slug, | ||
state::state, | ||
state_id, | ||
office_title AS title, | ||
seat, | ||
district, | ||
school_district, | ||
political_scope::political_scope, | ||
election_scope::election_scope, | ||
district_type::district_type, | ||
municipality, | ||
county | ||
FROM | ||
{{ ref('mn_sos_local_filings') }} |
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
Committee Name,Expenditure Amount,Candidate Full Name,Incumbant,Office,District | ||
Coombs for Aurora,22948.61,Alison Coombs,No,Council Member At Large,City Wide | ||
Curtis for Aurora,46238.63,Curtis Gardner,Yes,Council Member At Large,City Wide | ||
Francoise for Aurora,16715.43,Francoise Bergan,Yes,Council Member,Ward 6 | ||
Gray for Aurora,4673.6,Jonathan Gray,No,Council Member,Ward 4 | ||
Jono Scott for Aurora,12104.16,Jono Scott,No,Mayor,City Wide | ||
Lawson for Aurora City Council,2449.35,Angela Lawson,No,Council Member,Ward 5 | ||
Marcano for Aurora,36827.33,Juan Marcano,No,Mayor,City Wide | ||
Mayes for Aurora,18642.86,Thomas Mayes,No,Council Member At Large,City Wide | ||
Mike for Mayor,45464.270000000004,Mike Coffman,Yes,Mayor,City Wide | ||
Rhodes for Aurora,4829.0,Chris Rhodes,No,Council Member,Ward 5 | ||
,,Jeffrey Sanford,No,Mayor,City Wide |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
,Ballot Name,Name,Phone,Email Adress,Website,street_address,city,zipcode,state,Office,Filing Date | ||
0,Aaron Brockett,Aaron Brockett,,[email protected],,1601 Yellow Pine Ave,Boulder,80304,CO,Mayor,8-9-2023 | ||
1,Aaron Gabriel Neyer,Aaron Neyer,Work: 513-593-1721,[email protected],regenboulder.org,3340 Dover Dr.,Boulder,80305,CO,, | ||
2,Bob Yates,Robert Yates,,[email protected],https://www.bobyatesboulder.com,3820 Cloverleaf Dr,Boulder,80304,CO,Mayor,8-8-2023 | ||
3,Jacques Decalo,Jacques Decalo,,[email protected],https://www.jacquesforcouncil.org/,2815 Iliff St,Boulder,80305,CO,City Council,8-9-2023 | ||
4,Jenny Robins,Jennifer Robins,,[email protected],jennyforboulder.com,5278 Pinehurst Dr.,Boulder,80301,CO,, | ||
5,Nicole Speer,Nicole Speer,,[email protected],www.nicoleforboulder.com,,Boulder,80305,CO,Mayor,8-8-2023 | ||
6,Paul Tweedlie,Paul Tweedlie,,[email protected],,2717 4th Street,Boulder,80304,CO,Mayor,8-28-2023 | ||
7,Ryan Schuchard,Ryan Schuchard,Work: 970-5410902,[email protected],,2179 Kincaid Pl.,Boulder,80304,CO,City Council,8-16-2023 | ||
8,Silas Atkins,Silas Atkins,,[email protected],silasforboulder.com,3040 Ash Ave,Boulder,80305,CO,City Council,8-16-2023 | ||
9,Taishya Adams,Taishya Adams,,[email protected],,2140 Kohler Dr,Boulder,80305,CO,City Council,8-8-2023 | ||
10,Tara Winer,Tara Winer,,[email protected],taraforboulder.com,1510 Columbine Ave,Boulder,80302,CO,City Council,8-8-2023 | ||
11,Terri Brncic,Terri Brncic,,[email protected],https://www.terri4boulder.com/,3076 9th St,Boulder,80304,CO,City Council,8-10-2023 | ||
12,Tina Marquis,Christina Marquis,,[email protected],www.tinaforboulder.com,3091 6th St,Boulder,80304,CO,City Council,8-15-2023 | ||
13,Waylon Lewis,Waylon Lewis,,[email protected],,838 University Ave,Boulder,80302,CO,City Council,8-8-2023 | ||
14,Keith Pitcher,Keith Pitcher,,,,4621 Gordon Drive,Boulder,80302,CO,, | ||
15,Phillip J. Ziols,Phillip Ziols,,,,3518 Kirkwood Pl.,Boulder,80304,CO,, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
id,email,official_website_url,city,home_state,office,first_name,last_name | ||
0,[email protected],www.tweedlie4bouldermayor.com,Boulder,CO,Mayor,Aaron,Brockett | ||
1,[email protected],regenboulder.org,Boulder,CO,City Council,Aaron,Neyer | ||
2,[email protected],https://www.bobyatesboulder.com,Boulder,CO,Mayor,Robert,Yates | ||
3,[email protected],https://www.jacquesforcouncil.org/,Boulder,CO,City Council,Jacques,Decalo | ||
4,[email protected],jennyforboulder.com,Boulder,CO,City Council,Jennifer,Robins | ||
5,[email protected],www.nicoleforboulder.com,Boulder,CO,Mayor,Nicole,Speer | ||
6,[email protected],,Boulder,CO,Mayor,Paul,Tweedlie | ||
7,[email protected],,Boulder,CO,City Council,Ryan,Schuchard | ||
8,[email protected],silasforboulder.com,Boulder,CO,City Council,Silas,Atkins | ||
9,[email protected],www.adamsforboulder.com,Boulder,CO,City Council,Taishya,Adams | ||
10,[email protected],taraforboulder.com,Boulder,CO,City Council,Tara,Winer | ||
11,[email protected],https://www.terri4boulder.com/,Boulder,CO,City Council,Terri,Brncic | ||
12,[email protected],www.tinaforboulder.com,Boulder,CO,City Council,Christina,Marquis | ||
13,[email protected],,Boulder,CO,City Council,Waylon,Lewis |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i dont think these joins are needed as you can select from either of the CTEs in this final select statement to get exactly what you need