Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jon colorado data (boulder) #2

Merged
merged 14 commits into from
Nov 14, 2023
12 changes: 12 additions & 0 deletions aurora_municipal_candidates.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Committee Name,Expenditure Amount,Candidate Full Name,Incumbant,Office,District
Coombs for Aurora,22948.61,Alison Coombs,No,Council Member At Large,City Wide
Curtis for Aurora,46238.63,Curtis Gardner,Yes,Council Member At Large,City Wide
Francoise for Aurora,16715.43,Francoise Bergan,Yes,Council Member,Ward 6
Gray for Aurora,4673.6,Jonathan Gray,No,Council Member,Ward 4
Jono Scott for Aurora,12104.16,Jono Scott,No,Mayor,City Wide
Lawson for Aurora City Council,2449.35,Angela Lawson,No,Council Member,Ward 5
Marcano for Aurora,36827.33,Juan Marcano,No,Mayor,City Wide
Mayes for Aurora,18642.86,Thomas Mayes,No,Council Member At Large,City Wide
Mike for Mayor,45464.270000000004,Mike Coffman,Yes,Mayor,City Wide
Rhodes for Aurora,4829.0,Chris Rhodes,No,Council Member,Ward 5
,,Jeffrey Sanford,No,Mayor,City Wide
17 changes: 17 additions & 0 deletions boulder_candidates.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
,Ballot Name,Name,Phone,Email Adress,Website,street_address,city,zipcode,state,Office,Filing Date
0,Aaron Brockett,Aaron Brockett,,[email protected],,1601 Yellow Pine Ave,Boulder,80304,CO,Mayor,8-9-2023
1,Aaron Gabriel Neyer,Aaron Neyer,Work: 513-593-1721,[email protected],regenboulder.org,3340 Dover Dr.,Boulder,80305,CO,,
2,Bob Yates,Robert Yates,,[email protected],https://www.bobyatesboulder.com,3820 Cloverleaf Dr,Boulder,80304,CO,Mayor,8-8-2023
3,Jacques Decalo,Jacques Decalo,,[email protected],https://www.jacquesforcouncil.org/,2815 Iliff St,Boulder,80305,CO,City Council,8-9-2023
4,Jenny Robins,Jennifer Robins,,[email protected],jennyforboulder.com,5278 Pinehurst Dr.,Boulder,80301,CO,,
5,Nicole Speer,Nicole Speer,,[email protected],www.nicoleforboulder.com,,Boulder,80305,CO,Mayor,8-8-2023
6,Paul Tweedlie,Paul Tweedlie,,[email protected],,2717 4th Street,Boulder,80304,CO,Mayor,8-28-2023
7,Ryan Schuchard,Ryan Schuchard,Work: 970-5410902,[email protected],,2179 Kincaid Pl.,Boulder,80304,CO,City Council,8-16-2023
8,Silas Atkins,Silas Atkins,,[email protected],silasforboulder.com,3040 Ash Ave,Boulder,80305,CO,City Council,8-16-2023
9,Taishya Adams,Taishya Adams,,[email protected],,2140 Kohler Dr,Boulder,80305,CO,City Council,8-8-2023
10,Tara Winer,Tara Winer,,[email protected],taraforboulder.com,1510 Columbine Ave,Boulder,80302,CO,City Council,8-8-2023
11,Terri Brncic,Terri Brncic,,[email protected],https://www.terri4boulder.com/,3076 9th St,Boulder,80304,CO,City Council,8-10-2023
12,Tina Marquis,Christina Marquis,,[email protected],www.tinaforboulder.com,3091 6th St,Boulder,80304,CO,City Council,8-15-2023
13,Waylon Lewis,Waylon Lewis,,[email protected],,838 University Ave,Boulder,80302,CO,City Council,8-8-2023
14,Keith Pitcher,Keith Pitcher,,,,4621 Gordon Drive,Boulder,80302,CO,,
15,Phillip J. Ziols,Phillip Ziols,,,,3518 Kirkwood Pl.,Boulder,80304,CO,,
15 changes: 15 additions & 0 deletions boulder_updated_filings.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
,email,official_website_url,city,home_state,Office,first_name,last_name
0,[email protected],www.tweedlie4bouldermayor.com,Boulder,CO,Mayor,Aaron,Brockett
1,[email protected],regenboulder.org,Boulder,CO,City Council,Aaron,Neyer
2,[email protected],https://www.bobyatesboulder.com,Boulder,CO,Mayor,Robert,Yates
3,[email protected],https://www.jacquesforcouncil.org/,Boulder,CO,City Council,Jacques,Decalo
4,[email protected],jennyforboulder.com,Boulder,CO,City Council,Jennifer,Robins
5,[email protected],www.nicoleforboulder.com,Boulder,CO,Mayor,Nicole,Speer
6,[email protected],,Boulder,CO,Mayor,Paul,Tweedlie
7,[email protected],,Boulder,CO,City Council,Ryan,Schuchard
8,[email protected],silasforboulder.com,Boulder,CO,City Council,Silas,Atkins
9,[email protected],www.adamsforboulder.com,Boulder,CO,City Council,Taishya,Adams
10,[email protected],taraforboulder.com,Boulder,CO,City Council,Tara,Winer
11,[email protected],https://www.terri4boulder.com/,Boulder,CO,City Council,Terri,Brncic
12,[email protected],www.tinaforboulder.com,Boulder,CO,City Council,Christina,Marquis
13,[email protected],,Boulder,CO,City Council,Waylon,Lewis
1,423 changes: 1,423 additions & 0 deletions colorado_candidates_sos_september_2023.csv

Large diffs are not rendered by default.

67 changes: 67 additions & 0 deletions models/intermediate/co_boulder_city_filings.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
WITH transformed_filings AS (
SELECT
email AS email,
first_name AS first_name,
last_name AS last_name,
official_website_url AS official_website_url,
home_state AS home_state,
office AS office,
office AS office_title,
'city wide' AS district,
'Boulder' AS county,
CONCAT(first_name, ' ', last_name) AS candidate_full_name,
CASE
WHEN office ILIKE '%City Council%' THEN 'City Council'
END AS political_scope

FROM p6t_state_co.boulder_updated_filings
),

transformed_filings_1 AS (
SELECT
email,
first_name,
last_name,
official_website_url,
home_state,
office,
candidate_full_name,
political_scope,
'city' AS district_type,
SLUGIFY(candidate_full_name) AS politician_slug,
CASE
WHEN
office ILIKE '%City Council%' OR office ILIKE '%Mayor%'
THEN 'city'
ELSE 'district'
END AS election_scope,
CASE
WHEN office ILIKE '%Mayor%' THEN 'Mayor'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we want to set seat as "Mayor" in this case

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll also need to join this model to our existing public.politician table so that we can deduplicate politicians and insert the race_candidate records properly. Look at what i did in the mn intermediate model

ELSE 'At Large'
END AS seat

FROM transformed_filings
)

SELECT
tf.*,
tf1.politician_slug,
tf1.politician_slug AS slug,
tf1.election_scope,
tf1.district_type,
tf1.seat,
SLUGIFY(
tf.home_state
|| ' '
|| tf.office_title
|| ' '
|| tf.county
|| ' '
|| tf.district
|| ' '
|| tf1.seat
) AS office_slug
FROM
p6t_state_co.boulder_updated_filings AS f
LEFT JOIN transformed_filings AS tf ON f.email = tf.email
LEFT JOIN transformed_filings_1 AS tf1 ON tf.email = tf1.email
Comment on lines +67 to +68
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i dont think these joins are needed as you can select from either of the CTEs in this final select statement to get exactly what you need

Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
SELECT {{ dbt_utils.generate_surrogate_key(['email', 'first_name', 'last_name', 'office']) }} as _surrogate_key, * FROM p6t_state_co.boulder_updated_filings --noqa
15 changes: 15 additions & 0 deletions models/staging/stg_co_boulder_city_local_races.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
SELECT DISTINCT ON (office)
'general' AS race_type,
home_state AS state,
FALSE AS is_special_election,
CASE
WHEN office ILIKE '%Mayor%' THEN 'Mayor (Boulder)'
ELSE 'Council Member (Boulder)'
END AS title,
slugify(concat(county, ' ', office_title, ' ', '2023')) AS slug,
CASE
WHEN office ILIKE '%Mayor%' THEN NULL
ELSE 4
END AS num_elect
FROM
{{ ref('co_boulder_city_filings') }}
15 changes: 15 additions & 0 deletions models/staging/stg_co_boulder_city_offices.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
SELECT DISTINCT ON (office_slug)
office_slug AS ref_key,
office_slug AS slug,
home_state AS state,
office_title AS title,
seat,
district,
school_district,
political_scope AS political_scope,
election_scope AS election_scope,
district_type AS district_type,
'Boulder' AS municipality,
county AS county
FROM
{{ ref('co_boulder_city_filings') }}
9 changes: 9 additions & 0 deletions models/staging/stg_co_boulder_city_politicians.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
SELECT DISTINCT ON (politician_slug)
politician_slug AS ref_key,
politician_slug AS slug,
first_name,
last_name,
email,
home_state
FROM
{{ ref ('co_boulder_city_filings') }}
5 changes: 5 additions & 0 deletions models/staging/stg_co_boulder_city_race_candidates.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
SELECT DISTINCT ON (politician_id)
politician_id AS candidate_id,
race_id
FROM
{{ ref('co_boulder_city_filings') }}
33 changes: 0 additions & 33 deletions models/staging/stg_mn_sos_local_offices.sql
Original file line number Diff line number Diff line change
Expand Up @@ -15,36 +15,3 @@ SELECT DISTINCT ON (office_slug)
county
FROM
{{ ref('mn_sos_local_filings') }}


{# INSERT INTO office (
ref_key,
slug,
state,
state_id,
title,
seat,
district,
school_district,
political_scope,
election_scope,
district_type,
municipality,
county
)
SELECT
ref_key,
slug,
state,
state_id,
title,
seat,
district,
school_district,
political_scope,
election_scope,
district_type,
municipality,
county
FROM
dbt_wiley.stg_office #}
7 changes: 0 additions & 7 deletions models/staging/stg_mn_sos_local_races.sql
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,3 @@ SELECT DISTINCT ON (office_id)
slugify(concat(office_title_raw_no_choice, ' ', '2023')) AS slug
FROM
{{ ref('mn_sos_local_filings') }};


{# INSERT INTO race (
title, slug, office_id, race_type, state, is_special_election, num_elect
)
SELECT *
FROM dbt_models.stg_races #}
13 changes: 13 additions & 0 deletions populist.code-workspace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add this file to .gitignore

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"folders": [
{
"path": "."
},
{
"path": "../platform"
}
],
"settings": {
"dbt.queryLimit": 500
}
}
Loading