Skip to content

Commit

Permalink
updated readme, GA
Browse files Browse the repository at this point in the history
  • Loading branch information
JackDougherty committed Mar 4, 2024
1 parent 90443bb commit 314ab76
Show file tree
Hide file tree
Showing 3 changed files with 24 additions and 51 deletions.
57 changes: 10 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,60 +10,23 @@ https://ontheline.trincoll.edu
## Data
CT Open Data, Municipal Fiscal Indicators 2021 https://data.ct.gov/Local-Government/Municipal-Fiscal-Indicators-Economic-and-Grand-Lis/jrwx-mxhm/about_data
- Town
- Population 2021
- Income per capita 2021
- Population
- Income per capita
- Equalized Net Grand List (ENGL) as "taxable property"

### For annual updates,

### For annual updates
1. Locate the appropriate dataset on Socrata (https://data.ct.gov/)
1. Identify the dataset's API endpoint by clicking `Export` > `API Endpoint`
1. Identify appropriate column names for town name (`town` in 2021), population (`july_1_2021_population` in 2021), per capita income (`per_capita_income_calendar` in 2021), and equalized net grand list (`equalized_net_grand_list_fye_2021` in 2021)
1. Modify `index.html` (see lines 299-308) with all these details
1. Just in case, consider making a backup copy of the JSON file returned by Socrata so you can always switch to a local version. For 2021 data (as of March 2024), the URL is https://data.ct.gov/resource/jrwx-mxhm.json?$select=town,july_1_2021_population,per_capita_income_calendar,equalized_net_grand_list_fye_2021 and the data is saved to `data/data.20240302.backup.json`. It is printed in the Console (see index.html, line 311).
1. Just in case, consider making a backup copy of the JSON file returned by Socrata so you can always switch to a local version. For 2021 data (as of March 2024), the URL is https://data.ct.gov/resource/jrwx-mxhm.json?$select=town,july_1_2021_population,per_capita_income_calendar,equalized_net_grand_list_fye_2021 and the data is saved to `data/data.20240302.backup.json`. It is printed in the Console (see index.html, line 311). Alternate code line 311 would be:
```
$.getJSON('.data/data/20240302.backup.json', function (apiData) {
```

## Credits
- @kitzj Joe Kitz for redesigning code and interface
- @erose Eli Rose for helping me with the JavaScript logic
- @ilyankou Ilya Ilyankou for redesigning code to pull data directly from Socrata, 2024
- @kitzj Joe Kitz for redesigning code and interface, 2024
- @erose Eli Rose for assisting with JavaScript logic, 2015
- @nav10003 Natalia Vorotyntseva at UConn MAGIC (http://magic.lib.uconn.edu) for code to toggle on/off polygons, display results for multiple regions, and export results
- @alvinschang Alvin Chang for creating functions to display info window data, based on examples such as http://projects.ctmirror.org/content/2014/05/raceSchools/

## To Do
- update all README instructions below
- revise code to allow easier data import of Income Per Capita from source, and multiply by population for calculations in the code
- revise geojson and variable names to remove year
- display dollar signs in table
- right-justify numerical data in table
- check sums for display and exports, since they do not appear to match
- update to newer Leaflet hosted CDN https://leafletjs.com/download.html
- add GA code
- Create index-frame.html with caption, sources, credits


## Steps to construct this
Simplify instructions below: To join new data to the existing Connecticut towns geojson file, use http://mapshaper.org command line tools to jump over steps 2-3-4. See http://www.datavizforall.org/shape/mapshaper/index.html

1) Begin with Leaflet choropleth tutorial at http://leafletjs.com/examples/choropleth.html

2) Create simplified geoJson layer of Connecticut town boundaries
- Download town boundaries shapefile from UConn MAGIC http://magic.libraries.uconn.edu
- Simplify the polygon shapes to reduce size in geoJson format with this general approach http://chimera.labs.oreilly.com/books/1230000000345/ch12.html#_simplify_the_shapes
- Use MapShaper web tool (http://www.mapshaper.org/) to reduce size without changing too much detail. In this case, I reduced Connecticut boundaries down to 1.5%, around 100k.
- Export "shapefile - polygons", then copy and rename new .shp and .shx files into new folder with original .dbf (to retain data) and original .prj (projection) files

3) Join spreadsheet town data to the new simplified polygon shapefile
- in GIS application (such as ArcGIS or QGIS), load the new simplified polygon shapefile and spreadsheet data
- join data to shapefile, then export > save as new shapefile in new layer, and open attributes to delete unneeded columns
- create a new folder with four files (.shp, .shx, .dbf, .prj) and zip folder for next step

4) Convert new joined shapefile (simplified boundaries + data) into js format to use in Leaflet map
- Use ogre to ogre web client (http://ogre.adc4gis.com/) to convert zipped shapefile to geojson
- Copy and paste the geojson output into a new file, and modify into json this way:
- At top of new file, declare contents as a variable by adding this text:
```
var data =
```
- At the bottom of the new file, add a semicolon (;)
- Rename file suffix as .js, place inside local directory, and upload in the Leaflet HTML index template

5) See code comments in the Leaflet index.html template
Binary file added data/data-backup-screenshot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
18 changes: 14 additions & 4 deletions index.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,16 @@
<title>MetroMerger</title>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />

<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-5488840-4"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-5488840-4');
</script>

<link
rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/leaflet/1.9.4/leaflet.css"
Expand Down Expand Up @@ -288,7 +298,7 @@ <h3 id="region3_title">Metro Region 3</h3>
dblclick: resetFeature,
});
}

// Load data from Socrata via API
// https://data.ct.gov/Local-Government/Municipal-Fiscal-Indicators-Economic-and-Grand-Lis/jrwx-mxhm/about_data
// JSON URL for 2021 data (March 2024): https://data.ct.gov/resource/jrwx-mxhm.json?$select=town,july_1_2021_population,per_capita_income_calendar,equalized_net_grand_list_fye_2021
Expand All @@ -314,7 +324,7 @@ <h3 id="region3_title">Metro Region 3</h3>
var town2data = {};

for (let i = 0; i < apiData.length; i++) {

town2data[ apiData[i]['town'] ] = {
'population': parseInt(apiData[i][ apiDataColumns['population'] ]),
'incomePerCapita': parseInt(apiData[i][ apiDataColumns['incomePerCapita'] ]),
Expand All @@ -332,7 +342,7 @@ <h3 id="region3_title">Metro Region 3</h3>
town2data[ geojsonData.features[i].properties.name.toUpperCase() ]
);
}

geojson = L.geoJson(geojsonData, {
style: style,
onEachFeature: onEachFeature,
Expand Down Expand Up @@ -391,7 +401,7 @@ <h3 id="region3_title">Metro Region 3</h3>
}
function comma(val) {
if (isNaN(val)) {return '-'}

num = val;
val = Math.round(val);
while (/(\d+)(\d{3})/.test(val.toString())) {
Expand Down

0 comments on commit 314ab76

Please sign in to comment.