Search Analysis and Visualization Environment
The Search Analysis and Visualization Environment is a component of the Data+Climate collaboration, joining the diverse knowledge resources of Data Commons with tools such as the glue visualization library. This repository is a meta-package collecting the various components to allow for easy installation.
The core components of SAVE worked on or developed under the Data+Climate collaboration include:
- glue-jupyter -- glue in a Jupyter notebook
- glue-map -- a glue-jupyter plugin for:
- Loading geospatial data from a variety of vector formats
- The display and analysis of geo-spatial data, leveraging ipyleaflet and xarray-leaflet
- A customizable ipywidget showing how to dynamically add Data Commons data to an existing glue-jupyter session
- ipyleaflet -- An ipywidget for display of maps using leaflet.js
- xarray-leaflet -- to enable the visualization of large datasets efficiently
- glue-jupyter-plugin-template -- a template plug-in demonstrating how to turn ipywidgets into glue-jupyter widgets.
Two prototype plug-ins for the desktop (QT) version of glue allow access to external data:
- glue-dataverse -- access to Dataverse data resources through an integrated web client
- glue-data-commons -- access to DataCommons data resources
In addition, the following components are used in SAVE to produce standalone websites/dashboards:
- voila -- to host a SAVE analysis session as a stand-alone webpage
- voila gridstack -- the preferred voila template to allow for easy organization of different visualizations on the same page.
As part of the Climate and Justice Design Fellowship under the Data+Climate pilot project, SAVE was used to:
- Conduct data analysis and export specific visualizations for the California Map of Hazardous Waste Sites and Schools by Idalmis Vaquero
- Provide interactive visualizations for the Blue-Lining project by Carlos Claussel. A version of these interactive visualizations are available here (this server is not running all the time).
We recommend creating a new Python virtual environment in which to install and use SAVE. The libraries underlying GeoPandas (used to represent geo-spatial data within glue) may prove difficult to install with pip on some platforms, in which case we recommend using Conda/Mamba to install GeoPandas before attempting to pip install this package.
pip install git+https://github.com/jfoster17/SAVE.git
We have found that long-running deployments of SAVE via voila lead to the server running out of resources as old processes associated with the voila kernal continue to hang around, consuming memory. We recommend deploying voila with the option to cull old kernals like this:
voila --no-browser MY-SAVE-NOTEBOOK.ipynb --template=gridstack --preheat_kernel=True --pool_size=4 --ExecutePreprocessor.timeout=180 --MappingKernelManager.cull_interval=120 --MappingKernelManager.cull_idle_timeout=600