HAME regional land use planning database compatible with national Ryhti data model
HAME-Ryhti consists of
- a PostGIS database,
- various AWS Lambda functions to manage the database and import or export planning data, and
- QGIS project to connect to the database and create regional land use plans.
To manage Hame-Ryhti AWS resources, check the infra directory.
- Python 3.12
- Docker (Install Docker based on your platform's instructions.)
- Create a Python virtual environment and activate it.
pip install pip-tools
pip-sync requirements.txt requirements-dev.txt
pre-commit install
- Run tests with
make pytest
- Build and start the development containers with
docker-compose -f docker-compose.dev.yml up -d
(ormake rebuild
). - Fill the database with current data model by
make test-create-db
. - Populate national code tables from koodistot.suomi.fi by
make test-koodistot
. - Edit the lambda functions under database, run tests and rebuild again.
If test using pytest-docker get stuck, you can remove the dangling containers with:
docker ps --format '{{.Names}}' |grep pytest | awk '{print $1}' | xargs -I {} docker stop {}
docker ps --format '{{.Names}}' |grep pytest | awk '{print $1}' | xargs -I {} docker rm {}
docker network ls --format {{.Name}} |grep pytest | awk '{print $1}' | xargs -I {} docker network rm {}
- Database is defined using SQLAlchemy, so familiarize yourself with SQLAlchemy declarative style.
- Database is divided into two schemas:
codes
contains all the Ryhti specific national code lists, whilehame
contains all the data tables (plans, plan objects, plan regulations etc.). - If you want to change all tables in a schema (i.e. edit all the code tables, or add a field to all the data tables), the abstract base classes are in base.py.
- If you only want to change/add one code table or one data table, please edit/add the right table in codes.py or models.py.
- To get the changes tested and usable in your functions, create a new database revision with
make revision name="describe_your_changes"
, e.g.make revision name="add_plan_object_table"
. This creates a new random id (uuid
) for your migration, and a revision fileYYYY-MM-DD-HHMM-uuid-add_plan_object_table
in the alembic versions dir. Please check that the autogenerated revision file seems to do approximately sensible things.- Specifically, when adding geometry fields, please note GeoAlchemy2 bug with Alembic, which means you will have to manually remove
op.create_index
andop.drop_index
in the revision file. This is because GeoAlchemy2 already automatically creates geometry index whenever adding a geometry column.
- Specifically, when adding geometry fields, please note GeoAlchemy2 bug with Alembic, which means you will have to manually remove
- Run tests with
make pytest
to check that the revision file runs correctly. At minimum, you may have to change the tested table counts (codes_count and hame_count) in database test setup to reflect the correct number of tables in the database. - Run
make rebuild
andmake test-create-db
to start development instance with the new model.
- Commit your changes and the new revision file in alembic versions dir.
To add new requirements:
- Add the Python library in requirements.in (if used in production) or requirements-dev.in (if used in development/CI/CD).
pip-compile requirements.in
orpip-compile requirements-dev.in
pip-sync requirements.txt requirements-dev.txt
To update requirements to latest versions:
pip-compile requirements.in --upgrade
andpip-compile requirements-dev.in --upgrade
pip-sync requirements.txt requirements-dev.txt