Import data.gouv.fr Sirene in PostgreSQL with API REST

Status: Work in Progress, see issues

Import Sirene data from data.gouv.fr website to PostgreSQL.

PostgREST service is used to provide Sirene API REST.

Importation notes

Considering that the file StockUniteLegale_utf8.csv is very big 2.6G then some filters are applied before importation:

• drop Cessée etatAdministratifUniteLegale
• use --ape-filter to apply a filter on APE code

Prerequisites

• Docker Engine
• Docker Compose
• Python3

Homebrew instructions:

$ brew cask install docker
$ brew install python3 curl

Getting started

Download data from https://files.data.gouv.fr/insee-sirene/:

$ ./scripts/download-data.sh

Start and initialize PostgreSQL database:

$ ./scripts/up.sh

Setup Python environment:

$ python3 -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt

Import StockUniteLegale_utf8.csv file to PostgreSQL database:

$ ./import-csv-to-postgres.py --csv datas/StockUniteLegale_utf8.csv --db postgresql://postgres:[email protected]/postgres --ape-filter 62.01Z

Start REST API server (based on PostgREST):

$ docker-compose up -d

Urls:

• REST API: http://127.0.0.1:3000/
• Swagger doc: http://127.0.0.1:8080/