Skip to content

Commit

Permalink
chore(#1512): use lower case dbt (#1515)
Browse files Browse the repository at this point in the history
  • Loading branch information
andrablaj authored Aug 30, 2024
1 parent afccb3c commit 2874d33
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 10 deletions.
12 changes: 6 additions & 6 deletions content/en/apps/guides/data/analytics/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,36 +22,36 @@ The first time you run the commands from any of the sections below it will need
{{% /alert %}}

### Run all CHT Sync services locally
This setup involves starting couch2pg, PostgreSQL, pgAdmin, DBT, and CouchDB.
This setup involves starting couch2pg, PostgreSQL, pgAdmin, dbt, and CouchDB.

Run the Docker containers and wait for every container to be up and running:
```sh
docker-compose -f docker-compose.couchdb.yml -f docker-compose.postgres.yml -f docker-compose.yml up -d
```

You can verify this command worked by running `docker ps`. It should show 5 containers running including couch2pg, DBT, PostgreSQL, CouchDB and pgAdmin.
You can verify this command worked by running `docker ps`. It should show 5 containers running including couch2pg, dbt, PostgreSQL, CouchDB and pgAdmin.

Now that all services are running, use pgAdmin to connect to server `postgres:5432` with user `postgres` and password `postgres`. You should be able to see data being inserted into the `v1.medic` table when inserting sample data into the CouchDB instance.

### Separate CouchDB instance
This setup involves starting couch2pg, PostgreSQL, pgAdmin and DBT. It assumes you have a CouchDB instance running, and you updated the `.env` CouchDB variables accordingly.
This setup involves starting couch2pg, PostgreSQL, pgAdmin and dbt. It assumes you have a CouchDB instance running, and you updated the `.env` CouchDB variables accordingly.

Run the Docker containers locally and wait for every container to be up and running:
```sh
docker-compose -f docker-compose.postgres.yml -f docker-compose.yml up -d
```

You can verify this command worked by running `docker ps`. It should show 4 containers running including couch2pg, DBT, PostgreSQL, and pgAdmin.
You can verify this command worked by running `docker ps`. It should show 4 containers running including couch2pg, dbt, PostgreSQL, and pgAdmin.

### Separate CouchDB and PostgreSQL instances
This local setup involves starting couch2pg and DBT. It assumes that CouchDB and PostgreSQL instances are run separately from the Docker Compose provided with CHT Sync, and the `.env` variables were updated to match those instances details.
This local setup involves starting couch2pg and dbt. It assumes that CouchDB and PostgreSQL instances are run separately from the Docker Compose provided with CHT Sync, and the `.env` variables were updated to match those instances details.

Run the Docker containers locally and wait for every container to be up and running:
```sh
docker-compose -f docker-compose.yml up -d
```

You can verify this command worked by running `docker ps`. It should show 2 containers running: couch2pg and DBT.
You can verify this command worked by running `docker ps`. It should show 2 containers running: couch2pg and dbt.

### Cleanup
When you are done using the services, you can clean everything by running `down`.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,4 +37,4 @@ There is a Slack channel *#exercism* where engineers and tech enthusiasts meet a
| **Web Performance** | [High Performance Browser Networking](https://hpbn.co/)|
| **PostgreSQL** | [PostgreSQL Tutorial](https://www.postgresqltutorial.com/) <br> [PostgreSQL Cheat Sheet](https://www.postgresqltutorial.com/postgresql-cheat-sheet/) <br> [PostgreSQL Views Tutorial](https://www.postgresqltutorial.com/postgresql-views/) <br> [PostgreSQL Views Documentation](https://www.postgresql.org/docs/current/tutorial-views.html) <br> [Common Table Expressions (CTEs)](https://www.postgresqltutorial.com/postgresql-tutorial/postgresql-cte/) <br> [PostgreSQL Queries with CTEs Documentation](https://www.postgresql.org/docs/current/queries-with.html) |
| **Superset** | [Introduction to Superset](https://superset.apache.org/docs/intro/) <br> [Creating Your First Dashboard](https://superset.apache.org/docs/creating-charts-dashboards/creating-your-first-dashboard/) |
| **DBT** | [DBT Core Manual Installation Guide](https://docs.getdbt.com/docs/core/installation) <br> [DBT Video Playlist](https://www.youtube.com/playlist?list=PLohMhitTY9xuEVMpLG3xXhsKG9j2XCTeF) |
| **dbt** | [dbt Core Manual Installation Guide](https://docs.getdbt.com/docs/core/installation) <br> [dbt Video Playlist](https://www.youtube.com/playlist?list=PLohMhitTY9xuEVMpLG3xXhsKG9j2XCTeF) |
6 changes: 3 additions & 3 deletions content/en/core/overview/cht-sync.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Read more about setting up [CHT Sync]({{< relref "apps/guides/data/analytics/set
[CHT Sync](https://github.com/medic/cht-sync) uses `couch2pg` to replicate data from CouchDB to PostgreSQL in a real-time manner. It listens to changes in the CHT database, and updates the analytics database accordingly.
It is not designed to be accessed by users, and it does not have a user interface. It is designed to be run on the same server as the CHT, but it can be run on a separate server if necessary.

As CHT Sync puts all new data into a PostgreSQL database into a single table that has a `jsonb` column, this is not very useful for analytics. [CHT Pipeline](https://github.com/medic/cht-pipeline) is a set of SQL queries that transform the data in the `jsonb` column into a more useful format. It uses [DBT](https://www.getdbt.com/) to define the models that are translated into PostgreSQL tables or views, which makes it easier to query the data in the analytics platform of choice.
As CHT Sync puts all new data into a PostgreSQL database into a single table that has a `jsonb` column, this is not very useful for analytics. [CHT Pipeline](https://github.com/medic/cht-pipeline) is a set of SQL queries that transform the data in the `jsonb` column into a more useful format. It uses [dbt](https://www.getdbt.com/) to define the models that are translated into PostgreSQL tables or views, which makes it easier to query the data in the analytics platform of choice.

#### couch2pg

Expand All @@ -34,9 +34,9 @@ As CHT Sync puts all new data into a PostgreSQL database into a single table tha

A free and open source SQL database used for analytics queries. See more at the [PostgreSQL](https://www.postgresql.org) site.

#### DBT
#### dbt

Once the data is synchronized and stored in PostgreSQL, it undergoes transformation using predefined [DBT](https://www.getdbt.com/) models from the [cht-pipeline](https://github.com/medic/cht-pipeline). DBT is used to ingest raw JSON data from the PosgtreSQL database (`jsonb` column) and normalize it into a relational schema to make it easier to query. A daemon runs CHT Pipeline, and it updates the database whenever the data in the `jsonb` column changes.
Once the data is synchronized and stored in PostgreSQL, it undergoes transformation using predefined [dbt](https://www.getdbt.com/) models from the [cht-pipeline](https://github.com/medic/cht-pipeline). dbt is used to ingest raw JSON data from the PosgtreSQL database (`jsonb` column) and normalize it into a relational schema to make it easier to query. A daemon runs CHT Pipeline, and it updates the database whenever the data in the `jsonb` column changes.

#### Data Visualization

Expand Down

0 comments on commit 2874d33

Please sign in to comment.