Skip to content

Commit fd5e5ee

Browse files
authored
docs: update documentation (#210)
* docs: add configuartion page * docs: update installation page * docs: add cli commands in tour pages * docs: update guides with cli commands * docs: add cli reference, update introduction page * doc: update references * chore: update dockerfile with server commands * chore: update server commands in examples
1 parent 4524891 commit fd5e5ee

File tree

15 files changed

+1709
-290
lines changed

15 files changed

+1709
-290
lines changed

Dockerfile.dev

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,4 @@ RUN apk add ca-certificates
1414
# don't work well with musl libc that alpine uses
1515
RUN apk add libc6-compat
1616

17-
CMD ["compass", "serve"]
17+
CMD ["compass", "server", "start"]

README.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -166,27 +166,27 @@ docker build . -t compass
166166
Before serving Compass app, we need to run the migration first. Run this docker command to migrate Compass.
167167

168168
```text
169-
$ docker run --rm --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass migrate
169+
$ docker run --rm --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass server migrate
170170
```
171171

172172
If you are using Compass binary, you can run this command.
173173

174174
```text
175-
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> migrate"
175+
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server migrate"
176176
```
177177

178178
## Serving locally
179179

180180
Once the migration has been done, Compass server can be started with this command.
181181

182182
```text
183-
docker run --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass serve
183+
docker run --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass server start
184184
```
185185

186186
If you are using Compass binary, you can run this command.
187187

188188
```text
189-
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> serve"
189+
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> server start"
190190
```
191191

192192
## Running tests

docs/docs/configuration.md

Lines changed: 287 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,287 @@
1+
# Configuration
2+
3+
Compass binary contains both the CLI client and the server. Each has it's own configuration in order to run. Server configuration contains information such as database credentials, elastic search brokers, log severity, etc. while CLI client configuration only has configuration about which server to connect.
4+
## Server Setup
5+
6+
There are several approaches to setup Compass Server
7+
8+
1. [Using the CLI](#using-the-cli)
9+
1. [Using the Docker](#using-the-docker)
10+
2. [Using the Helm Chart](#using-the-helm-chart)
11+
12+
#### General pre-requisites
13+
14+
- PostgreSQL (version 13 or above)
15+
- ElasticSearch v7 (optional)
16+
17+
## Using the CLI
18+
### Using config file
19+
20+
Create a config file with the following command
21+
22+
```sh
23+
$ compass config init
24+
```
25+
26+
alternatively you can [use `--config` flag](#using---config-flag) to customize to config file location.
27+
28+
You can also [use environment variables](#using-environment-variable) to provide the server configuration.
29+
30+
Setup up the Postgres database, and ElasticSearch instance and provide the details as shown in the example below.
31+
32+
> If you're new to YAML and want to learn more, see [Learn YAML in Y minutes.](https://learnxinyminutes.com/docs/yaml/)
33+
34+
Following is a sample server configuration yaml:
35+
36+
```yaml title="compass.yaml"
37+
log_level: info # debug|info|warning|error|fatal|trace|panic - default: info
38+
39+
elasticsearch:
40+
brokers: http://localhost:9200 #required
41+
42+
db:
43+
host: localhost #required
44+
port: 5432 #required
45+
name: compass #required
46+
user: compass #required
47+
password: compass_password #required
48+
sslmode: disable #optional
49+
50+
service:
51+
host: localhost #required
52+
port: 8080 #required
53+
identity:
54+
headerkey_uuid: Compass-User-UUID #required
55+
headerkey_email: Compass-User-Email #optional
56+
provider_default_name: shield #optional
57+
grpc:
58+
port: 8081 #required
59+
max_send_msg_size: 33554432
60+
max_recv_msg_size: 33554432
61+
```
62+
### Using environment variable
63+
64+
All the server configurations can be passed as environment variables using underscore _ as the delimiter between nested keys.
65+
66+
See [configuration reference](./reference/configuration.md) for the list of all the configuration keys.
67+
68+
```sh title=".env"
69+
LOG_LEVEL=info
70+
ELASTICSEARCH_BROKERS=http://localhost:9200
71+
DB_HOST=localhost
72+
DB_PORT=5432
73+
DB_NAME=compass
74+
DB_USER=compass
75+
DB_PASSWORD=compass_password
76+
DB_SSLMODE=disable
77+
SERVICE_HOST=localhost
78+
SERVICE_PORT=8080
79+
SERVICE_IDENTITY_HEADERKEY_UUID=Compass-User-UUID
80+
SERVICE_IDENTITY_HEADERKEY_EMAIL=Compass-User-Email
81+
SERVICE_IDENTITY_PROVIDER_DEFAULT_NAME=shield
82+
SERVICE_GRPC_PORT=8081
83+
SERVICE_GRPC_MAX_SEND_MSG_SIZE=33554432
84+
SERVICE_GRPC_MAX_RECV_MSG_SIZE=33554432
85+
```
86+
87+
Set the env variable using export
88+
```bash
89+
$ export DB_PORT = 5432
90+
```
91+
### Starting the server
92+
93+
Database migration is required during the first server initialization. In addition, re-running the migration command might be needed in a new release to apply the new schema changes (if any). It's safer to always re-run the migration script before deploying/starting a new release.
94+
95+
To initialize the database schema, Run Migrations with the following command:
96+
```bash
97+
$ compass server migrate
98+
```
99+
100+
To run the Compass server use command:
101+
102+
```bash
103+
$ compass server start
104+
```
105+
106+
#### Using `--config` flag
107+
108+
```bash
109+
$ compass server migrate --config=<path-to-file>
110+
```
111+
112+
```bash
113+
$ compass server start --config=<path-to-file>
114+
```
115+
116+
## Using the Docker
117+
To run the Compass server using Docker, you need to have Docker installed on your system. You can find the installation instructions [here](https://docs.docker.com/get-docker/).
118+
119+
You can choose to set the configuration using environment variables or a config file. The environment variables will override the config file.
120+
121+
If you use Docker to build compass, then configuring networking requires extra steps. Following is one of doing it by running postgres and elasticsearch inside with `docker-compose` first.
122+
123+
Go to the root of this project and run `docker-compose`.
124+
125+
```bash
126+
$ docker-compose up
127+
```
128+
Once postgres and elasticsearch has been ready, we can run Compass by passing in the config of postgres and elasticsearch defined in `docker-compose.yaml` file.
129+
130+
### Using config file
131+
Alternatively you can use the `compass.yaml` config file defined [above](#using-config-file) and run the following command.
132+
133+
```bash
134+
$ docker run -d \
135+
--restart=always \
136+
-p 8080:8080 \
137+
-v $(pwd)/compass.yaml:/compass.yaml \
138+
--name compass-server \
139+
odpf/compass:<version> \
140+
server start -c /compass.yaml
141+
```
142+
143+
### Using environment variables
144+
145+
All the configs can be passed as environment variables using underscore `_` as the delimiter between nested keys. See the example as discussed [above](#using-environment-variable)
146+
147+
Run the following command to start the server
148+
149+
```bash
150+
$ docker run -d \
151+
--restart=always \
152+
-p 8080:8080 \
153+
--env-file .env \
154+
--name compass-server \
155+
odpf/compass:<version> \
156+
server start
157+
```
158+
159+
## Using the Helm chart
160+
161+
### Pre-requisites for Helm chart
162+
Compass can be installed in Kubernetes using the Helm chart from https://github.com/odpf/charts.
163+
164+
Ensure that the following requirements are met:
165+
- Kubernetes 1.14+
166+
- Helm version 3.x is [installed](https://helm.sh/docs/intro/install/)
167+
168+
### Add ODPF Helm repository
169+
170+
Add ODPF chart repository to Helm:
171+
172+
```
173+
helm repo add odpf https://odpf.github.io/charts/
174+
```
175+
176+
You can update the chart repository by running:
177+
178+
```
179+
helm repo update
180+
```
181+
182+
### Setup helm values
183+
184+
The following table lists the configurable parameters of the Compass chart and their default values.
185+
186+
See full helm values guide [here](https://github.com/odpf/charts/tree/main/stable/compass#values)
187+
188+
```yaml title="values.yaml"
189+
app:
190+
image:
191+
repository: odpf/compass
192+
pullPolicy: Always
193+
tag: "0.3.0"
194+
container:
195+
command:
196+
- compass
197+
args:
198+
- server start
199+
livenessProbe:
200+
httpGet:
201+
path: /ping
202+
port: tcp
203+
readinessProbe:
204+
httpGet:
205+
path: /ping
206+
port: tcp
207+
208+
migration:
209+
enabled: true
210+
command:
211+
- compass
212+
args:
213+
- server migrate
214+
215+
service:
216+
annotations: {}
217+
218+
ingress:
219+
enabled: true
220+
annotations:
221+
kubernetes.io/ingress.class: contour
222+
hosts:
223+
- host: compass.example.com
224+
paths:
225+
- path: /
226+
pathType: ImplementationSpecific
227+
backend:
228+
service:
229+
# name: backend_01
230+
port:
231+
number: 80
232+
233+
config:
234+
COMPASS_SERVICE_PORT: 8080
235+
COMPASS_SERVICE_GRPC_PORT: 8081
236+
# COMPASS_SERVICE_HOST: 0.0.0.0
237+
# COMPASS_STATSD_ENABLED: false
238+
# COMPASS_STATSD_PREFIX: compass
239+
# COMPASS_NEWRELIC_ENABLED: false
240+
# COMPASS_NEWRELIC_APPNAME: compass
241+
# COMPASS_LOG_LEVEL: info
242+
243+
secretConfig: {}
244+
# COMPASS_ELASTICSEARCH_BROKERS: ~
245+
# COMPASS_STATSD_ADDRESS: ~
246+
# COMPASS_NEWRELIC_LICENSEKEY: ~
247+
# COMPASS_DB_HOST: ~
248+
# COMPASS_DB_PORT: 5432
249+
# COMPASS_DB_NAME: ~
250+
# COMPASS_DB_USER: ~
251+
# COMPASS_DB_PASSWORD: ~
252+
# COMPASS_DB_SSLMODE: disable
253+
```
254+
255+
And install it with the helm command line along with the values file:
256+
257+
```bash
258+
$ helm install my-release -f values.yaml odpf/compass
259+
```
260+
261+
## Client Initialisation
262+
263+
Add client configurations in the same `~/compass.yaml` file in root of current directory. Open this file to configure client.
264+
265+
```yml
266+
client:
267+
host: localhost:8081
268+
serverheaderkey_uuid: Compass-User-UUID
269+
serverheadervalue_uuid: [email protected]
270+
```
271+
272+
#### Required Header/Metadata in API
273+
Compass has a concept of [User](./concepts/user.md). In the current version, all HTTP & gRPC APIs in Compass requires an identity header/metadata in the request. The header key is configurable but the default name is `Compass-User-UUID`.
274+
275+
Compass APIs also expect an additional optional e-mail header. This is also configurable and the default name is `Compass-User-Email`. The purpose of having this optional e-mail header is described in the [User](./concepts/user.md) section.
276+
277+
278+
If everything goes ok, you should see something like this:
279+
```bash
280+
time="2022-04-27T09:18:08Z" level=info msg="compass starting" version=v0.2.0
281+
time="2022-04-27T09:18:08Z" level=info msg="connected to elasticsearch cluster" config="\"docker-cluster\" (server version 7.6.1)"
282+
time="2022-04-27T09:18:08Z" level=info msg="New Relic monitoring is disabled."
283+
time="2022-04-27T09:18:08Z" level=info msg="statsd metrics monitoring is disabled."
284+
time="2022-04-27T09:18:08Z" level=info msg="connected to postgres server" host=postgres port=5432
285+
time="2022-04-27T09:18:08Z" level=info msg="server started"
286+
```
287+

docs/docs/guides/discussion.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
import Tabs from "@theme/Tabs";
2+
import TabItem from "@theme/TabItem";
3+
14
# Discussion
25

36
Discussion is a new feature in Compass. One could create a discussion and all users can put comment in it. Currently, there are three types of discussions `issues`, `open ended`, and `question and answer`. Depending on the type, the discussion could have multiple possible states. In the current version, all types only have two states: `open` and `closed`. A newly created discussion will always be assign an `open` state.
@@ -6,6 +9,23 @@ Discussion is a new feature in Compass. One could create a discussion and all us
69

710
A discussion thread can be created with the Discussion API. The API contract is available [here](https://github.com/odpf/compass/blob/main/third_party/OpenAPI/compass.swagger.json).
811

12+
<Tabs groupId="cli" >
13+
<TabItem value="CLI" label="CLI">
14+
15+
```bash
16+
$ compass discussion post --body=<filepath to discussion body>
17+
```
18+
19+
```json
20+
{
21+
"title": "The first discussion",
22+
"body": "This is the first discussion thread in Compass",
23+
"type": "openended"
24+
}
25+
```
26+
</TabItem>
27+
<TabItem value="HTTP" label="HTTP">
28+
929
```bash
1030
$ curl --request POST 'http://localhost:8080/v1beta1/discussions' \
1131
--header 'Compass-User-UUID:[email protected]' \
@@ -15,15 +35,28 @@ $ curl --request POST 'http://localhost:8080/v1beta1/discussions' \
1535
"type": "openended"
1636
}'
1737
```
38+
</TabItem>
39+
</Tabs>
1840

1941
## Fetching All Discussions
2042

2143
The Get Discussions will fetch all discussions in Compass.
2244

45+
<Tabs groupId="cli" >
46+
<TabItem value="CLI" label="CLI">
47+
48+
```bash
49+
$ compass discussion list
50+
```
51+
</TabItem>
52+
<TabItem value="HTTP" label="HTTP">
53+
2354
```bash
2455
$ curl 'http://localhost:8080/v1beta1/discussions' \
2556
--header 'Compass-User-UUID:[email protected]'
2657
```
58+
</TabItem>
59+
</Tabs>
2760

2861
The response will be something like
2962
```javascript

0 commit comments

Comments
 (0)