Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/initial dev #1

Merged
merged 143 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
a2e1d52
feat(endpoint): add initial features
RobertRosca Jul 26, 2023
9ccbc76
feat(endpoint): allow for image uploads
RobertRosca Jul 26, 2023
0c7bca4
feat(endpoint): add very rudimentary authentication
RobertRosca Jul 26, 2023
93749d1
chore: add docstrings, add cli as script
RobertRosca Jul 26, 2023
73787f9
style: sort imports
RobertRosca Jul 26, 2023
941b1b0
refactor: rename classes/methods to maybe make more sense
RobertRosca Jul 26, 2023
cefd962
feat: Zulip client setup in service, add message/upload methods to model
RobertRosca Jul 26, 2023
c9282c4
refactor: more renaming, proposal to client, token to key
RobertRosca Jul 26, 2023
d397b88
fix(endpoint): use proper header auth
RobertRosca Jul 26, 2023
1b12f77
feat(cli): allow specifying client stream/topic in CLI
RobertRosca Jul 26, 2023
c737986
build(docker): add Dockerfile
RobertRosca Jul 26, 2023
eea3516
feat: store clients/zuliprc in cwd for easy config/start
RobertRosca Jul 26, 2023
6fcab7c
chore(docker): use default port 8000
RobertRosca Jul 26, 2023
355631b
feat: better exception handling
RobertRosca Jul 26, 2023
4476091
docs: add readme
RobertRosca Jul 26, 2023
8b3473d
chore: update Zulip client injection
RobertRosca Jul 26, 2023
b79d2eb
fix: newline before image URI insertion
RobertRosca Jul 26, 2023
fde64f2
feat(endpoint): add upload image endpoint
RobertRosca Jul 26, 2023
df81dc1
chore(docker): use buildkit caching
RobertRosca Jul 26, 2023
f2ef5e3
chore(docker): add compose file for easier pull/up/down
RobertRosca Jul 26, 2023
47b719d
build(deps): lower Python version to ^3.9
RobertRosca Jul 26, 2023
a12c2f1
chore(deps): make compatible with Python 3.9
RobertRosca Jul 26, 2023
63a42a1
chore(docker): use .env to set port
RobertRosca Jul 26, 2023
5bcc1c8
docs: reference Zulip docs, drop response validation
RobertRosca Jul 26, 2023
8393659
feat(endpoint): allow topic to be set by API call
RobertRosca Aug 2, 2023
3a13c0c
feat: add Admin client, use locking for client.json writes
RobertRosca Aug 2, 2023
92c3d3e
style: more consistent imports
RobertRosca Aug 2, 2023
1046ae6
docs: update usage documentation
RobertRosca Aug 2, 2023
8fa741c
fix: oversight on admin check
RobertRosca Aug 2, 2023
58d6f18
feat(endpoint): add list topics endpoint
RobertRosca Aug 2, 2023
6293878
refactor: rename to plural
RobertRosca Aug 2, 2023
9ab1e34
chore(deps): use pydantic 2.0 field validator decorator
RobertRosca Aug 2, 2023
86f94db
style: formatting improvements
RobertRosca Aug 2, 2023
caf7a3f
build(add): add pytest
RobertRosca Aug 2, 2023
9eeca36
fix: store files in config directory to deal with docker mount
RobertRosca Aug 2, 2023
fbd8a03
refactor: remove redundant response model
RobertRosca Aug 2, 2023
7f55125
fix: update response for invalid key
RobertRosca Aug 2, 2023
51a24e3
fix: deal with mix of client/admins in repo list
RobertRosca Aug 2, 2023
24bedd3
refactor: use factory to return same object instead of default
RobertRosca Aug 2, 2023
e20d814
test: add initial tests
RobertRosca Aug 2, 2023
545653c
feat: add bot to stream when client is created
RobertRosca Aug 2, 2023
a57a861
feat: use poe task to serve
RobertRosca Aug 2, 2023
8e72e3d
test: update tests for default factory ScopedClient._client
RobertRosca Aug 3, 2023
d0fe6ee
feat(endpoint): add `/me` endpoint to get details on what stream you use
RobertRosca Aug 3, 2023
68d7e81
feat(model): change key field to SecretStr
RobertRosca Aug 3, 2023
4b9566f
test: add more tests
RobertRosca Aug 3, 2023
00e602b
build(add): add ruff for linting/analysis
RobertRosca Aug 3, 2023
da59a58
build(add): add pyright, update lint command
RobertRosca Aug 3, 2023
8fae04a
refactor: use FastAPI Annotated style instead of kwarg style
RobertRosca Aug 3, 2023
2b15cc3
fix: lint/type issues
RobertRosca Aug 3, 2023
3eebbe3
build: switch to pytest-cov
RobertRosca Aug 3, 2023
644e4ca
fix: incorrect annotation
RobertRosca Aug 3, 2023
d54a4dd
build: split up lint tasks
RobertRosca Aug 3, 2023
932a3ce
ci(test): create tests.yml
RobertRosca Aug 3, 2023
099422a
ci(lint): create lint.yml
RobertRosca Aug 3, 2023
f57b511
ci(style): consistent action names
RobertRosca Aug 3, 2023
8c6bbc1
fix: test issue when no zuliprc file present
RobertRosca Aug 3, 2023
868631f
refactor: import module not funcs, add annotations
RobertRosca Aug 3, 2023
4d63088
style: fix formatting
RobertRosca Aug 3, 2023
e311214
ci(test): upload test coverage
RobertRosca Aug 3, 2023
435729d
ci(docker): add ci to build and push docker image
RobertRosca Aug 4, 2023
489f2c7
chore(deploy): configure auto update and load balancer
RobertRosca Aug 4, 2023
c3d943d
chore(deploy): remove watchtower, use compose 'update_config'
RobertRosca Aug 4, 2023
78e2221
feat(endpoint): add health endpoint
RobertRosca Aug 4, 2023
398c054
chore(deploy): change docker compose update config
RobertRosca Aug 4, 2023
cb30aa4
docs: add deployment section
RobertRosca Aug 4, 2023
1d5c56e
chore(deploy): add nginx healthcheck
RobertRosca Aug 4, 2023
aa7e345
chore(deploy): pin nginx version
RobertRosca Aug 4, 2023
681b370
ci(docker): do not build image on pr trigger
RobertRosca Aug 4, 2023
26fea8b
build(add): add commitizen
RobertRosca Aug 4, 2023
1d2af5f
chore: add black task, add format task, set lint to check only
RobertRosca Aug 7, 2023
18d79dd
ci: continue on error for lint checks
RobertRosca Aug 7, 2023
8397b0d
build(deps): add black to lint group
RobertRosca Aug 7, 2023
0c4e5ec
ci: replace continue on error with if always
RobertRosca Aug 7, 2023
79d9fce
ci: rename lint black step, add check flag
RobertRosca Aug 7, 2023
38c4196
style: set line length to 88
RobertRosca Aug 7, 2023
5c6f76a
style: reformat
RobertRosca Aug 7, 2023
1077c70
feat(endpoint): deprecate `upload_image`, rename to `upload_file`
RobertRosca Aug 10, 2023
bd9d23f
feat(endpoint): remove `upload_image`
RobertRosca Aug 10, 2023
18c8bc9
build: add makefile for easier up/down with stacks
RobertRosca Aug 10, 2023
f06c5e0
docs(deploy): update documentation w/ make commands
RobertRosca Aug 10, 2023
c53ce7d
test: replace some references to image with generic file
RobertRosca Aug 10, 2023
5be14e2
build(deps): update pyright
RobertRosca Aug 10, 2023
6437abe
ci: create reusable composite action for project setup
RobertRosca Aug 10, 2023
c3e5d89
ci: use reusable actions for lint and test
RobertRosca Aug 10, 2023
3d87c3e
ci: do not build on all branches, set workflow call/dispatch
RobertRosca Aug 10, 2023
f8309bb
ci: build image after successful tests
RobertRosca Aug 10, 2023
b3db128
feat(endpoint): rename `/message` to `/send_message`, deprecate `/mes…
RobertRosca Aug 14, 2023
9dd2a41
feat(endpoint): add `/update_message` endpoint and model methods
RobertRosca Aug 14, 2023
d66c6e6
fix(deploy): add `git pull` to make up, fix compose typo
RobertRosca Aug 14, 2023
1f26a0e
fix(deploy): remove quotes around nginx port
RobertRosca Aug 14, 2023
224c06b
chore(deploy): erorr on missing port env var
RobertRosca Aug 14, 2023
fb6a902
build(deploy): parse compose config with sed before pipe to stack deploy
RobertRosca Aug 14, 2023
9a12078
ci(deploy): set push to true
RobertRosca Aug 14, 2023
f74ae96
fix(deploy): set head sha on pr event
RobertRosca Aug 14, 2023
ca6995e
fix(deploy): use pr number instead of branch name
RobertRosca Aug 14, 2023
1fb4246
chore(deploy): update image url to pull from european-xfel
RobertRosca Aug 15, 2023
231d58e
chore(deploy): remove `replicas: 2` since `order: start-first` achiev…
RobertRosca Aug 20, 2023
619a9eb
feat(endpoint): remove deprecated '/message' endpoint
RobertRosca Sep 20, 2023
9743052
feat(endpoint): make message contents a body parameter
RobertRosca Sep 20, 2023
c66b2a8
test: update tests after removing deprecated endpoint
RobertRosca Sep 20, 2023
4e0da10
feat: remane `list_topics` to `get_stream_topics`
RobertRosca Sep 22, 2023
78848dd
feat: `get_stream_topics` return zulip response instead of custom error
RobertRosca Sep 22, 2023
195d606
docs: document poe commands
RobertRosca Sep 22, 2023
19a3f64
docs: simplify readme, update endpoints
RobertRosca Oct 6, 2023
67ed257
build: bump pyright
RobertRosca Oct 6, 2023
eed510f
fix(cli): run services setup before creating client
RobertRosca Oct 10, 2023
746b123
fix(models): do not log stream as `extra`
RobertRosca Oct 10, 2023
271ae8d
fix: rename `get_topics` to `get_stream_topics`
RobertRosca Oct 12, 2023
0345ddc
feat(endpoint): make topic/content conditionally required for update …
RobertRosca Oct 16, 2023
da05178
test: exclude cli module from coverage
RobertRosca Oct 16, 2023
702c8b3
build(deps): update pyright
RobertRosca Oct 16, 2023
8fd34b1
fix: add `from __future__ import annotations`
RobertRosca Oct 16, 2023
add5557
test: remove python 3.9 from test matrix
RobertRosca Oct 27, 2023
d5ca16b
build: bump python, fastapi, pydantic, ruff
RobertRosca Oct 27, 2023
d34b549
test(endpoint): add tests for `/me` endpoint
RobertRosca Oct 27, 2023
7372aa1
fix(endpoint): fix `/me` endpoint response type missmatch
RobertRosca Oct 27, 2023
4807824
fix(endpoint): update mypy ignore categories
RobertRosca Oct 27, 2023
2679005
feat(endpoint): explicitly exclude `Client.key` from `/me` response
RobertRosca Oct 27, 2023
a75dc1c
feat(endpoint): tag `/health` as `Admin`
RobertRosca Oct 27, 2023
fb84c37
fix(endpoint): return client key on `/create_client` call
RobertRosca Oct 27, 2023
7050660
test(endpoint): test that `/create_client` returns exposed secret
RobertRosca Oct 27, 2023
aa60883
ci: bump python version for lint workflow to 3.11
RobertRosca Oct 27, 2023
cdf7a67
build: bump max python version to 3.13
RobertRosca Oct 27, 2023
cef3faa
build(deploy): use port 8080 as standard
RobertRosca Nov 7, 2023
8c12aa7
build(deploy): allow setting image and tag via env vars
RobertRosca Nov 7, 2023
3bda45b
build(dev): add 'dev-docker' target to Makefile
RobertRosca Nov 7, 2023
cfdd9ae
build: use poetry-dynamic-versioning
RobertRosca Nov 7, 2023
632612d
feat(endpoint): include version info in health response
RobertRosca Nov 7, 2023
e8e1303
feat(logging): exclude '/health' endpoint calls from logs
RobertRosca Nov 7, 2023
18d2fc0
docs: add todo section
RobertRosca Nov 7, 2023
008fbcd
docs: add client creation section
RobertRosca Nov 7, 2023
5e985a9
docs: update deployment section
RobertRosca Nov 7, 2023
fc72e4c
docs: update development section
RobertRosca Nov 7, 2023
9dbbcee
docs: replace exfldadev01 with localhost
RobertRosca Nov 7, 2023
7cdd57b
style: format, fix mypy error
RobertRosca Nov 7, 2023
6f8e069
build: update lockfile
RobertRosca Nov 7, 2023
3eb9a13
docs: add dependabot to todo list
RobertRosca Nov 7, 2023
5353274
docs: add debugging section
RobertRosca Nov 7, 2023
9cfd9a2
ci: add dynamic versioning plugin during setup
RobertRosca Nov 13, 2023
42aed39
build(deploy): expose port 8080 in dockerfile
RobertRosca Nov 13, 2023
bb837a0
fix(endpoint): import version info from module __init__ instead of fr…
RobertRosca Nov 14, 2023
ec09a7c
docs: add build-image workflow test and client bot config to todo list
RobertRosca Nov 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/setup/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Setup Step

description: |
This step sets up Python and Poetry, and caches the virtual environment.
It is intended to be used as a step in a workflow that runs Python code.

inputs:
python-version:
required: true
description: "Python version to use with `actions/setup-python`"

runs:
using: "composite"
steps:
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ inputs.python-version }}

- name: Set up Poetry
shell: bash
run: |
python -m pip install --upgrade pip poetry
poetry self add "poetry-dynamic-versioning[plugin]"
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local

- name: Use venv cache
uses: actions/cache@v3
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}-${{ inputs.python-version }}

- name: Install project
shell: bash
run: |
poetry --no-ansi --no-interaction install
56 changes: 56 additions & 0 deletions .github/workflows/build-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Build Image

on:
push:
tags:
- "v**"
workflow_call:
workflow_dispatch:

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}

jobs:
build-and-push-image:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write

steps:
- name: Checkout repository
uses: actions/checkout@v3

- name: Log in to the Container registry
uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=sha

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build and push Docker image
uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
with:
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
40 changes: 40 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Lint

on:
push:
branches: [main]
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
lint:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11"]
steps:
- name: Checkout project
uses: actions/checkout@v3

- uses: ./.github/setup
with:
python-version: ${{ matrix.python-version }}

- name: Lint - ruff
if: always()
run: poetry run poe ruff

- name: Lint - black
if: always()
run: poetry run poe black --check

- name: Lint - mypy
if: always()
run: poetry run poe mypy

- name: Lint - pyright
if: always()
run: poetry run poe pyright
40 changes: 40 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: CI

on:
push:
branches: [main]
pull_request:

concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11", "3.12-dev"]
steps:
- name: Checkout project
uses: actions/checkout@v3

- uses: ./.github/setup
with:
python-version: ${{ matrix.python-version }}

- name: Test
run: poetry run poe test

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

build-image:
needs: test
uses: ./.github/workflows/build-image.yml
permissions:
contents: read
packages: write
secrets: inherit
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
.env
zuliprc
clients.json
src/zulip_write_only_proxy/_version.py

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
repos:
- hooks:
- id: commitizen
- id: commitizen-branch
stages:
- push
repo: https://github.com/commitizen-tools/commitizen
rev: v2.42.1
20 changes: 20 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
FROM python:3.11

WORKDIR /app

RUN --mount=type=cache,target=/root/.cache \
python3 -m pip install --upgrade poetry pip

COPY ./poetry.lock ./pyproject.toml ./README.md /app
COPY ./src /app/src

RUN poetry config virtualenvs.create false --local

RUN --mount=type=cache,target=/root/.cache\
poetry install

CMD ["uvicorn", "zulip_write_only_proxy.main:app", "--host", "0.0.0.0", "--port", "8080"]

EXPOSE 8080

HEALTHCHECK CMD curl --fail http://localhost:8080/health || exit 1
11 changes: 11 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
up:
git pull
# See: https://github.com/docker/cli/issues/1073
docker compose config | sed '/published:/ s/"//g' | sed "/name:/d" | docker stack deploy -c - zwop

down:
docker stack rm zwop

dev-docker:
docker build . --tag zwop:dev
docker run -it --rm -v $(PWD):/app -p 8080:8080 zwop:dev
Loading