Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
ed1a3b0
Update alcf-ai
masalim2 Jun 2, 2026
333125c
alcf-ai 0.5.0
masalim2 Jun 2, 2026
710f161
WIP
masalim2 Jun 3, 2026
404753c
Reorg to modern uv workspace file layout
masalim2 Jun 5, 2026
f2cd8a1
WIP
masalim2 Jun 5, 2026
46ad16d
update
masalim2 Jun 8, 2026
7a0c66c
Flatten layout
masalim2 Jun 15, 2026
4012168
WIP
masalim2 Jun 16, 2026
978d215
WIP
masalim2 Jun 16, 2026
cf32801
WIP
masalim2 Jun 16, 2026
ff3ca3a
WIP
masalim2 Jun 16, 2026
41c3f6e
WIP
masalim2 Jun 17, 2026
2e19d38
WIP
masalim2 Jun 17, 2026
5a39aac
WIP
masalim2 Jun 18, 2026
4d67163
WIP
masalim2 Jun 18, 2026
0bdd39a
WIP
masalim2 Jun 18, 2026
f76fa6f
WIP
masalim2 Jun 18, 2026
000093f
WIP
masalim2 Jun 18, 2026
92fa8cd
WIP
masalim2 Jun 18, 2026
afe6641
WIP
masalim2 Jun 18, 2026
0e42a7c
WIP
masalim2 Jun 18, 2026
3bc9572
WIP
masalim2 Jun 18, 2026
d043b90
WIP
masalim2 Jun 18, 2026
8744e4c
WIP
masalim2 Jun 18, 2026
520aea0
Update CI
masalim2 Jun 19, 2026
2def933
Add workflow_dispatch to CI
masalim2 Jun 19, 2026
f9323c8
Add PKI tool
masalim2 Jun 19, 2026
b7488d2
WIP: start nginx process manager
masalim2 Jun 19, 2026
a5c6d56
WIP
masalim2 Jun 20, 2026
5978533
Pilot job implementation
masalim2 Jun 22, 2026
a67afcc
WIP
masalim2 Jun 23, 2026
fd83986
Add scheduler interface
masalim2 Jun 23, 2026
434f26b
update CI
masalim2 Jun 23, 2026
dd925d4
Update certmanager
masalim2 Jun 23, 2026
279a82f
Rename
masalim2 Jun 23, 2026
16d678f
Update schemas
masalim2 Jun 23, 2026
e1f7324
Add PilotSubmitter
masalim2 Jun 23, 2026
f3923e8
WIP
masalim2 Jun 23, 2026
fea6449
Update tests
masalim2 Jun 23, 2026
2036303
WIP
masalim2 Jun 26, 2026
c26a399
WIP
masalim2 Jun 27, 2026
f621b37
Remove archived/ and update docs
masalim2 Jun 27, 2026
7b3dcd3
Update docs
masalim2 Jun 27, 2026
fb08265
Update docs
masalim2 Jun 28, 2026
654b994
Update docs
masalim2 Jun 28, 2026
e5bd316
Update docs
masalim2 Jun 28, 2026
140a090
WIP
masalim2 Jun 28, 2026
20c0168
WIP
masalim2 Jun 28, 2026
e3e02d4
WIP
masalim2 Jun 28, 2026
6a2c371
WIP
masalim2 Jun 29, 2026
4ab6545
WIP
masalim2 Jun 29, 2026
965a8b3
Add soft delete layer
masalim2 Jun 29, 2026
e110449
Add reconcile backoff and reset
masalim2 Jul 1, 2026
a071de1
Update docs
masalim2 Jul 1, 2026
de0b727
Merge branch 'worktree-resource-backoff' into firstv2
masalim2 Jul 1, 2026
865fe3d
Add .worktreeinclude
masalim2 Jul 1, 2026
50845fe
Add alembic
masalim2 Jul 2, 2026
437307c
Add status store
masalim2 Jul 2, 2026
dcfc603
reorg
masalim2 Jul 2, 2026
61203a7
Add controller
masalim2 Jul 2, 2026
c473119
Add RetentionSweeper
masalim2 Jul 2, 2026
8058902
Add metrics server
masalim2 Jul 2, 2026
5ebd7e0
Add Load observer
masalim2 Jul 2, 2026
b5c8c67
Add live in-flight gauge
masalim2 Jul 2, 2026
d6ba865
Update docs
masalim2 Jul 2, 2026
85f3b62
update docs
masalim2 Jul 8, 2026
c679eab
Update status observers
masalim2 Jul 8, 2026
1d24cdb
Update spec
masalim2 Jul 8, 2026
3b28d5c
Refactoring
masalim2 Jul 8, 2026
dfec15b
Update tests
masalim2 Jul 8, 2026
f96ef08
Adding redis store
masalim2 Jul 8, 2026
64c6c15
Admission controller
masalim2 Jul 9, 2026
390d911
Review/docs on admission controller
masalim2 Jul 9, 2026
bf41454
Updates
masalim2 Jul 10, 2026
96339d7
Rename replica->backend in data plane
masalim2 Jul 10, 2026
4034048
Update redis repo pattern
masalim2 Jul 10, 2026
8fd8b46
Align API paths with control/catalog split
masalim2 Jul 10, 2026
36d1113
Centralize redis access keys/patterns
masalim2 Jul 10, 2026
65e5523
Add RouterConfigObserver
masalim2 Jul 10, 2026
2d74399
update health
masalim2 Jul 11, 2026
8ea93a8
Updates
masalim2 Jul 12, 2026
4dd04de
Updates
masalim2 Jul 12, 2026
3e3aeb7
Add PilotJobObserver
masalim2 Jul 12, 2026
935f652
Add Observers
masalim2 Jul 13, 2026
000ae19
Update AdmissionController
masalim2 Jul 14, 2026
4eaecf5
Update InflightObserver
masalim2 Jul 14, 2026
cfde50f
fix tests
masalim2 Jul 14, 2026
b1e51e9
Updates
masalim2 Jul 14, 2026
968522d
Update controller.md. Mitigate getfqdn() DNS hang. Add assign/unass…
masalim2 Jul 15, 2026
515dd26
Revise PilotDeployment aggregate state logic
masalim2 Jul 15, 2026
9bd1d3f
Update tests
masalim2 Jul 15, 2026
762c3c7
Updates
masalim2 Jul 15, 2026
dc08356
Update docs
masalim2 Jul 16, 2026
b02a97c
Update replica controllers design doc: clarify
masalim2 Jul 17, 2026
78613fb
updates
masalim2 Jul 18, 2026
1acce2d
Migrate to Redis pubsub for events
masalim2 Jul 18, 2026
d40487f
Add ReplicaReconciler
masalim2 Jul 18, 2026
250ab1a
Ensure qsub idempotent
masalim2 Jul 18, 2026
d0a774a
Add ReplicaPlacer
masalim2 Jul 18, 2026
74c7cb3
Update ReplicaStartError
masalim2 Jul 18, 2026
58147c7
Add Replica Launcher
masalim2 Jul 19, 2026
fcc7b45
Add Replica Drainer
masalim2 Jul 19, 2026
aae99c2
Update docs
masalim2 Jul 19, 2026
ae49330
Update docs
masalim2 Jul 19, 2026
d1ee682
Add autoscaler
masalim2 Jul 19, 2026
01b70f4
Updates
masalim2 Jul 20, 2026
f6e6de3
Add Health Alerter
masalim2 Jul 21, 2026
710bb08
Update local dev config
masalim2 Jul 21, 2026
a6d2702
Update HealthAlerter
masalim2 Jul 22, 2026
2c05c20
Add monitor-redis shortcut
masalim2 Jul 22, 2026
b761adf
Add RouterConfig Snapshot Manager
masalim2 Jul 22, 2026
1163cd2
Add Prometheus SD
masalim2 Jul 22, 2026
c900dc6
update nginx config and fix openapi
masalim2 Jul 23, 2026
22b1aa7
Bootstrap admin-console
masalim2 Jul 23, 2026
76dfd17
Update
masalim2 Jul 23, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
14 changes: 14 additions & 0 deletions .env.compose
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# !!! WARNING !!! Do not put secrets in this file.
# It is tracked in git.
# Sensitive variables should go into .env.secret instead.

# Settings specific to services running inside deploy/compose.yaml.
# This mainly serves to set hostnames correctly for compose-managed networks.

POSTGRES_DB=first_gateway_dev
POSTGRES_USER=first_gateway
POSTGRES_PASSWORD=password

FIRST_DB_URL="postgresql+psycopg://first_gateway:password@postgres:5432/first_gateway_dev"
FIRST_REDIS_URL="redis://redis:6379/1"
FIRST_GATEWAY_HEALTH_URL=http://inference-gateway:7000/health
19 changes: 19 additions & 0 deletions .env.default
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# !!! WARNING !!! Do not put secrets in this file.
# It is tracked in git.
# Sensitive variables should go into .env.secret instead.

# Baseline settings: common among local dev / compose stack / production

# Auth
FIRST_GLOBUS__POLICIES=["f7b3f89c-d8d2-453d-9fc7-3576bc27c421"]
FIRST_GLOBUS__USER_GROUPS=[] # Not restricted by group membership
FIRST_GLOBUS__ADMIN_GROUP="1e943dcd-32ae-11f1-b41c-0e1cdb0e3035"
FIRST_GLOBUS__AUTHORIZED_IDP_DOMAINS=["anl.gov", "alcf.anl.gov"]
FIRST_GLOBUS__AUTHORIZED_GROUPS_PER_IDP='
{
"alcf.anl.gov": [
"fa12c8aa-9352-11f0-a103-0affe2f6e001",
"75fc8c22-9f3d-11f0-a6ad-0afff51aa6ef"
]
}
'
8 changes: 8 additions & 0 deletions .env.local
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# !!! WARNING !!! Do not put secrets in this file.
# It is tracked in git.
# Sensitive variables should go into .env.secret instead.

# Settings specific to local testing outside of the Docker Compose stack.
# This mainly serves to set hostnames correctly for localhost.
FIRST_DB_URL="postgresql+psycopg://first_gateway:password@localhost:5432/first_gateway_dev"
FIRST_REDIS_URL="redis://localhost:6379/1"
5 changes: 5 additions & 0 deletions .env.prod
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# !!! WARNING !!! Do not put secrets in this file.
# It is tracked in git.
# Sensitive variables should go into .env.secret instead.

# Non-secret settings specific to production deployment.
97 changes: 97 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
name: CI

on:
push:
branches: [main]
pull_request:
workflow_dispatch:

concurrency:
group: ci-${{ github.ref }}
cancel-in-progress: true

jobs:
checks:
name: Checks & Tests
runs-on: ubuntu-latest

services:
postgres:
image: postgres:18
env:
POSTGRES_DB: first_gateway_dev
POSTGRES_USER: first_gateway
POSTGRES_PASSWORD: password
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U first_gateway"
--health-interval 5s
--health-timeout 5s
--health-retries 10

redis:
image: redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 3s
--health-retries 10

env:
FIRST_GLOBUS__APP_ID: ci-dummy-app-id
FIRST_GLOBUS__APP_SECRET: ci-dummy-app-secret
FIRST_PILOT_CA_CRT: "certificate"
FIRST_PILOT_CA_KEY: "begin fake private key"

steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v6
with:
python-version-file: ".python-version"

- uses: astral-sh/setup-uv@v7
with:
enable-cache: true

- run: make format-check
- run: make lint
- run: make mypy
- run: make test

docker:
name: Build & Push Image
runs-on: ubuntu-latest
needs: checks
if: github.event_name == 'push' && github.ref == 'refs/heads/main'

permissions:
contents: read
packages: write

steps:
- uses: actions/checkout@v4

- uses: docker/setup-buildx-action@v3

- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Compute short SHA
id: sha
run: echo "short=$(git rev-parse --short HEAD)" >> "$GITHUB_OUTPUT"

- uses: docker/build-push-action@v6
with:
context: .
file: deploy/Dockerfile
push: true
tags: ghcr.io/${{ github.repository_owner }}/first_gateway:${{ steps.sha.outputs.short }}
cache-from: type=gha
cache-to: type=gha,mode=max
60 changes: 0 additions & 60 deletions .github/workflows/deploy-docs.yml

This file was deleted.

97 changes: 0 additions & 97 deletions .github/workflows/django_tests.yml

This file was deleted.

42 changes: 0 additions & 42 deletions .github/workflows/oci-builds.yml

This file was deleted.

4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
__pycache__/
.DS_Store
.env
.env.secret
docs/images/.$Diagrams.drawio.bkp
update_batch_status_error.log
update_batch_status_output.log
create_pg_backup_log.log
Expand All @@ -13,7 +15,7 @@ email.out
.vscode/
.editorconfig

# Django
# Django
info.log
db.sqlite3
media/
Expand Down
2 changes: 2 additions & 0 deletions .worktreeinclude
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.env
.env.secret
Loading