Skip to content

Commit 51b592c

Browse files
mfontaninilumasepaPablo Hernándezmanel1874Jmgr
committed
feat: add python client
Co-authored-by: Sergio Medina <[email protected]> Co-authored-by: Matias Fontanini <[email protected]> Co-authored-by: Pablo Hernández <[email protected]> Co-authored-by: manel1874 <[email protected]> Co-authored-by: Jmgr <[email protected]> Co-authored-by: Andreas Abros <[email protected]> Co-authored-by: Jeff Wolski <[email protected]>
0 parents  commit 51b592c

File tree

130 files changed

+14071
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

130 files changed

+14071
-0
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 222 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,222 @@
1+
name: CI
2+
on: [ push ]
3+
defaults:
4+
run:
5+
shell: bash
6+
env:
7+
NILLION_REPO_PATH: ./nilvm
8+
9+
jobs:
10+
start-runner:
11+
name: Start self-hosted EC2 runner
12+
runs-on: ubuntu-latest
13+
outputs:
14+
label: ${{ steps.start-ec2-runner.outputs.label }}
15+
ec2-instances-ids: ${{ steps.start-ec2-runner.outputs.ec2-instances-ids }}
16+
steps:
17+
- name: Configure AWS credentials
18+
uses: aws-actions/[email protected]
19+
with:
20+
aws-access-key-id: ${{ secrets.GH_AWS_ACCESS_KEY }}
21+
aws-secret-access-key: ${{ secrets.GH_AWS_SECRET_KEY }}
22+
aws-region: "eu-west-1"
23+
- name: Start EC2 runner
24+
id: start-ec2-runner
25+
uses: NillionNetwork/[email protected]
26+
with:
27+
mode: start
28+
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN_RUNNER }}
29+
runners-per-machine: 1
30+
number-of-machines: 1
31+
ec2-image-id: ami-085ff9583e27b17b0
32+
ec2-instance-type: c6in.2xlarge
33+
subnet-id: subnet-0ec4c353621eabae2
34+
security-group-id: sg-03ee5c56e1f467aa0
35+
key-name: production-github-runner-key
36+
iam-role-name: github-runners-production-github-runner-ec2
37+
aws-resource-tags: >
38+
[
39+
{"Key": "Name", "Value": "github-runner-${{ github.run_id }}-${{ github.run_number }}"},
40+
{"Key": "GitHubRepository", "Value": "${{ github.repository }}"},
41+
{"Key": "KeyName", "Value": "github-runners-key"},
42+
{"Key": "Deployment", "Value": "github-runners"},
43+
{"Key": "Type", "Value": "GithubRunner"},
44+
{"Key": "User", "Value": "ec2-user"},
45+
{"Key": "Environment", "Value": "production"}
46+
]
47+
ci:
48+
name: CI
49+
runs-on: ${{ needs.start-runner.outputs.label }}
50+
needs: [ start-runner ]
51+
container: public.ecr.aws/x8g8t2h7/python-client-builder:0.1.0
52+
steps:
53+
- uses: actions/checkout@v4
54+
with:
55+
submodules: 'recursive'
56+
57+
# TODO uv is installed in container but not found
58+
- name: Install uv
59+
uses: astral-sh/setup-uv@v3
60+
with:
61+
version: "0.4.24"
62+
63+
- name: Set up Python
64+
run: uv python install
65+
66+
# TODO rustup is installed in container but not found
67+
- name: Install rustup
68+
uses: dtolnay/rust-toolchain@master
69+
with:
70+
toolchain: nightly-2024-07-01
71+
72+
- name: Install the project
73+
run: uv sync --all-extras --dev --prerelease=allow
74+
75+
- name: Check format python
76+
run: just python-format-check
77+
78+
- name: Check format rust
79+
run: just rust-format-check
80+
81+
- name: Check Typing Python
82+
run: just python-type-check
83+
84+
- name: Cargo Clippy
85+
run: just clippy
86+
87+
- name: Client Core Tests
88+
run: just test-client-core
89+
90+
- name: Get SDK Version
91+
id: get-sdk-version
92+
run: |
93+
VERSION=$(cat .nil-sdk.toml | sed -e "s|.*\"\(.*\)\".*|\1|g")
94+
echo "version=$VERSION" > $GITHUB_OUTPUT
95+
96+
- name: Install nillion SDK
97+
uses: NillionNetwork/nillion-setup-action@main
98+
with:
99+
version: ${{ steps.get-sdk-version.outputs.version }}
100+
101+
- name: Client Tests
102+
run: just test-client run-devnet
103+
104+
- name: Tests Report
105+
uses: mikepenz/[email protected]
106+
if: ${{ ! cancelled() }}
107+
with:
108+
report_paths: "./*.junit.xml"
109+
check_name: "Tests Report"
110+
111+
- name: Package
112+
run: just package
113+
114+
- name: Store nillion-client-core distribution packages
115+
uses: actions/upload-artifact@v4
116+
if: github.ref == 'refs/heads/main'
117+
with:
118+
name: nillion-client-core-distributions
119+
path: client-core/dist/*.whl
120+
121+
- name: Store nillion-client-proto distribution packages
122+
uses: actions/upload-artifact@v4
123+
if: github.ref == 'refs/heads/main'
124+
with:
125+
name: nillion-client-proto-distributions
126+
path: client-proto/dist/*.whl
127+
128+
- name: Store nillion-client distribution packages
129+
uses: actions/upload-artifact@v4
130+
if: github.ref == 'refs/heads/main'
131+
with:
132+
name: nillion-client-distributions
133+
path: dist/*.whl
134+
135+
publish:
136+
name: Publish
137+
runs-on: ubuntu-latest
138+
needs: [ ci ]
139+
if: github.ref == 'refs/heads/main'
140+
strategy:
141+
matrix:
142+
pkg:
143+
- name: nillion-client-core
144+
pyproject-path: "./client-core/pyproject.toml"
145+
package-name: "nillion_client_core"
146+
token-secret-name: PYPI_NILLION_CLIENT_CORE_TOKEN
147+
- name: nillion-client-proto
148+
pyproject-path: "./client-proto/pyproject.toml"
149+
package-name: "nillion_client_proto"
150+
token-secret-name: PYPI_NILLION_CLIENT_PROTO_TOKEN
151+
- name: nillion-client
152+
pyproject-path: "./pyproject.toml"
153+
package-name: "nillion-client"
154+
token-secret-name: PYPI_NILLION_CLIENT_TOKEN
155+
steps:
156+
- uses: actions/checkout@v4
157+
158+
- name: Install uv
159+
uses: astral-sh/setup-uv@v3
160+
with:
161+
version: "0.4.24"
162+
163+
- name: Set up Python
164+
run: uv python install
165+
166+
- name: Check version
167+
id: versioncheck
168+
run: >-
169+
cd scripts/version_checker && uv run ./version_checker.py ../../${{ matrix.pkg.pyproject-path }}
170+
171+
- name: Download dists
172+
if: steps.versioncheck.outputs.local_version_is_higher == 'true'
173+
uses: actions/download-artifact@v4
174+
with:
175+
name: ${{ matrix.pkg.name }}-distributions
176+
path: dist/${{ matrix.pkg.name }}/
177+
178+
- name: Publish package distributions to PyPI
179+
if: steps.versioncheck.outputs.local_version_is_higher == 'true'
180+
uses: pypa/gh-action-pypi-publish@release/v1
181+
with:
182+
password: ${{ secrets[matrix.pkg.token-secret-name] }}
183+
packages-dir: dist/${{ matrix.pkg.name }}/
184+
185+
- name: Create GH Release
186+
id: create_release
187+
if: steps.versioncheck.outputs.local_version_is_higher == 'true'
188+
env:
189+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
190+
run: |
191+
[[ "${{ steps.versioncheck.outputs.local_version }}" == *"rc"* ]] && PRERELEASE="-p" || PRERELEASE=""
192+
gh release create '${{ matrix.pkg.name }}-v${{ steps.versioncheck.outputs.local_version }}' --generate-notes $PRERELEASE --title "${{ matrix.pkg.name }}-v${{ steps.versioncheck.outputs.local_version }}"
193+
194+
- name: Upload artifact to GH Release
195+
if: steps.versioncheck.outputs.local_version_is_higher == 'true'
196+
env:
197+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
198+
run: >-
199+
gh release upload '${{ matrix.pkg.name }}-v${{ steps.versioncheck.outputs.local_version }}' dist/${{ matrix.pkg.name }}/**
200+
201+
stop-runner:
202+
name: Stop self-hosted EC2 runner
203+
needs: [ start-runner, ci, publish ]
204+
runs-on: ubuntu-latest
205+
if: ${{ always() }}
206+
steps:
207+
- name: Configure AWS credentials
208+
uses: aws-actions/configure-aws-credentials@v1
209+
with:
210+
aws-access-key-id: ${{ secrets.GH_AWS_ACCESS_KEY }}
211+
aws-secret-access-key: ${{ secrets.GH_AWS_SECRET_KEY }}
212+
aws-region: "eu-west-1"
213+
214+
- name: Stop EC2 runner
215+
uses: NillionNetwork/[email protected]
216+
with:
217+
mode: stop
218+
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN_RUNNER }}
219+
label: ${{ needs.start-runner.outputs.label }}
220+
ec2-instances-ids: ${{ needs.start-runner.outputs.ec2-instances-ids }}
221+
222+

‎.gitignore‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
/.env
2+
target/
3+
*.egg-info
4+
*.pyc
5+
.idea
6+
*.log
7+
*.junit.xml
8+
dist/
9+
.intentionally-empty-file.o
10+
build/

‎.gitmodules‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
[submodule "nilvm"]
2+
path = nilvm
3+
url = [email protected]:NillionNetwork/nilvm.git

‎.nil-sdk.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
version = "v0.9.0-rc.62"

‎.python-version‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
3.10

‎README.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# Python Nillion client
2+
3+
Nillion-client is a Python client for building on top of the Nillion Network.
4+
It can be used to manage Nada programs, store and retrieve values, and run
5+
computations.
6+
7+
See the official Nillion documentation site for more about [Nillion
8+
Clients][clients], [Python Client Examples][examples] and the [Python Client
9+
Reference][reference].
10+
11+
[clients]: https://docs.nillion.com/nillion-client
12+
13+
[examples]: https://docs.nillion.com/python-client-examples
14+
15+
[reference]: https://docs.nillion.com/python-client-reference
16+
17+
## How to develop
18+
19+
This project uses [uv](https://docs.astral.sh/uv/) to manage itself.
20+
Be sure that you have it installed.
21+
22+
23+
To install dependencies and setup project
24+
```shell
25+
uv sync
26+
```
27+
28+
### Run tests
29+
First ensure that you have the nillion-devnet running
30+
```shell
31+
./tests/resources/scripts/run_devnet.sh
32+
```
33+
34+
Then in a new terminal
35+
```
36+
uv run pytest
37+
```
38+
39+
### Format code and Linting
40+
This project uses [ruff](https://docs.astral.sh/ruff/) as linter and formatter, it gets installed as a dependency.
41+
42+
To format code:
43+
```shell
44+
uv run ruff format
45+
```
46+
47+
To lint:
48+
```shell
49+
uv run ruff check
50+
```
51+
52+
## Generating docs
53+
54+
In order to generate the documentation first install the dependencies and set up the project:
55+
56+
57+
```shell
58+
uv sync
59+
```
60+
61+
Then activate the virtual environment:
62+
63+
```shell
64+
source .venv/bin/activate
65+
66+
# Use .venv/bin/activate.fish if you're using fish shell
67+
```
68+
69+
And finally run:
70+
71+
```shell
72+
./docs/generate.sh output_directory
73+
```
74+
75+
The docs will be generated in `output_directory`.
76+
77+
78+
## Release Process
79+
80+
### Release Candidates
81+
Release candidates are published on every merge to the `main` branch
82+
83+
### Stable Releases
84+
85+
To release a new version of the client non rc, follow these steps:
86+
87+
run:
88+
89+
```shell
90+
just release
91+
```
92+
93+
and bump the version in the `pyproject.toml` file for the new rc.

‎README.pypi.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# nillion-client
2+
3+
nillion-client is a Python client for building on top of the Nillion Network.
4+
It can be used to manage Nada programs, store and retrieve values, and run
5+
computations.
6+
7+
See the official Nillion documentation site for more about [Nillion
8+
Clients][clients], [Python Client Examples][examples] and the [Python Client
9+
Reference][reference].
10+
11+
[clients]: https://docs.nillion.com/nillion-client
12+
13+
[examples]: https://docs.nillion.com/python-client-examples
14+
15+
[reference]: https://docs.nillion.com/python-client-reference
16+

‎builder.dockerfile‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
FROM debian:12-slim
2+
3+
ENV SELF_VERSION=0.1.0
4+
5+
RUN apt update && \
6+
apt install -y --no-install-recommends cmake git ca-certificates curl build-essential
7+
8+
RUN mkdir /opt/macos-sdk && \
9+
git clone https://github.com/hexops-graveyard/sdk-macos-12.0.git /opt/macos-sdk/12.0
10+
11+
ENV SDKROOT=/opt/macos-sdk/12.0/root
12+
13+
RUN curl -LsSf https://astral.sh/uv/0.4.24/install.sh | sh
14+
15+
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
16+
17+
ENV PATH=/root/.cargo/bin:$PATH
18+
19+
RUN cargo install just

0 commit comments

Comments
 (0)