Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

API for codecarbon #145

Merged
merged 77 commits into from
May 19, 2021
Merged

API for codecarbon #145

merged 77 commits into from
May 19, 2021
+3,252 −6

Conversation

benoit-cty
Copy link
Contributor

@benoit-cty benoit-cty commented Apr 12, 2021 •
edited
Loading

We build an API for CodeCarbon that allow to store emissions in a database.

The model used is:

  • Emissions : a list of emissions attached to a run (an experiment)
  • Run : a run of the application (a single ML training for example)
    • Experiment : a config of run (hardware/cloud), attached to a project.
      • Project : an application, attached to a team
        • Team : a group of people working on projects, attached to an organization
        • Organization : the top level, for a company or a non-profit.

Main packages used:

  • FastAPI
  • SQLAlchemy
  • Alembic

TODO before merge :

@SaboniAmine SaboniAmine mentioned this pull request Apr 12, 2021
Closed
marc alencon and others added 10 commits April 12, 2021 22:30
Saving Teams in Database and Connect with API
0617e5b
Saving Organization to DB and connection with API
13a4ff0
Add codecarbondb scheme
c539847
@alencon
Update readme
0fd9b5d
@alencon
Api repository (#152)
f290baa 
Adding repository to emissions persistence router
@alencon
Add files via upload
840b0ff
@SaboniAmine @SabAmine @benoit-cty
Testing the API (#149)
c73ae3a 
* refacto: extracting server as standalone package, api test setup (example on one router)

* feat: in memory emissions repository tests (to test the interface)

Co-authored-by: Amine Saboni <asaboni@octo.com>
Co-authored-by: benoit-cty <6603048+benoit-cty@users.noreply.github.com>
@SabAmine
refacto: isolate infra implementation domain objects, experiments / o…
87ec46f 
…rg in memory repo tests
@benoit-cty
fix pre-commit
cb889ba
@benoit-cty
PR160 refacto: isolate infra implementation domain objects,
fcd32d0 
Merge pull request #160 from mlco2/api_refacto_repositories
@benoit-cty
Copy link
Contributor Author

@SaboniAmine Why the database model is in models.py is in the "domain" folder instead of "database" folder ?
And the "schemas.py", representing the Pydantic interface of the API is in "database" folder ?

I've read Octo blog https://blog.octo.com/application-domain-infrastructure-des-mots-de-la-layered-hexagonal-clean-architecture/ ;)

I will put Alembic file in database, is it ok ?

@benoit-cty benoit-cty requested review from Lothiraldan, vict0rsch, LiamConnell and SaboniAmine and removed request for Lothiraldan, LiamConnell and vict0rsch May 2, 2021 15:21
alencon and others added 13 commits May 17, 2021 09:47
@alencon
Update CodecarbonDB model
8db50cc
@benoit-cty
PR#172 run carbonserver with docker-compose
9b65336 
run carbonserver with docker-compose
@benoit-cty
PR168 Many improvements
370ef2c 
Modifications of API Database
Merge pull request #168 from mlco2/api_db
@SabAmine
tests: postman collection + added tests
1d21a0a
@SabAmine
refacto: database/schemas.py -> api/schemas.py. Extract db_to_class as
7024deb 
static method
@SabAmine
wip: migration refacto: Run_id should be stored as string from db
01fc31d 
creation
@SabAmine
pre-commit & typos
ff8c22c
@SabAmine
refacto: isolate unit from integration tests execution, doc
d65e1d8
@SabAmine
refacto: change id types to uuid, alembic migration new initial revision
bc7f649
@SabAmine
ci: test integration
e86640a
@SabAmine
ci: added build_server job, with act patched Dockerfile for local debug
b14b158
@SabAmine
update doc
8d590ea
@SaboniAmine
Merge pull request #174 from mlco2/api_routers_testing
0a89dce 
Api routers testing
@SaboniAmine
Copy link
Collaborator

SaboniAmine commented May 19, 2021 •
edited
Loading

Finally good for me, sorry for the delay.
Tests on routers (postman / tox) still do not pass with a PG database, but those are modelling / logic issues that we still did not address.

@benoit-cty benoit-cty marked this pull request as ready for review May 19, 2021 12:10
@benoit-cty benoit-cty changed the title [WIP] API for codecarbon API for codecarbon May 19, 2021
@benoit-cty benoit-cty merged commit 441fb9f into master May 19, 2021
@benoit-cty benoit-cty deleted the api branch May 19, 2021 15:20
@benoit-cty benoit-cty linked an issue Jul 29, 2021 that may be closed by this pull request
CarbonBoard refactor: Entities #124
Closed
@benoit-cty benoit-cty mentioned this pull request Jul 29, 2021
Closed
@benoit-cty benoit-cty linked an issue Jul 29, 2021 that may be closed by this pull request
CarbonBoard Refactor: Frontend-backend model – REST API for custom frontend extensions #123
Closed
@benoit-cty benoit-cty mentioned this pull request Jul 29, 2021
Closed
This was linked to issues Jul 29, 2021
TODO: setup github actions on the api branch #143
Closed
Better storage of results #183
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vict0rsch vict0rsch

@SaboniAmine SaboniAmine

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
7 participants
@benoit-cty @SaboniAmine @vict0rsch @alencon @lhproust @SabAmine @khillion