Reducing Overhead Costs by Merging Deployments (Aug.)

[elisabettai]

@GitHK
@pcrespov
@sanderegg
@bisgaard-itis
@giancarloromeo

Description

Reducing overhead costs by merging of o²S²PARC and Sim4Life deployment: After Y8, it will be desirable to merge the currently distinct o²S²PARC and Sim4Life deployments, to avoid duplication of infrastructure overhead (orchestration, storage, service catalog, authentication, monitoring, metrics scraping, dashboard, logging Services; web server, databases, testing infrastructure) and DevOps.
Successful migration requires: merging users, groups and organizations; merging S3 data, merging projects. These data and metadata "live" in different places (e.g., postgres, storage db, S3) and are strongly correlated. Carefully crafted scripts and APIs must be created to automate the process in a robust and reliable manner.

Deliverable

Revised platform deployment

Acceptance criteria:
o²S²PARC and Sim4Life share a common deployment

Deadline

Y8Q4 (Aug.)

Wrike link (account/access required)

[mrnicegyu11]

ITISFoundation/osparc-ops-environments#818

[mrnicegyu11]

Sprint planning TheAwakening

The plan is to insert / upsert the DB of osparc into the DB of sim4life. To do this, all osparc postgres tables will be audited and made ready for this procedure. Some tables might be empty (by design), some tables might not need to be moved and can be dropped, and some tables might need adjustments.

In order to avoid collisions in the DB, such as with tables that have monotonically increasing primary keys starting from 1, we have identified two ways to make the DB-tables of osparc.io "ready":

1. Add a large integer (needs to be determined individually) to all primary keys of osparc.io OR sim4life.io, to avoid collisions. [Note: This doesnt always work, e.g. in the users DB mail adresses are presumably expected to be unique even though they are not enforced to be unique in postgres?
2. Add the product as a second primary key and thereby make combinations of user-id & product etc. unique.

Likely, both ways would go via a PR with an alembic migration

Tables and responisble persons:

see #1867

Please reference this issue in PRs that do related DB changes or cleanups

Further tasks identified:

• move of osparc.io: Ops tasks (Epic) osparc-ops-environments#993
• move of osparc.io: Check Registry #1864
• move of osparc.io: full autoscaling #1865
• move of osparc.io: updating legacy services #1866