Add colab extra for Colab-compatible dependency bounds - #49

Pull request overview

This PR updates packaging metadata to improve Google Colab installation compatibility by relaxing lower bounds on core dependencies and introducing a dedicated colab extra with upper caps aligned to Colab’s commonly preinstalled packages.

Changes:

• Relax base dependency lower bounds for rich, opentelemetry-sdk, and opentelemetry-exporter-otlp.
• Add a colab optional dependency extra that caps rich and key OpenTelemetry packages to <1.44 / <15 for Colab-friendly resolution.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

TL;DR

1. Document uv add for uv projects and pip / uv pip for existing envs like Colab
2. Lower runtime floors to the oldest versions we actually need, across the board
3. Point Dependabot at uv.lock only (lockfile-only)
4. Leave pyproject.toml floors alone until a real API or compatibility reason appears

Explanation

I dug deeper and have findings and proposals on how to tackle this problem, not only for Colab but in general.

This is the current status:

• When we use pip install in Colab and get errors, the package was actually installed, but now dependencies were changed for other packages already pre-installed in the Colab venv. This is a problem for user experience (seeing errors) and from a technical perspective (we made the Colab env internally inconsistent)

$ pip install git+https://github.com/elastic/elastic-evals-sdk-pytho

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-adk 2.7.0 requires opentelemetry-api<=1.43,>=1.39, but you have opentelemetry-api 1.44.0 which is incompatible.
google-adk 2.7.0 requires opentelemetry-sdk<=1.43,>=1.39, but you have opentelemetry-sdk 1.44.0 which is incompatible.
pyiceberg 0.11.1 requires rich<15.0.0,>=10.11.0, but you have rich 15.0.0 which is incompatible.
bigframes 2.42.0 requires rich<14,>=12.4.4, but you have rich 15.0.0 which is incompatible.

• When we use uv pip install, the user experience is better because we do not see any errors, but the "technical" problem is still there (we broke the Colab env)

$ uv pip install "git+https://github.com/elastic/elastic-evals-sdk-python.git"

Using Python 3.12.13 environment at: /usr
Resolved 39 packages in 1.25s
Prepared 10 packages in 1.79s
Uninstalled 4 packages in 64ms
Installed 10 packages in 16ms
 + elastic-evals==0.1.0 (from git+https://github.com/elastic/elastic-evals-sdk-python.git@dafa8a79640fea4246ac92abdc3283cb1a1dfdd3)
 - opentelemetry-api==1.43.0
 + opentelemetry-api==1.44.0
 + opentelemetry-exporter-otlp==1.44.0
 + opentelemetry-exporter-otlp-proto-common==1.44.0
 + opentelemetry-exporter-otlp-proto-grpc==1.44.0
 + opentelemetry-exporter-otlp-proto-http==1.44.0
 + opentelemetry-proto==1.44.0
 - opentelemetry-sdk==1.43.0
 + opentelemetry-sdk==1.44.0
 - opentelemetry-semantic-conventions==0.64b0
 + opentelemetry-semantic-conventions==0.65b0
 - rich==13.9.4
 + rich==15.0.0

Expanding on the existing problem:

• We want to loosen the floors for our dependencies, but not only for Colab-affected packages. We need to do it for all of them
• This is not only a problem in Colab, but it is also a problem in any customer venvs that are not up to date, or CI images, etc.
• That way me make sure that we are aligned with any future Colab images, CI images, and customer venvs

Solution that I would take:

• Loosen floors: all project dependencies should be the "oldest versions we actually need" and that are secure
• Dependabot update: we need to set versioning-strategy: lockfile-only so Dependabot updates only uv. lock, and it does not constantly increase versions in pyproject.toml
• Readme update: our current readme promotes uv add, which is okay, but it does not work in Colab because it is not a uv project. But it has uv installed, and we should add another option, which is uv pip install "git+https://github.com/elastic/elastic-evals-sdk-python.git, besides uv add and an explanation of when to use which
• Manual pyproject.toml updates: uv.lock will be updated weekly as it is currently, but pyproject.toml will now be updated manually, and it should be relatively rare. We should update pyproject.toml when code needs a newer API, a specific version is known to be broken, or because of a security fix

I ran both approaches locally and then in Colab. The results show that you were right that we need a dedicated dependencies group for Colab specifically.

Loosening the floors is enough in a venv: if Rich 13.9.4 is already installed and we ask for rich>=13, pip keeps 13.9.4. Same for api/sdk 1.43 against >=1.20.

Colab is different because opentelemetry-exporter-otlp is not preinstalled. Pip fetches 1.44, and that wheel requires opentelemetry-sdk~=1.44.0, so it upgrades api/sdk and fights google-adk. But rich for example never moved.

So I agree we should have a [colab] extra. I would only cap the OTEL stack there (>=1.20,<1.44), not Rich.

I would still suggest that the rest of proposals for last review be incorporated for beter user experience in general:

• lower floors on all runtime deps, not only for the Colab
• README: uv add for uv projects, uv pip / pip for existing envs, and elastic-evals[colab] for Colab
• Dependabot versioning-strategy: lockfile-only
• a min-versions workflow so the floors stay honest (this is new)

I already made these changes on update-dependencies-boundaries branch so you can take a look and use it. You can pull that into this PR, or we can merge this one, and I will open a new PR from that branch later.

Thanks @masci