add interfaces and tests for nearest neighbors and embeddings

[gschoeni]

from oxen import Workspace, DataFrame

# Create a workspace off main
workspace = Workspace(remote_repo, "main")

# Create a data frame index
remote_df = DataFrame(workspace, "question_embeddings.parquet")

# Check if nearest neighbors is enabled
column = "question_embeddings"
is_enabled = remote_df.is_nearest_neighbors_enabled(column=column)

# Index the embeddings column
remote_df.enable_nearest_neighbors(column=column)

# Do a nearest neighbors search
rows = remote_df.query(
    find_embedding_where={"title": "A"}, sort_by_similarity_to="question_embeddings"
)

assert len(rows) > 1
assert rows[0]["id"] == "290"
assert rows[0]["title"] == "A"

[gschoeni]

@jcelliott @EloyMartinez there are test parquet files in the oxen/tests/data/QuestionEmbeddings/ folder if you want to give it a spin with the python snippet above. There are also a few unit tests for reference.

Let me know if the APIs make sense and are intuitive to use or if we should iterate on them.

Also we should do a python deep dive sometime this week with the team.

[jcelliott]

I believe this will make package managers try to install torch and tensorflow as dependencies for oxen. Is that what we want?

[gschoeni]

I was debating, because they are used in our data loaders (that are undocumented) but are a heavy install. Maybe we do just keep them in the dev dependencies for now...

[jcelliott]

Worked for me, and I think the interface makes sense.

Not really related to this PR, but I ran into that issue with the PyRemoteRepo vs RemoteRepo. It looks like remote_repo.get_repo actually returns a PyRemoteRepo, but the docs say it returns a RemoteRepo.