Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable re-use of a fitted Categorify operator in another workflow #972

Closed
Tracked by #211
EvenOldridge opened this issue May 10, 2023 · 2 comments
Closed
Tracked by #211
Assignees
Milestone

Comments

@EvenOldridge
Copy link
Member

EvenOldridge commented May 10, 2023

In order to support pretrained embeddings we need to be able to use categorify from another workflow

This could be solved by subgraphs and is our long term solution.

Categorify fit creates a unique values file for its own internal use which doesn't contain the reserved values. It also produces a categorify metadata file that includes counts, buckets, etc.

We need to use those files to transform the uuid column in an embedding table in order to ensure it matches the categorified column within the dataset.

@oliverholworthy
Copy link
Member

Subgraph operator added in NVIDIA-Merlin/core#334

@oliverholworthy oliverholworthy changed the title Modify Categorify to be able to accept an existing fit from another workflow Enable re-use of a fitted Categorify operator in another workflow Jun 12, 2023
@karlhigley
Copy link
Contributor

There's also an example of how to do this in the tests added by NVIDIA-Merlin/NVTabular#1827

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants