New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run evaluation on GPT2 models #21

Closed

jack89roberts opened this issue Apr 30, 2024 · 0 comments

Milestone

Milestone 1: Work...

Contributor

jack89roberts commented Apr 30, 2024

Compare:

GPT2 base model [Expect: highest forget quality, lowest utility on TOFU retain set]
GPT2 trained on retain only [Expect: high forget quality, high utility on TOFU retain set]
GPT2 trained on full TOFU then fine-tuned to unlearn the forget set [Expect: lower forget quality, lower utility on TOFU retain set]

jack89roberts added this to the Milestone 1: Working pipeline on small novel usecase milestone

jack89roberts closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment