Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust reward model's score module and pooler module order for reducing computation #1956

Merged
merged 9 commits into from
Nov 8, 2024

Conversation

aqweteddy
Copy link
Contributor

Motivation

Modifications

  • Adjust the order of the pooler and score modules in class LlamaForSequenceClassification and class Gemma2ForSequenceClassification to reduce computation.
  • Remove redundant model-loading code in gemma2 reward model.

Checklist

  • [V] Format your code according to the Contributor Guide.
  • [V] Add unit tests as outlined in the Contributor Guide.
  • [V] Update documentation as needed, including docstrings or example tutorials.


def load_weights(self, weights: Iterable[Tuple[str, torch.Tensor]]):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also simplify the weight loader of LlamaForSequenceClassification?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done & verified.
image

@merrymercy
Copy link
Contributor

Can you fix the lint error?

.pre-commit-config.yaml Outdated Show resolved Hide resolved
@merrymercy merrymercy merged commit 4ade15d into sgl-project:main Nov 8, 2024
11 of 12 checks passed
@merrymercy
Copy link
Contributor

@aqweteddy Thanks for the contribution. It is merged.

@aqweteddy aqweteddy deleted the gemma2-rm branch November 8, 2024 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants