Skip to content

Commit

Permalink
error line fix
Browse files Browse the repository at this point in the history
  • Loading branch information
nmowmita committed Feb 16, 2024
1 parent 2bf4c39 commit 329ea57
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lm_eval/tasks/crowspairsde.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
success across many NLP tasks. However, there is ample evidence that they use the cultural biases that are
undoubtedly present in the corpora they are trained on, implicitly creating harm with biased representations. To
measure some forms of social bias in language models against protected demographic groups in the US, we introduce the
Crowdsourced Stereotype Pairs benchmark (CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing
Crowdsources Stereotype Pairs benchmark (CrowS-Pairs). CrowS-Pairs has 1508 examples that cover stereotypes dealing
with nine types of bias, like race, religion, and age. In CrowS-Pairs a model is presented with two sentences: one
that is more stereotyping and another that is less stereotyping. The data focuses on stereotypes about historically
disadvantaged groups and contrasts them with advantaged groups. We find that all three of the widely-used MLMs we
Expand Down

0 comments on commit 329ea57

Please sign in to comment.