Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

If hypothesis="1 2 1 2" reference="1 2 3 4 5 1 2 6", what is the right result? #61

Open
DtYXs opened this issue Jul 15, 2021 · 1 comment

Comments

@DtYXs
Copy link

DtYXs commented Jul 15, 2021

from rouge import Rouge

hypothesis="1 2 1 2"
reference="1 2 3 4 5 1 2 6"
rouge = Rouge()
scores = rouge.get_scores(hypothesis, reference)

so, what is the right result of scores?

When I use pip install rouge to install, the result:

[{'rouge-1': {'f': 0.6666666622222223, 'p': 1.0, 'r': 0.5},
  'rouge-2': {'f': 0.3999999958, 'p': 0.6666666666666666, 'r': 0.2857142857142857},
  'rouge-l': {'f': 0.4999999962500001, 'p': 1.0, 'r': 0.3333333333333333}}]

When I use git clone xxx python setup.py install to install, the result:

[{'rouge-1': {'f': 0.4999999962500001, 'p': 1.0, 'r': 0.3333333333333333},
  'rouge-2': {'f': 0.24999999625000005, 'p': 0.5, 'r': 0.16666666666666666},
  'rouge-l': {'f': 0.4999999962500001, 'p': 1.0, 'r': 0.3333333333333333}}]

I want to know which is the right result?

Thanks

@sanghyuk-choi
Copy link

@DtYXs // The Official ROUGE Script (ROUGE1.5.5.pl) results

1 ROUGE-1 Average_R: 0.50000 (95%-conf.int. 0.50000 - 0.50000)
1 ROUGE-1 Average_P: 1.00000 (95%-conf.int. 1.00000 - 1.00000)
1 ROUGE-1 Average_F: 0.66667 (95%-conf.int. 0.66667 - 0.66667)
---------------------------------------------
1 ROUGE-2 Average_R: 0.28571 (95%-conf.int. 0.28571 - 0.28571)
1 ROUGE-2 Average_P: 0.66667 (95%-conf.int. 0.66667 - 0.66667)
1 ROUGE-2 Average_F: 0.40000 (95%-conf.int. 0.40000 - 0.40000)
---------------------------------------------
1 ROUGE-L Average_R: 0.50000 (95%-conf.int. 0.50000 - 0.50000)
1 ROUGE-L Average_P: 1.00000 (95%-conf.int. 1.00000 - 1.00000)
1 ROUGE-L Average_F: 0.66667 (95%-conf.int. 0.66667 - 0.66667)

So. I think pip version is the right result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants