Skip to content

Commit

Permalink
correct the link to create_reference_file.py
Browse files Browse the repository at this point in the history
  • Loading branch information
zhaowei-wang-nlp committed Dec 2, 2023
1 parent 55a3656 commit 0214b5e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion eval/automatic/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Then, you can evaluate your prediction output as follows:
python evaluation.py --prediction_file=predictions.jsonl --reference_file=references.jsonl
```

Each line of `predictions.jsonl` should correspond to a json object that contains `id` and `prediction` fields. `reference.jsonl` should have `id`, `references`, `task_id`, `task_category` and `track` in each line. To produce the reference file of our official test set, you can run the script in [`eval/automatic/leaderboard/create_reference_file.py`](eval/automatic/leaderboard/create_reference_file.py) as follows:
Each line of `predictions.jsonl` should correspond to a json object that contains `id` and `prediction` fields. `reference.jsonl` should have `id`, `references`, `task_id`, `task_category` and `track` in each line. To produce the reference file of our official test set, you can run the script in [`eval/automatic/leaderboard/create_reference_file.py`](eval/leaderboard/create_reference_file.py) as follows:

```bash
python eval/automatic/leaderboard/create_reference_file.py

0 comments on commit 0214b5e

Please sign in to comment.