Skip to content

Commit d71d2b6

Browse files
authored
Update README.md
1 parent e4ca494 commit d71d2b6

File tree

1 file changed

+2
-17
lines changed

1 file changed

+2
-17
lines changed

README.md

Lines changed: 2 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,10 @@
33
Huggingface Datasets is a great library, but it lacks standardization, and datasets require preprocessing work to be used interchangeably.
44
`tasksource` automates this and facilitates multi-task learning scaling and reproducibility.
55

6-
```python
7-
import tasksource
8-
from datasets import load_dataset
9-
10-
tasksource.bigbench(load_dataset('bigbench', 'movie_recommendation')) # standardized MultipleChoice dataset
11-
```
12-
136
Each dataset is standardized to either `MultipleChoice`, `Classification`, or `TokenClassification` dataset with identical fields.
147
We do not support generation tasks as they are addressed by [promptsource](https://github.com/bigscience-workshop/promptsource).
158

16-
All implemented preprocessings can be found in [tasks.py](https://github.com/sileod/tasksource/blob/main/src/tasksource/tasks.py) or [tasks.md](https://github.com/sileod/tasksource/blob/main/tasks.md). A preprocessing is a function that takes a dataset as input and returns a standardized dataset. The preprocessing annotation is designed to be concise human-readable:
17-
18-
```python
19-
cos_e = tasksource.MultipleChoice(
20-
'question',
21-
choices_list='choices',
22-
labels= lambda x: x['choices_list'].index(x['answer']),
23-
config_name='v1.0')
24-
```
9+
All implemented preprocessings can be found in [tasks.py](https://github.com/sileod/tasksource/blob/main/src/tasksource/tasks.py) or [tasks.md](https://github.com/sileod/tasksource/blob/main/tasks.md). A preprocessing is a function that takes a dataset as input and returns a standardized dataset. The preprocessing annotation is designed to be concise human-readable.
2510

2611
### Installation and usage:
2712
`pip install tasksource`
@@ -43,7 +28,7 @@ See supported 490+ tasks in [tasks.md](https://github.com/sileod/tasksource/blob
4328
I pretrained models on tasksource and obtained state-of-the-art results:
4429
<https://huggingface.co/sileod/deberta-v3-base-tasksource-nli>
4530

46-
### Contact
31+
### Contact and citation
4732
I can help you integrate tasksource in your experiments. `[email protected]`
4833

4934
More details on this [article:](https://arxiv.org/abs/2301.05948)

0 commit comments

Comments
 (0)