Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Please make sure ./data/safety.txt is correct #1787

Open
2 tasks done
wccccp opened this issue Dec 27, 2024 · 0 comments
Open
2 tasks done

[Bug] Please make sure ./data/safety.txt is correct #1787

wccccp opened this issue Dec 27, 2024 · 0 comments

Comments

@wccccp
Copy link

wccccp commented Dec 27, 2024

Prerequisite

Type

I'm evaluating with the officially supported tasks/models/datasets.

Environment

Name: opencompass
Version: 0.3.8
Summary: A comprehensive toolkit for large model evaluation
Home-page: https://github.com/open-compass/opencompass
Author: OpenCompass Contributors
Author-email:
License: Apache License 2.0
Location: /work/opencompass
Editable project location: /work/opencompass
Requires: absl-py, accelerate, cpm_kernels, datasets, einops, evaluate, func_timeout, fuzzywuzzy, gradio-client, h5py, httpx, huggingface_hub, immutabledict, importlib-metadata, jieba, json5, jsonlines, mmengine-lite, nltk, numpy, openai, OpenCC, opencv-python-headless, pandas, prettytable, protobuf, pyext, python-Levenshtein, rank_bm25, rapidfuzz, requests, retrying, rich, rouge, rouge_chinese, rouge_score, sacrebleu, scikit_learn, seaborn, sentence_transformers, tabulate, tiktoken, timeout_decorator, tokenizers, torch, tqdm, transformers, tree-sitter, tree_sitter_languages, typer
Required-by:

Reproduces the problem - code/configuration sample

python run.py
--models vllm_qwen2p5_7b_instruct
--datasets safety_gen\

Reproduces the problem - command or script

python run.py
--models vllm_qwen2p5_7b_instruct
--datasets safety_gen\

Reproduces the problem - error message

Traceback (most recent call last):
File "/work/opencompass/run.py", line 4, in
main()
File "/work/opencompass/opencompass/cli/main.py", line 299, in main
tasks = partitioner(cfg)
File "/work/opencompass/opencompass/partitioners/base.py", line 83, in call
tasks = self.partition(**model_and_dataset_args,
File "/work/opencompass/opencompass/partitioners/num_worker.py", line 69, in partition
dataset_size = self.get_size(dataset)
File "/work/opencompass/opencompass/partitioners/num_worker.py", line 141, in get_size
dataset = build_dataset_from_cfg(dataset)
File "/work/opencompass/opencompass/utils/build.py", line 13, in build_dataset_from_cfg
return LOAD_DATASET.build(dataset_cfg)
File "/usr/local/lib/python3.10/dist-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/usr/local/lib/python3.10/dist-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/work/opencompass/opencompass/datasets/base.py", line 12, in init
self.dataset = self.load(**kwargs)
File "/work/opencompass/opencompass/datasets/safety.py", line 14, in load
path = get_data_path(path, local_mode=True)
File "/work/opencompass/opencompass/utils/datasets.py", line 30, in get_data_path
return download_dataset(local_path, cache_dir)
File "/work/opencompass/opencompass/utils/datasets.py", line 90, in download_dataset
assert dataset_name, f'No valid url for {data_path}!\n' + \
AssertionError: No valid url for ./data/safety.txt!
Please make sure ./data/safety.txt is correct

Other information

What is the dataset corresponding to "safety"? I have already downloaded the relevant zip file as per the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant