Skip to content

npy and pkl not found in llm-topla\llm-topla-main\results\mmlu_hf extracted from "Basemodel Outputs" compressed file. #2

@xsthunder

Description

@xsthunder

Hi, thanks for sharing the code. i got into trouble when trying the runing the mmlu_hf task.

Image

as i dived into the code, i found pkl and npy is needed:

llm-topla/helper.py

Lines 158 to 165 in a01e555

def load_mmlu_multi_run(model_names, dataset_name):
data_dir = os.path.join(RESULT_DIR, "mmlu")
# find number of samples for each model
data = []
choices = ["A", "B", "C", "D"]
for model_n in model_names:
arr_path = os.path.join(data_dir, model_n, dataset_name, f"latest_predictions.npy")
outputs_path = os.path.join(data_dir, model_n, dataset_name, f"latest_outputs.pkl")

however pkl and npy is not presented in the compressed file ur provided. There is only csv files.

llm-topla/README.md

Lines 29 to 35 in a01e555

## Basemodel Outputs
The base model outputs of these datasets are can be downloaded from the link below and must be put inside `results/`
```
https://drive.google.com/drive/folders/17_5JN5koFKsnyty9klNz1nBGYLKSE4tw?usp=sharing
```

PS: the gsm8k folder has npy and pkl and was successful

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions