BERT_ML: Alloy Property Prediction Experiments

中文说明

This repository contains Python workflows for alloy property prediction with traditional ML, BERT-based embeddings, TabPFN, and LLMProp. The project is organized for full experiment reproducibility: all runnable experiment entrypoints are Python modules or Python scripts, model assets live under models/, and historical repair utilities are archived separately.

Quick Start

Create the environment from the project root:

conda env create -f environment.yml
conda activate llm

or install with pip:

pip install -r requirements.txt

Download model assets when they are not already present:

python models/download_bert_models.py
python models/download_llmprop_models.py

SteelBERT is a gated Hugging Face model. Log in with huggingface-cli login and request access at MGE-LLMs/SteelBERT.

Python-Only Entrypoints

Shell wrappers are intentionally not used. Run experiments with Python only:

python -m src.pipelines.run_batch --list
python -m src.pipelines.run_batch --config experiment1_all_ml_models
python -m src.pipelines.run_batch --all --dry_run

OOD experiments:

python -m src.pipelines.run_batch_ood --list
python -m src.pipelines.run_batch_ood --config experiment1_all_ml_models_extrapolation --dry_run
python -m src.pipelines.run_cv_k_sweep --dry_run

TabPFN experiments:

python -m src.TabPFN.train_tabpfn --all --backend local --feature_mode numeric
python -m src.TabPFN.train_tabpfn_ood --all --backend local --feature_mode numeric
python -m src.TabPFN.run_tabpfn_ood_batch --list

LLMProp OOD single-run entrypoint:

python -m src.LLMProp.run_llmprop_ood --help

Model Assets

Model assets are centralized under models/:

• models/matscibert/
• models/scibert/
• models/steelbert/
• models/llmprop/
• models/tabpfn/tabpfn-v2-regressor.ckpt

Generated training checkpoints stay under output/**/checkpoints/ because they are experiment outputs, not reusable base assets.

Data And Outputs

Tracked input datasets are under datasets/. The active configuration files reference these normalized CSV paths:

• datasets/Ti_alloys/titanium.csv
• datasets/Al_Alloys/aluminum.csv
• datasets/HEA_data/hea.csv
• datasets/Steel/steel.csv
• datasets/HEA_data/Pitting_potential_data_xiongjie.csv
• datasets/matbench_convert/matbench_steels.csv
• datasets/matbench_convert/matbench_steels_ood.csv

Generated features, logs, figures, and model outputs are written to ignored directories such as Features/, Features_extrapolation/, output/, and logs/.

Reporting Scripts

Active analysis and figure scripts remain in Scripts/. Examples:

python Scripts/reporting/batch_summarize_extrapolation_results.py --base-dir output/ood_results
python Scripts/reporting/batch_summarize_bert_extrapolation_results.py --base-dir output/ood_results
python Scripts/reporting/batch_summarize_tabpfn_extrapolation_results.py
python Scripts/reporting/batch_summarize_llmprop_ood_results.py --base-dir output/ood_results
python Scripts/reporting/batch_summarize_combined_ood_reports.py --reports-root output/ood_summary_reports
python Scripts/figures/build_bestplus_tabpfn_triptych.py --config configs/figures/build_bestplus_tabpfn_triptych.paper.config.yaml

Optional external OOD sources are configured in configs/ood_reporting/external_ood_model_sources.yaml. Set EXTERNAL_OOD_ROOT before running combined summaries if external LLM results should be merged.

Documentation

• English reproducibility guide
• 中文复现指南
• Script inventory
• 脚本清单
• Batch run guide
• TabPFN guide

Historical local repair and migration utilities are preserved in archive/legacy_scripts/. They are not recommended reproduction entrypoints.