Search results

Is there any way to disable some language to run test

Status: Open.

#91 In scaleapi/SWE-bench_Pro-os;

· sxqqslf opened on Apr 10, 2026

Question: Any plan to add Agent Framework results to SWE-Bench Pro Public Leaderboard

Status: Open.

#90 In scaleapi/SWE-bench_Pro-os;

· yangyixiang-cc opened on Apr 7, 2026

can i ask abt the discrepancy of the public set results for GLM?

Status: Open.

#89 In scaleapi/SWE-bench_Pro-os;

· simkimsia opened on Mar 28, 2026

Missing registry.py breaks file editing tools in Scale's SWE-agent fork

Status: Open.

#87 In scaleapi/SWE-bench_Pro-os;

· aksakalmustafa opened on Mar 26, 2026

Date-dependent P2P test names become stale over time (6 openlibrary instances)

Status: Open.

#85 In scaleapi/SWE-bench_Pro-os;

· sspriyadharshini31 opened on Mar 25, 2026

Any public experiment repo for https://labs.scale.com/leaderboard/sweatlas-qna?

Status: Open.

#84 In scaleapi/SWE-bench_Pro-os;

· zhimin-z opened on Mar 25, 2026

Config file to be used for swebench submission

Status: Open.

#82 In scaleapi/SWE-bench_Pro-os;

· gokulr-cerebras opened on Mar 24, 2026

wrong test names in fail_to_pass in HF dataset

Status: Open.

#81 In scaleapi/SWE-bench_Pro-os;

· wistuba opened on Mar 20, 2026

Bad dockerfile for element-hq

Status: Open.

#80 In scaleapi/SWE-bench_Pro-os;

· zhouyuyang2002 opened on Mar 19, 2026

API rate limit errors silently counted as benchmark failures in SWE-bench Pro evaluation

Status: Open.

#78 In scaleapi/SWE-bench_Pro-os;

· tfidfwastaken opened on Mar 10, 2026

Test name mismatches cause false negatives in eval scoring (whitespace + truncated quotes in fail_to_pass)

Status: Open.

#76 In scaleapi/SWE-bench_Pro-os;

· pedropnaves opened on Feb 28, 2026

Inconsistency between images on dockerhub and Dockerfiles in repo

Status: Open.

#75 In scaleapi/SWE-bench_Pro-os;

· andre15silva opened on Feb 17, 2026