File tree 2 files changed +6
-4
lines changed
2 files changed +6
-4
lines changed Original file line number Diff line number Diff line change 9
9
** /logs /**
10
10
** /tmp /**
11
11
integration /**
12
-
12
+ test.sh
13
13
# -------
14
14
15
15
# Created by https://www.toptal.com/developers/gitignore/api/python
Original file line number Diff line number Diff line change @@ -33,9 +33,11 @@ SciCode sources challenging and realistic research-level coding problems across
33
33
34
34
| Models | Main Problem Resolve Rate | <span style =" color :grey " >Subproblem</span > |
35
35
| --------------------------| -------------------------------------| -------------------------------------|
36
- | 🥇 OpenAI o3-mini | <div align =" center " >** 9.2** </div > | <div align =" center " style =" color :grey " >33.0</div > |
37
- | 🥈 OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
38
- | 🥉 Deepseek-R1 | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >28.5</div > |
36
+ | 🥇 OpenAI o3-mini-low | <div align =" center " >** 10.8** </div > | <div align =" center " style =" color :grey " >33.3</div > |
37
+ | 🥈 OpenAI o3-mini-high | <div align =" center " >** 9.2** </div > | <div align =" center " style =" color :grey " >34.4</div > |
38
+ | 🥉 OpenAI o3-mini-medium | <div align =" center " >** 9.2** </div > | <div align =" center " style =" color :grey " >33.0</div > |
39
+ | OpenAI o1-preview | <div align =" center " >** 7.7** </div > | <div align =" center " style =" color :grey " >28.5</div > |
40
+ | Deepseek-R1 | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >28.5</div > |
39
41
| Claude3.5-Sonnet | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >26.0</div > |
40
42
| Claude3.5-Sonnet (new) | <div align =" center " >** 4.6** </div > | <div align =" center " style =" color :grey " >25.3</div > |
41
43
| Deepseek-v3 | <div align =" center " >** 3.1** </div > | <div align =" center " style =" color :grey " >23.7</div > |
You can’t perform that action at this time.
0 commit comments