Complete Testing Report #1
Description
Hello,
I am a former tutor at 42 Paris, and I wrote my own tester for the rush01 project. I would like to share the results I obtained.
Issues identified
-
Some basic errors are not properly handled
→ the program either produces incorrect output or triggers a segmentation fault. -
Some valid grids are not solved.
-
The reported execution times seem incorrect, especially for 8×8 grids and larger.
-
All tested cases show memory leaks.
Resources
You can find my tester along with the full report (tests performed without fsanitize) here:
Feel free to contact me if you need more details about the test cases or the testing methodology.
=============================================
| test mandatory Error 4 x 4 |
=============================================
./rush-01
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
Error
./rush-01
./test.sh : ligne 11 : 110803 Erreur de segmentation (core dumped) "$@"
./rush-01 4 3 2 1 1 2 2 2 4 d 2 1 1 2 2 2
Error
./rush-01 4 3 2 0 1 2 2 2 4 3 2 1 1 2 2 2
Error
./rush-01 -4 -3 -2 -1 -1 -2 -2 -2 -4 -3 -2 -1 -1 -2 -2 -2
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
1 2 3 4
2 3 4 1
3 4 1 2
4 1 2 3
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
1 2 3 4
2 3 4 1
3 4 1 2
4 1 2 3
./rush-01 3321123333211233
Error
./rush-01 3321123333211233
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
1 2 3 4
2 3 4 1
3 4 1 2
4 1 2 3
./rush-01 2 1 2 4 224 2 1 2 1 3 2 3 3 2 1
Error
./rush-01 2 1 2 4 2 4 2 1 2 1 3 2 3 3 2 1
Error
./rush-01
./test.sh : ligne 11 : 110823 Erreur de segmentation (core dumped) "$@"
./rush-01 1111111111111111111111111111111
Error
./rush-01 4 2 1 1 2 2 24 3 3 2 1 1 2 2 2 2
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2 2
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 2 2 2
Error
./rush-01 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Error
./rush-01 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2
Error
./rush-01 3 1 2 2 2 3 3 1 2 3 1 2 2 1 3 2
Error
./rush-01 1 4 2 2 3 2 1 2 1 3 2 2 3 1 2 2
Error
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
Error
=============================================
| test mandatory OK 4 x 4 |
=============================================
./rush-01 1 2 2 3 3 2 2 1 1 3 2 3 4 2 2 1
4 3 2 1$
1 2 4 3$
3 4 1 2$
2 1 3 4$
./rush-01 1 2 3 2 3 1 2 2 1 4 2 2 4 1 2 2
4 3 2 1$
1 2 3 4$
3 1 4 2$
2 4 1 3$
./rush-01 1 3 3 2 4 1 2 2 1 2 3 2 2 1 2 3
Error
Error: "invalid grid"
./rush-01 1 4 2 2 4 1 2 2 1 2 3 2 2 1 2 3
4 1 2 3$
3 2 1 4$
2 3 4 1$
1 4 3 2$
./rush-01 2 3 1 2 2 1 3 2 3 1 3 2 2 4 1 2
1 2 4 3$
4 3 2 1$
2 1 3 4$
3 4 1 2$
./rush-01 2 1 2 3 2 3 2 1 2 3 1 2 3 2 3 1
2 4 3 1$
1 2 4 3$
4 3 1 2$
3 1 2 4$
./rush-01 2 1 2 4 2 4 2 1 2 1 3 2 3 3 2 1
2 4 3 1$
4 3 1 2$
1 2 4 3$
3 1 2 4$
./rush-01 2 4 1 2 2 1 3 3 2 4 1 2 2 1 3 3
2 1 4 3$
1 2 3 4$
4 3 1 2$
3 4 2 1$
./rush-01 2 2 1 3 2 3 2 1 2 2 1 3 2 2 3 1
Error
Error: "invalid grid"
./rush-01 2 3 2 1 3 1 2 2 3 1 2 2 1 3 2 2
Error
Error: "invalid grid"
./rush-01 2 2 2 1 2 1 3 4 3 3 1 2 1 2 3 3
2 3 1 4$
1 2 4 3$
4 1 3 2$
3 4 2 1$
./rush-01 2 4 2 1 2 1 2 4 2 1 3 2 1 2 2 3
3 1 2 4$
4 2 1 3$
1 3 4 2$
2 4 3 1$
./rush-01 2 2 1 3 3 2 2 1 3 1 2 4 2 2 2 1
2 3 4 1$
4 1 2 3$
3 4 1 2$
1 2 3 4$
./rush-01 3 2 1 2 1 2 3 2 3 2 2 1 2 1 3 3
1 2 4 3$
3 1 2 4$
2 4 3 1$
4 3 1 2$
./rush-01 3 1 2 3 2 2 3 1 2 2 1 3 3 2 2 1
Error
Error: "invalid grid"
./rush-01 3 2 2 1 2 2 1 3 4 2 1 3 1 2 2 2
1 2 3 4$
3 4 1 2$
4 1 2 3$
2 3 4 1$
./rush-01 3 1 2 2 2 3 2 1 2 2 1 4 2 2 4 1
Error
Error: "invalid grid"
./rush-01 3 1 3 2 2 4 1 2 2 3 1 2 2 1 3 2
1 4 2 3$
2 3 1 4$
4 2 3 1$
3 1 4 2$
./rush-01 4 2 1 2 1 2 3 3 3 3 2 1 2 1 2 4
1 2 4 3$
2 1 3 4$
3 4 1 2$
4 3 2 1$
./rush-01 4 3 2 1 1 2 2 2 4 3 2 1 1 2 2 2
1 2 3 4$
2 3 4 1$
3 4 1 2$
4 1 2 3$
=============================================
| Partie bonus |
=============================================
=============================================
| test Erreur 1 x 1 |
=============================================
./rush-01 1 1 1 1 1
Error
./rush-01 1 1 1
Error
./rush-01 1 1 1 1
1
./rush-01 1 1 1 1
1
./rush-01 1 1 1 1
1
./rush-01 1111111
Error
./rush-01
./test.sh : ligne 11 : 110924 Erreur de segmentation (core dumped) "$@"
./rush-01 1 111 1
Error
./rush-01 1 1 1 1
Error
./rush-01 1 a 1 1
Error
./rush-01 1 2 1 1
Error
=============================================
| test OK 1 x 1 |
=============================================
./rush-01 1 1 1 1
1$
=============================================
| test Erreur 2 x 2 |
=============================================
./rush-01 2 1 1 2 2 1 1 2 1
Error
./rush-01 2 1 1 2 2 1 1
Error
./rush-01 2 1 1 2 2 1 1 2
1 2
2 1
./rush-01 2 1 1 2 2 1 1 2
1 2
2 1
./rush-01 2 1 1 2 2 1 1 2
1 2
2 1
./rush-01 221111222211112
Error
./rush-01
./test.sh : ligne 11 : 110944 Erreur de segmentation (core dumped) "$@"
./rush-01 2 1 1 212 1 1 2
Error
./rush-01 2 1 1 2 2 1 1 2
Error
./rush-01 2 1 a 2 2 1 1 2
Error
./rush-01 2 1 3 2 2 1 1 2
Error
./rush-01 2 1 2 1 2 1 1 2
Error
=============================================
| test OK 2 x 2 |
=============================================
./rush-01 2 1 1 2 2 1 1 2
1 2$
2 1$
./rush-01 1 2 2 1 1 2 2 1
2 1$
1 2$
=============================================
| test Erreur 3 x 3 |
=============================================
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2 1
Error
./rush-01 2 2 1 2 1 3 3 1 2 1 2
Error
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2
1 2 3
3 1 2
2 3 1
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2
1 2 3
3 1 2
2 3 1
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2
1 2 3
3 1 2
2 3 1
./rush-01 22221122113333112211222
Error
./rush-01
./test.sh : ligne 11 : 110969 Erreur de segmentation (core dumped) "$@"
./rush-01 2 2 1 211 3 3 1 2 1 2 2
Error
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2
Error
./rush-01 2 2 1 2 a 3 3 1 2 1 2 2
Error
./rush-01 2 2 1 2 1 3 3 1 4 1 2 2
Error
./rush-01 2 2 1 2 1 2 3 1 2 1 2 2
Error
=============================================
| test OK 3 x 3 |
=============================================
./rush-01 2 2 1 2 1 3 3 1 2 1 2 2
1 2 3$
3 1 2$
2 3 1$
./rush-01 1 2 2 3 1 2 1 2 2 3 1 2
3 2 1$
2 1 3$
1 3 2$
./rush-01 3 1 2 1 2 2 2 2 1 2 1 3
1 3 2$
2 1 3$
3 2 1$
./rush-01 2 1 3 2 2 1 2 1 3 2 2 1
2 3 1$
3 1 2$
1 2 3$
./rush-01 2 3 1 2 1 2 2 1 2 1 3 2
2 1 3$
3 2 1$
1 3 2$
./rush-01 1 3 2 2 1 2 1 3 2 2 1 2
3 1 2$
1 2 3$
2 3 1$
./rush-01 2 1 2 1 3 2 2 3 1 2 1 2
2 3 1$
1 2 3$
3 1 2$
./rush-01 2 1 2 2 3 1 2 1 2 2 3 1
1 3 2$
3 2 1$
2 1 3$
./rush-01 1 2 2 3 2 1 1 2 3 2 2 1
3 1 2$
2 3 1$
1 2 3$
./rush-01 3 2 1 1 2 2 3 2 1 1 2 2
1 2 3$
2 3 1$
3 1 2$
./rush-01 1 2 3 2 2 1 1 2 2 3 2 1
3 2 1$
1 3 2$
2 1 3$
./rush-01 2 2 1 1 2 3 2 2 1 1 2 3
2 1 3$
1 3 2$
3 2 1$
=============================================
| test Erreur 5 x 5 |
=============================================
=============================================
| test OK 5 x 5 |
=============================================
./rush-01 3 2 1 2 4 2 4 4 2 1 3 2 3 1 2 3 4 2 2 1
Error
Error: "invalid grid"
./rush-01 5 2 1 3 2 1 4 2 2 2 3 2 3 2 1 2 3 2 1 3
Error
Error: "invalid grid"
./rush-01 2 1 2 3 3 2 4 2 3 1 2 2 3 1 2 4 2 2 3 1
Error
Error: "invalid grid"
./rush-01 3 2 3 1 2 2 1 2 5 3 3 3 1 3 2 2 1 4 2 2
Error
Error: "invalid grid"
./rush-01 2 5 2 1 2 2 1 2 3 4 2 3 1 3 2 2 1 2 2 4
Error
Error: "invalid grid"
=============================================
| test Erreur 6 x 6 |
=============================================
=============================================
| test OK 6 x 6 |
=============================================
./rush-01 4 4 2 2 1 3 1 2 3 3 4 2 3 4 3 2 2 1 2 2 3 3 1 5
3 1 5 2 6 4$
1 3 4 6 2 5$
2 4 6 1 5 3$
4 6 3 5 1 2$
5 2 1 3 4 6$
6 5 2 4 3 1$
./rush-01 4 3 2 2 1 4 1 2 4 4 3 2 4 3 2 4 2 1 2 2 2 1 3 2
Error
Error: "invalid grid"
./rush-01 4 1 3 2 2 3 2 3 2 1 4 3 2 3 3 1 3 2 3 2 1 5 2 2
1 6 4 5 2 3$
3 1 5 2 6 4$
4 2 1 3 5 6$
6 5 2 4 3 1$
2 3 6 1 4 5$
5 4 3 6 1 2$
./rush-01 3 3 3 2 1 2 2 4 2 3 4 1 3 4 2 1 3 2 2 3 3 3 4 1
Error
Error: "invalid grid"
./rush-01 3 2 2 2 1 3 1 2 3 3 3 2 3 3 3 2 4 1 2 2 2 3 1 4
Error
Error: "invalid grid"
=============================================
| test Erreur 7 x 7 |
=============================================
=============================================
| test OK 7 x 7 |
=============================================
./rush-01 4 6 2 1 4 3 2 1 2 2 4 2 3 5 3 7 5 4 3 2 1 2 1 2 2 3 3 5
Error
Error: "invalid grid"
./rush-01 7 2 1 4 5 2 3 1 2 3 2 3 3 3 3 4 3 2 3 2 1 3 2 1 3 3 3 4
Error
Error: "invalid grid"
./rush-01 5 2 2 3 3 1 2 1 2 2 4 3 5 4 4 7 3 2 2 2 1 2 1 2 3 4 4 5
Error
Error: "invalid grid"
./rush-01 5 2 4 2 3 2 1 1 2 2 3 3 5 4 3 6 3 2 2 2 1 1 2 3 4 4 4 3
Error
Error: "invalid grid"
./rush-01 5 5 1 2 3 3 2 1 2 3 2 3 3 4 2 6 4 3 3 2 1 4 1 2 3 3 2 5
Error
Error: "invalid grid"
=============================================
| test Erreur 8 x 8 |
=============================================
=============================================
| test OK 8 x 8 |
=============================================
./rush-01 2 4 3 1 3 2 2 3 2 1 3 3 3 2 4 2 2 4 3 2 1 2 4 2 4 2 2 4 4 3 1 3
^C