-
Notifications
You must be signed in to change notification settings - Fork 46
Batch RECTESTS into one process #1316
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
(it's me. i'm the dummy)
.REC files are now automatically picked up from the RECTESTS folder.
3c6adb1 to
8186384
Compare
|
timings in CI: |
|
I don't think we should run with --warp. --warp skips ticks (IIRC), which is why we have not used it to run rectests before (I added --warp to the CLI so I could run rectests faster, until I discovered this problem). |
|
|
||
| # Define your tests here (description:filename) | ||
| tests=( | ||
| "There is a 3 tick delay between jumps:JUMP_DELAY.REC" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dislike losing the rectest description as a side effect here, the description was very handy in quickly understanding what you broke when you started to see rectests fail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you mind if we embedded them in the .REC files as some sort of comment?
Then the rec controller can find them when an assert fails
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe? or just have a lookup table we can use to print a better description on a failure?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure we have room for an artitrary string in the REC
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i was thinking add another "move" that's just a comment about what we're about to test.
when the rec fails, the rec controller would rewind looking for the last rec test comment & print it (again).
or we'll do a lookup table as you suggest, have that loaded from a .txt file in the rectests folder
|
warp runs every dynamic tick. if warp skips any dynamic ticks, that's a bug. |
|
I believe warp IS buggy, but I don't have i
I believe warp IS buggy, I just can't prove it. |
All 60 rectests are now executed in a single process.
Benchmarks (msvc debug builds on windows with address sanitizer):
As an added bonus, running all 60 rectests in series is enough frames to quickly exhaust our renderer atlas:
