Skip to content

Conversation

@Nopey
Copy link
Member

@Nopey Nopey commented Sep 25, 2025

All 60 rectests are now executed in a single process.

Benchmarks (msvc debug builds on windows with address sanitizer):

4m9.633s (master)
1m18.524s (master, but with --warp turned on)
0m25.373s (this branch, running all the RECs in series)

As an added bonus, running all 60 rectests in series is enough frames to quickly exhaust our renderer atlas:
image

@Nopey Nopey marked this pull request as ready for review September 25, 2025 08:41
@Nopey
Copy link
Member Author

Nopey commented Sep 25, 2025

timings in CI:
master: 3m 47s (6m 30s incl. build, menu tests, etc..)
this branch: 31s (5m 1s incl build, menu tests, etc..)

@Vagabond
Copy link
Member

I don't think we should run with --warp. --warp skips ticks (IIRC), which is why we have not used it to run rectests before (I added --warp to the CLI so I could run rectests faster, until I discovered this problem).


# Define your tests here (description:filename)
tests=(
"There is a 3 tick delay between jumps:JUMP_DELAY.REC"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dislike losing the rectest description as a side effect here, the description was very handy in quickly understanding what you broke when you started to see rectests fail.

Copy link
Member Author

@Nopey Nopey Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind if we embedded them in the .REC files as some sort of comment?
Then the rec controller can find them when an assert fails

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe? or just have a lookup table we can use to print a better description on a failure?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we have room for an artitrary string in the REC

Copy link
Member Author

@Nopey Nopey Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was thinking add another "move" that's just a comment about what we're about to test.
when the rec fails, the rec controller would rewind looking for the last rec test comment & print it (again).

or we'll do a lookup table as you suggest, have that loaded from a .txt file in the rectests folder

@Nopey
Copy link
Member Author

Nopey commented Sep 25, 2025

warp runs every dynamic tick.
the timing between dynamic ticks and static ticks is not guaranteed (on any speed).

if warp skips any dynamic ticks, that's a bug.

@Vagabond
Copy link
Member

I believe warp IS buggy, but I don't have i

warp runs every dynamic tick. the timing between dynamic ticks and static ticks is not guaranteed (on any speed).

if warp skips any dynamic ticks, that's a bug.

I believe warp IS buggy, I just can't prove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants