Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jemalloc assertion that time goes backwards #249

Closed
arichardson opened this issue Jan 16, 2018 · 6 comments
Closed

Jemalloc assertion that time goes backwards #249

arichardson opened this issue Jan 16, 2018 · 6 comments
Assignees

Comments

@arichardson
Copy link
Member

root@qemu-cheri256-alr48:~ # gdb-mips -ex=r --args /root/tst_qtendian
uchar *in;GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "mips64-unknown-freebsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /root/tst_qtendian...<jemalloc>: jemalloc_arena.c:821: Failed assertion: "nstime_compare(&decay->epoch, &time) <= 0"

This sounds like time is going backwards again:
Probably related to
CTSRD-CHERI/qemu#44
CTSRD-CHERI/qemu#54
CTSRD-CHERI/qemu#55

@arichardson
Copy link
Member Author

arichardson commented Jan 19, 2018

Pinning QEMU to a single CPU didn't help here so it is not related to QEMU using different counts across CPUs.

I think the issue here might be (if I understand sys/mips/tick.c correctly) that the timer is only 32 bits. It seems like all it does is read the low 32 bits of the count register (https://github.com/CTSRD-CHERI/cheribsd/blob/master/sys/mips/mips/tick.c#L205).
This suggest we should se a wraparound after 0xffffffff / (50× 1000 × 1000) = 85.5 seconds. Probably because of CTSRD-CHERI/qemu#55 it actually happens after about 1 minute and 9 seconds.

arichardson added a commit to CTSRD-CHERI/qemu that referenced this issue Jan 25, 2018
This should hopefully address CTSRD-CHERI/cheribsd#249. At the very least
the test_cp0_wait.py now actually waits exactly 1 second if CP0_Compare
gets set to CP0_Count + 50 million.
arichardson added a commit to CTSRD-CHERI/qemu that referenced this issue Jan 25, 2018
…econds)

FreeBSD writes 0xffffffff when it is not waiting for a timer which could
previously cause a very early timer interrupt if the wrapped CP0_count was
close to UINT32_MAX. This is not a correct fix for CTSRD-CHERI/cheribsd#249,
but at least I can now actually get beyond the reading symbols phase in GDB.
LawrenceEsswood pushed a commit to CTSRD-CHERI/qemu that referenced this issue Mar 15, 2018
This should hopefully address CTSRD-CHERI/cheribsd#249. At the very least
the test_cp0_wait.py now actually waits exactly 1 second if CP0_Compare
gets set to CP0_Count + 50 million.
LawrenceEsswood pushed a commit to CTSRD-CHERI/qemu that referenced this issue Mar 15, 2018
…econds)

FreeBSD writes 0xffffffff when it is not waiting for a timer which could
previously cause a very early timer interrupt if the wrapped CP0_count was
close to UINT32_MAX. This is not a correct fix for CTSRD-CHERI/cheribsd#249,
but at least I can now actually get beyond the reading symbols phase in GDB.
@arichardson
Copy link
Member Author

Closing this since with workaround I added to QEMU (CTSRD-CHERI/qemu@fd6f754) I can now run gdb without this assertion triggering.

@arichardson
Copy link
Member Author

Seems like the workaround is not sufficient: Just saw this when running the testsuite (in
lib.libc.db.db_test.btree_tricky_page_split):

Assertion failed: (nstime_compare(&decay->epoch, &time) <= 0), function arena_maybe_decay, file jemalloc_arena.c, line 821.

@arichardson arichardson reopened this Apr 26, 2018
@arichardson arichardson changed the title Jemalloc assertion when running GDB Jemalloc assertion that time goes backwards Apr 26, 2018
@brooksdavis
Copy link
Member

Worked around in 52ec185.

@trasz
Copy link
Contributor

trasz commented Sep 16, 2020

FWIW, it's breaking MIPS runs on FreeBSD CI: https://ci.freebsd.org/job/FreeBSD-head-mips64-test/14/console. Would it make sense to upstream it, or perhaps add as a port-local patch?

@jrtc27
Copy link
Member

jrtc27 commented Sep 16, 2020

Or we fix CLOCK_MONOTONIC to not go backwards? Having QEMU's time go backwards is one thing, but having CLOCK_MONOTONIC go backwards as a result is another and a true FreeBSD bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants