test_models: add tx fuzzy test #32897

bongbui321 · 2024-07-03T14:42:12Z

Uses FuzzyGenerator to generate fuzzy CarControl messages

hypothesis example generation is extremly slow.

Nissan speed: 4 wheels in OP and 2 rear wheels in panda
Nissan steering angle out of -1311 - 1310 -> CANPacker wrong value
Tesla has clip to current steering angle

* stash * staaaaash * clean up * more clean up * fix fix * fix * stash clean up * more * this is just simpler to understand * explicitly check

…ctionary (commaai#32592) Optimize gear shifter parsing for improved performance

Fingerprint for 2017 Forester

* Loading torque data only once and reusing it across function calls. * apply review * still check only one entry * fix that --------- Co-authored-by: Shane Smiskol <[email protected]>

use non-2020 specs

* improve performance * remove DEPRECATED * formatting * catch kjException --------- Co-authored-by: Shane Smiskol <[email protected]>

* improve prev event counters * just rename --------- Co-authored-by: Shane Smiskol <[email protected]>

increase lag in optima to 0.2

remove redundant

* squash * fix doc * compile device * compile device * Update ref * add msgq to precommit exclusions * No service ports * fix compile * address comments * More comments * Delete logger * Update opendbc * Linting * bump msgq

* Dead cereal stuff * Dead code * also dead * More cleanup

from commaai@fe7d342

* rm generate_javascript.sh * Unused

* blacklist more files from release * dbcs * fix * just use the lfs * rm that too * cleanup * cleanup

* card: remove old canRcvtimeout field * deprecate

* bump * bump2 * bump3

* remove wrapper that is functools.cache * format

* is this even used? * remove

* pandad: fix loopback test * clear all params * try this --------- Co-authored-by: Comma Device <[email protected]>

@AlexandreSato

…Non-US only) 2020-23" entry (commaai#32658) The term "Non-US" in cars.md for the "Corolla Hybrid (Non-US only) 2020-23" is not precise enough and should be changed to South America Users may think "Non-US" includes Europe or Canada, which is not the case for this entry. It has two issues: * The 2023 Corolla Hybrid in Europe and Canada include a new Toyota Safety Sense 3.0 that has yet to be supported due to it being a new version and the presence of Toyota Security Key. * The 2020-2022 Corolla Hybrid in Europe is the same as the 2020-2022 Corolla Hybrid in North America, which is supported by Openpilot and has full-speed ACC due to the presence of an electronic parking brake which is not present in the 2020-2022 Corolla Hybrid in South America and the reason why it is listed as not having full-speed ACC. The entry should be reverted to at least "Corolla Hybrid (South America) 2020-23" which was proposed but not merged in the pull request: commaai@28454c0 Pull request for that change that does not include that commit: commaai#26943 Examples of confusion throughout the community: comma.ai Discord: https://discord.com/channels/469524606043160576/524327905937850394/1235264758580772996 https://discord.com/channels/469524606043160576/524327905937850394/1228090600272691370 https://discord.com/channels/469524606043160576/954493346250887168/1209952008329633813 https://discord.com/channels/469524606043160576/524327905937850394/1182093384647721140 https://discord.com/channels/469524606043160576/954493346250887168/1147751657543848017 openpilot community Discord: https://discord.com/channels/771493367246094347/771495215570747403/1247737844727021629 > I think that the supported cars list (https://github.com/commaai/openpilot/blob/master/docs/CARS.md) is incorrect. > > I have a UK 2020 Toyota Corolla Hybrid > > In the cars list it's listed as "no accel below" 17 mph and no resume from > stop. > > This is not my experience. For me it works just fine down to a complete stop and it can resume from a stop too. > > CC: @AlexandreSato

* Replace markdown-it-py with small function in common/ * simple test * unused * lock * linting fixes

)

* add livepose * Formatting * Add to sevices * Update locationd to publish livePose * Remove fields and increase decimation * Fix field indices * Remove the line * Add livePose to pubmaster * Fix llk decimation * Update ref commit * XYZ measurements instead of lists * Update locationd * Update ref commit * Lower the qlog size in test_onroad * Update lower and upper boundary --------- Co-authored-by: Kacper Rączy <[email protected]>

reduce cameraOdometry decimation

change longitudinalPlan decimation

bongbui321 · 2024-07-18T03:46:34Z

Might merge tx fuzzy with carstate fuzzy.

Occasionally catches commaai/panda#1948. Not entirely sure why hypothesis doesn't consistenly repro that failed data generation

* decimate option * clean up * check exists

* tici: reset panda boot0 * ooops

* Fix msgq context overrides * Remove unneccesary changes * add reset_context helper * Remove redundant import

…d_fuzzy_tx

bongbui321 · 2024-07-19T21:37:52Z

@sshane ready for review

adeebshihadeh

I don't think we merged all your bugfixes yet, so how is this passing? Are we not getting good coverage on a single run?

If so, what would it take to hit all the known bugs on a single run? How long would that take in CI?

adeebshihadeh · 2024-07-23T23:58:55Z

selfdrive/car/tests/test_models.py

@@ -323,10 +329,13 @@ def test_car_controller(car_control):
  @settings(max_examples=MAX_EXAMPLES, deadline=None,
            phases=(Phase.reuse, Phase.generate, Phase.shrink))
  @given(data=st.data())
-  def test_panda_safety_carstate_fuzzy(self, data):
+  def test_panda_safety_carstate_tx_fuzzy(self, data):


let's split the new TX stuff into a separate test case

oh is this merged for speed?

bongbui321 · 2024-07-25T20:54:12Z

I don't think we merged all your bugfixes yet, so how is this passing? Are we not getting good coverage on a single run?

I don't think we merged all your bugfixes yet, so how is this passing? Are we not getting good coverage on a single run?

Nissan max steering angle [issue](https://github.com/commaai/openpilot/issues/32978) is opened and included in this PR

Tesla [issue](https://github.com/commaai/openpilot/issues/32999): This might happen with a few runs, and I suspect that if run locally, hypothesis will try to recreate that failing examples pretty consistently. That requires a more thoughtful fix, which I'm hesitant to open a fix PR since I'm not familiar with the expected behaviour of the platform

The Nissan speed issue is addressed here:

    # Nissan calculate vEgoRaw with avg speed of 4 wheels
    # while panda get vehicle_speed from rear wheels only,
    # so there might be a big enough difference between the two
    if self.CP.carName == "nissan":
      panda_vehicle_speed_last = self.safety.get_vehicle_speed_last() / VEHICLE_SPEED_FACTOR
      if abs(self.CI.CS.out.vEgoRaw - panda_vehicle_speed_last) > 0.2:
        cs_msg = self.CI.CS.out.to_dict()
        cs_msg["vEgoRaw"] = panda_vehicle_speed_last
        self.CI.CS.out = car.CarState.new_message(**cs_msg).as_reader()

This realistically shouldn't happen front wheels and rear wheel speed should be close

adeebshihadeh · 2024-09-01T17:56:27Z

Can you rebase?

maxime-desroches · 2024-09-04T06:31:03Z

This PR was closed because of a git history rewrite.
Please read #33399 for what to do with your fork and your PRs.

adeebshihadeh and others added 30 commits June 5, 2024 17:32

fix linter

260e18f

Restore lateral limits report (commaai#32640)

7cfd91b

* stash * staaaaash * clean up * more clean up * fix fix * fix * stash clean up * more * this is just simpler to understand * explicitly check

car/interfaces.py: optimize parse_gear_shifter() with predefined di…

91fd918

…ctionary (commaai#32592) Optimize gear shifter parsing for improved performance

Fingerprint for 2017 Subaru Forester (commaai#32584)

a418c9e

Fingerprint for 2017 Forester

improve get_torque_params (commaai#32586)

045b9f2

* Loading torque data only once and reusing it across function calls. * apply review * still check only one entry * fix that --------- Co-authored-by: Shane Smiskol <[email protected]>

Genesis G70 2020: use common specs (commaai#32641)

ae37509

use non-2020 specs

fuzzy_generation.py: improve performance (commaai#32591)

3a43f5d

* improve performance * remove DEPRECATED * formatting * catch kjException --------- Co-authored-by: Shane Smiskol <[email protected]>

events.py: rename event counters (commaai#32596)

983063c

* improve prev event counters * just rename --------- Co-authored-by: Shane Smiskol <[email protected]>

[Lag] Increase Optima Lag (commaai#32583)

e8857a6

increase lag in optima to 0.2

CI/tools_test: remove redundant git lfs pull (commaai#32642)

4c54977

remove redundant

Split cereal into cereal/msgq (commaai#32631)

e71ec4e

* squash * fix doc * compile device * compile device * Update ref * add msgq to precommit exclusions * No service ports * fix compile * address comments * More comments * Delete logger * Update opendbc * Linting * bump msgq

Msgq cleanups (commaai#32643)

83209e0

* Dead cereal stuff * Dead code * also dead * More cleanup

update TOTAL_SCONS_NODES

280a284

from commaai@fe7d342

controlsd: remove last mention of CAN (commaai#32646)

a7ed190

update msgq repo url (commaai#32647)

ccd26de

More cereal cleanups (commaai#32644)

e6caaf4

* rm generate_javascript.sh * Unused

blacklist more files from release (commaai#32616)

5c3c06d

* blacklist more files from release * dbcs * fix * just use the lfs * rm that too * cleanup * cleanup

build_release: run tests with pytest

2bad3c2

card: remove old canRcvtimeout field (commaai#32650)

b42b156

* card: remove old canRcvtimeout field * deprecate

bump opendbc (commaai#32651)

9ce380d

* bump * bump2 * bump3

remove old cache wrapper (commaai#32653)

451e171

* remove wrapper that is functools.cache * format

test_encoder: adjust file size tolerance

b514720

remove old fingerprints test (commaai#32654)

90523b3

* is this even used? * remove

pandad: fix loopback test (commaai#32656)

57f21ab

* pandad: fix loopback test * clear all params * try this --------- Co-authored-by: Comma Device <[email protected]>

tools/sim: remove special docker image (commaai#32659)

541b5cb

Replace markdown-it-py with small function in common/ (commaai#32664)

5265d13

* Replace markdown-it-py with small function in common/ * simple test * unused * lock * linting fixes

Update BOUNTIES.md re. Toyota SecOC and community bounty. (commaai#32661

5e1c54a

)

python dependency notes (commaai#32668)

284512b

pandad: fix receive buffer size on failed transfer (commaai#32670)

1093a4a

haraschax and others added 2 commits July 17, 2024 16:17

increase cameraOdometry decimation (commaai#33010)

7f9cdf8

reduce cameraOdometry decimation

bongbui321 mentioned this pull request Jul 18, 2024

add set_gas_pressed_prev() helper commaai/panda#1986

Merged

bongbui321 and others added 3 commits July 17, 2024 23:32

almost

0d07b2d

remove that constant

0477e20

increase longitudinalPlan decimation (commaai#33009)

3c48a61

change longitudinalPlan decimation

sshane and others added 9 commits July 17, 2024 21:57

qlog_size: decimate rlog option (commaai#33011)

f31ad97

* decimate option * clean up * check exists

tici: set STM_BOOT0 in reset (commaai#33015)

560e37c

* tici: reset panda boot0 * ooops

nightly build takes longer than this

fd5549b

Fix msgq context overrides (commaai#33014)

c96dbd5

* Fix msgq context overrides * Remove unneccesary changes * add reset_context helper * Remove redundant import

passes all

c27929f

Merge branch 'master' of https://github.com/commaai/openpilot into ad…

6e35ed6

…d_fuzzy_tx

bump panda and add comment to tx_fuzzy test

b5c9a80

move import down

0f58e96

max steering angle for nissan

894e653

bongbui321 force-pushed the add_fuzzy_tx branch 2 times, most recently from 057ec11 to 894e653 Compare July 19, 2024 17:28

bongbui321 added 3 commits July 19, 2024 16:59

merge 2 tests

b43e66a

cleanup

99788c8

unbump that

a55d514

bongbui321 marked this pull request as ready for review July 19, 2024 21:37

adeebshihadeh reviewed Jul 24, 2024

View reviewed changes

adeebshihadeh closed this Sep 4, 2024

adeebshihadeh force-pushed the master branch from 29af572 to 0058ea5 Compare September 4, 2024 05:57

bongbui321 mentioned this pull request Sep 19, 2024

test_models: add fuzzy tx test #33587

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test_models: add tx fuzzy test #32897

test_models: add tx fuzzy test #32897

bongbui321 commented Jul 3, 2024 •

edited

Loading

bongbui321 commented Jul 18, 2024

bongbui321 commented Jul 19, 2024

adeebshihadeh left a comment •

edited

Loading

adeebshihadeh Jul 23, 2024

adeebshihadeh Jul 24, 2024

bongbui321 commented Jul 25, 2024

adeebshihadeh commented Sep 1, 2024

maxime-desroches commented Sep 4, 2024

test_models: add tx fuzzy test #32897

test_models: add tx fuzzy test #32897

Conversation

bongbui321 commented Jul 3, 2024 • edited Loading

bongbui321 commented Jul 18, 2024

bongbui321 commented Jul 19, 2024

adeebshihadeh left a comment • edited Loading

Choose a reason for hiding this comment

adeebshihadeh Jul 23, 2024

Choose a reason for hiding this comment

adeebshihadeh Jul 24, 2024

Choose a reason for hiding this comment

bongbui321 commented Jul 25, 2024

adeebshihadeh commented Sep 1, 2024

maxime-desroches commented Sep 4, 2024

bongbui321 commented Jul 3, 2024 •

edited

Loading

adeebshihadeh left a comment •

edited

Loading