MG test for JDC on quick templates #931

plebhash · 2024-05-24T16:14:22Z

eventually this will close #923

this is where I stopped:

message-generator/mock/jds-mock-quick-templates.json is creating jd_messages that are leading to broken frame_builders.

STD OUT: EXECUTING ../../test/message-generator/mock/jds-mock-quick-templates.json
STD OUT: 
STD ERR: thread 'main' panicked at src/parser/frames.rs:40:44:
STD ERR: Missing messages message_id allocate_mining_job_token_success_0

test/message-generator/test/jdc-quick-template-update/jdc-quick-template-update.json needs:
- finish doc description with a list of steps that the test performs
- test action asserting SubmitShares.Success from pool
- more stuff? maybe I'm missing something, this todo list should not be seen as complete

github-actions · 2024-05-24T16:18:53Z

Bencher

Report	Thu, June 6, 2024 at 23:57:28 UTC
Project	Stratum v2 (SRI)
Branch	931/merge
Testbed	sv1

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client-submit-serialize	✅ (view plot)	7,196.20 (+3.57%)	7,349.31 (97.92%)
client-submit-serialize-deserialize	✅ (view plot)	7,972.50 (+1.38%)	8,324.48 (95.77%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle	✅ (view plot)	8,654.30 (+2.62%)	8,843.64 (97.86%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle	✅ (view plot)	897.58 (-0.10%)	926.65 (96.86%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize	✅ (view plot)	686.21 (-1.51%)	717.93 (95.58%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize	✅ (view plot)	246.42 (-0.67%)	255.17 (96.57%)
client-sv1-get-authorize/client-sv1-get-authorize	✅ (view plot)	157.02 (-0.09%)	162.45 (96.66%)
client-sv1-get-submit	✅ (view plot)	6,761.80 (+0.82%)	7,131.17 (94.82%)
client-sv1-get-subscribe/client-sv1-get-subscribe	✅ (view plot)	283.38 (+1.66%)	291.50 (97.22%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle	✅ (view plot)	751.84 (+0.64%)	776.17 (96.86%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize	✅ (view plot)	608.28 (-0.96%)	637.59 (95.40%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize	✅ (view plot)	206.89 (+0.08%)	220.03 (94.03%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-05-24T16:18:54Z

Bencher

Report	Thu, June 6, 2024 at 23:57:29 UTC
Project	Stratum v2 (SRI)
Branch	mg-test-jdc-quick-templates
Testbed	sv2

🚨 3 ALERTS: Threshold Boundary Limits exceeded!

Benchmark	Measure (units)	View	Value	Upper Boundary
client_sv2_handle_message_mining	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	87.11 (+19.14%)	81.73 (106.57%)
client_sv2_open_channel_serialize_deserialize	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	568.17 (+48.73%)	453.24 (125.36%)
client_sv2_setup_connection	Latency (nanoseconds (ns))	🚨 (view plot \| view alert)	191.29 (+16.14%)	178.03 (107.45%)

Click to view all benchmark results

Benchmark	Latency	Latency Results nanoseconds (ns) \| (Δ%)	Latency Upper Boundary nanoseconds (ns) \| (%)
client_sv2_handle_message_common	✅ (view plot)	44.77 (+0.48%)	45.25 (98.94%)
client_sv2_handle_message_mining	🚨 (view plot \| view alert)	87.11 (+19.14%)	81.73 (106.57%)
client_sv2_mining_message_submit_standard	✅ (view plot)	14.65 (-0.02%)	14.69 (99.71%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	267.92 (+1.33%)	284.12 (94.30%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	594.32 (+0.01%)	628.06 (94.63%)
client_sv2_open_channel	✅ (view plot)	161.89 (-2.22%)	171.33 (94.49%)
client_sv2_open_channel_serialize	✅ (view plot)	290.67 (+2.54%)	293.47 (99.05%)
client_sv2_open_channel_serialize_deserialize	🚨 (view plot \| view alert)	568.17 (+48.73%)	453.24 (125.36%)
client_sv2_setup_connection	🚨 (view plot \| view alert)	191.29 (+16.14%)	178.03 (107.45%)
client_sv2_setup_connection_serialize	✅ (view plot)	475.27 (+0.03%)	505.39 (94.04%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	1,026.60 (+5.66%)	1,041.21 (98.60%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-05-24T16:18:54Z

Bencher

Report	Thu, June 6, 2024 at 23:57:23 UTC
Project	Stratum v2 (SRI)
Branch	mg-test-jdc-quick-templates
Testbed	sv1

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
get_authorize	✅ (view plot)	8,440.00 (+0.08%)	8,724.54 (96.74%)	✅ (view plot)	3,746.00 (+0.15%)	3,856.71 (97.13%)	✅ (view plot)	5,250.00 (+0.10%)	5,403.58 (97.16%)	✅ (view plot)	8.00 (+0.23%)	10.43 (76.71%)	✅ (view plot)	90.00 (+0.04%)	93.91 (95.83%)
get_submit	✅ (view plot)	95,545.00 (-0.01%)	96,151.12 (99.37%)	✅ (view plot)	59,439.00 (-0.05%)	59,783.10 (99.42%)	✅ (view plot)	85,350.00 (-0.07%)	85,842.77 (99.43%)	✅ (view plot)	58.00 (+4.71%)	62.54 (92.74%)	✅ (view plot)	283.00 (+0.32%)	287.79 (98.33%)
get_subscribe	✅ (view plot)	8,011.00 (+0.41%)	8,275.59 (96.80%)	✅ (view plot)	2,841.00 (+0.40%)	2,940.93 (96.60%)	✅ (view plot)	3,966.00 (+0.32%)	4,100.80 (96.71%)	✅ (view plot)	18.00 (+10.20%)	19.80 (90.93%)	✅ (view plot)	113.00 (+0.30%)	117.01 (96.57%)
serialize_authorize	✅ (view plot)	12,197.00 (-0.06%)	12,504.04 (97.54%)	✅ (view plot)	5,317.00 (+0.11%)	5,427.71 (97.96%)	✅ (view plot)	7,412.00 (+0.06%)	7,566.90 (97.95%)	✅ (view plot)	12.00 (+10.58%)	13.46 (89.16%)	✅ (view plot)	135.00 (-0.37%)	140.19 (96.30%)
serialize_deserialize_authorize	✅ (view plot)	24,472.00 (+0.02%)	24,706.23 (99.05%)	✅ (view plot)	9,898.00 (-0.02%)	10,027.94 (98.70%)	✅ (view plot)	13,957.00 (-0.07%)	14,153.66 (98.61%)	✅ (view plot)	38.00 (+3.43%)	41.62 (91.30%)	✅ (view plot)	295.00 (+0.08%)	297.48 (99.17%)
serialize_deserialize_handle_authorize	✅ (view plot)	30,157.00 (+0.05%)	30,332.49 (99.42%)	✅ (view plot)	12,101.00 (+0.05%)	12,211.71 (99.09%)	✅ (view plot)	17,122.00 (+0.03%)	17,282.25 (99.07%)	✅ (view plot)	59.00 (+0.41%)	64.73 (91.15%)	✅ (view plot)	364.00 (+0.07%)	366.61 (99.29%)
serialize_deserialize_handle_submit	✅ (view plot)	126,392.00 (-0.02%)	127,039.63 (99.49%)	✅ (view plot)	73,224.00 (-0.04%)	73,625.57 (99.45%)	✅ (view plot)	104,942.00 (-0.05%)	105,519.55 (99.45%)	✅ (view plot)	125.00 (+3.64%)	131.59 (94.99%)	✅ (view plot)	595.00 (+0.03%)	599.22 (99.30%)
serialize_deserialize_handle_subscribe	✅ (view plot)	27,485.00 (+0.11%)	27,602.67 (99.57%)	✅ (view plot)	9,643.00 (+0.12%)	9,742.93 (98.97%)	✅ (view plot)	13,635.00 (+0.09%)	13,776.88 (98.97%)	✅ (view plot)	68.00 (+2.91%)	73.94 (91.97%)	✅ (view plot)	386.00 (+0.06%)	388.54 (99.35%)
serialize_deserialize_submit	✅ (view plot)	115,019.00 (-0.06%)	115,661.96 (99.44%)	✅ (view plot)	68,001.00 (-0.08%)	68,408.16 (99.40%)	✅ (view plot)	97,554.00 (-0.10%)	98,158.47 (99.38%)	✅ (view plot)	70.00 (+0.56%)	75.25 (93.03%)	✅ (view plot)	489.00 (+0.19%)	492.41 (99.31%)
serialize_deserialize_subscribe	✅ (view plot)	22,904.00 (+0.13%)	23,114.43 (99.09%)	✅ (view plot)	8,195.00 (+0.11%)	8,298.75 (98.75%)	✅ (view plot)	11,539.00 (+0.09%)	11,684.41 (98.76%)	✅ (view plot)	40.00 (+1.17%)	44.06 (90.78%)	✅ (view plot)	319.00 (+0.15%)	321.52 (99.22%)
serialize_submit	✅ (view plot)	99,862.00 (-0.04%)	100,479.13 (99.39%)	✅ (view plot)	61,483.00 (-0.05%)	61,832.56 (99.43%)	✅ (view plot)	88,197.00 (-0.06%)	88,696.38 (99.44%)	✅ (view plot)	58.00 (+3.57%)	62.08 (93.43%)	✅ (view plot)	325.00 (+0.09%)	329.26 (98.71%)
serialize_subscribe	✅ (view plot)	11,340.00 (+0.11%)	11,611.12 (97.67%)	✅ (view plot)	4,188.00 (+0.27%)	4,287.93 (97.67%)	✅ (view plot)	5,825.00 (+0.23%)	5,960.13 (97.73%)	✅ (view plot)	18.00 (+9.46%)	19.06 (94.44%)	✅ (view plot)	155.00 (-0.17%)	159.83 (96.98%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

github-actions · 2024-05-24T16:18:55Z

Bencher

Report	Thu, June 6, 2024 at 23:57:29 UTC
Project	Stratum v2 (SRI)
Branch	mg-test-jdc-quick-templates
Testbed	sv2

Click to view all benchmark results

Benchmark	Estimated Cycles	Estimated Cycles Results estimated cycles \| (Δ%)	Estimated Cycles Upper Boundary estimated cycles \| (%)	Instructions	Instructions Results instructions \| (Δ%)	Instructions Upper Boundary instructions \| (%)	L1 Accesses	L1 Accesses Results accesses \| (Δ%)	L1 Accesses Upper Boundary accesses \| (%)	L2 Accesses	L2 Accesses Results accesses \| (Δ%)	L2 Accesses Upper Boundary accesses \| (%)	RAM Accesses	RAM Accesses Results accesses \| (Δ%)	RAM Accesses Upper Boundary accesses \| (%)
client_sv2_handle_message_common	✅ (view plot)	2,057.00 (+0.40%)	2,125.94 (96.76%)	✅ (view plot)	473.00 (+0.50%)	486.49 (97.23%)	✅ (view plot)	737.00 (+0.64%)	755.63 (97.53%)	✅ (view plot)	5.00 (-26.59%)	10.38 (48.17%)	✅ (view plot)	37.00 (+0.98%)	38.57 (95.94%)
client_sv2_handle_message_mining	✅ (view plot)	8,207.00 (+0.13%)	8,337.12 (98.44%)	✅ (view plot)	2,137.00 (+0.47%)	2,171.53 (98.41%)	✅ (view plot)	3,162.00 (+0.56%)	3,216.55 (98.30%)	✅ (view plot)	36.00 (-6.42%)	43.64 (82.49%)	✅ (view plot)	139.00 (+0.11%)	142.00 (97.89%)
client_sv2_mining_message_submit_standard	✅ (view plot)	6,242.00 (-0.52%)	6,389.22 (97.70%)	✅ (view plot)	1,750.00 (+0.03%)	1,763.31 (99.25%)	✅ (view plot)	2,557.00 (+0.12%)	2,576.30 (99.25%)	✅ (view plot)	16.00 (-7.73%)	22.16 (72.19%)	✅ (view plot)	103.00 (-0.80%)	106.94 (96.32%)
client_sv2_mining_message_submit_standard_serialize	✅ (view plot)	14,757.00 (-0.14%)	15,036.28 (98.14%)	✅ (view plot)	4,694.00 (+0.01%)	4,707.31 (99.72%)	✅ (view plot)	6,757.00 (+0.04%)	6,775.59 (99.73%)	✅ (view plot)	46.00 (-2.40%)	51.67 (89.02%)	✅ (view plot)	222.00 (-0.22%)	230.02 (96.51%)
client_sv2_mining_message_submit_standard_serialize_deserialize	✅ (view plot)	27,454.00 (-0.09%)	27,847.69 (98.59%)	✅ (view plot)	10,545.00 (+0.03%)	10,558.67 (99.87%)	✅ (view plot)	15,344.00 (+0.03%)	15,361.21 (99.89%)	✅ (view plot)	84.00 (+0.45%)	89.09 (94.29%)	✅ (view plot)	334.00 (-0.26%)	345.44 (96.69%)
client_sv2_open_channel	✅ (view plot)	4,527.00 (+0.73%)	4,612.44 (98.15%)	✅ (view plot)	1,461.00 (+0.06%)	1,474.58 (99.08%)	✅ (view plot)	2,152.00 (-0.03%)	2,172.99 (99.03%)	✅ (view plot)	13.00 (+8.16%)	15.31 (84.94%)	✅ (view plot)	66.00 (+1.24%)	68.20 (96.78%)
client_sv2_open_channel_serialize	✅ (view plot)	14,232.00 (+0.07%)	14,460.58 (98.42%)	✅ (view plot)	5,064.00 (+0.02%)	5,077.58 (99.73%)	✅ (view plot)	7,317.00 (-0.01%)	7,339.44 (99.69%)	✅ (view plot)	39.00 (+4.92%)	41.55 (93.86%)	✅ (view plot)	192.00 (+0.03%)	198.82 (96.57%)
client_sv2_open_channel_serialize_deserialize	✅ (view plot)	22,700.00 (+0.25%)	23,030.59 (98.56%)	✅ (view plot)	7,987.00 (+0.05%)	8,001.24 (99.82%)	✅ (view plot)	11,615.00 (+0.02%)	11,636.77 (99.81%)	✅ (view plot)	75.00 (+2.53%)	81.32 (92.23%)	✅ (view plot)	306.00 (+0.43%)	315.12 (97.11%)
client_sv2_setup_connection	✅ (view plot)	4,699.00 (0.00%)	4,765.26 (98.61%)	✅ (view plot)	1,502.00 (+0.06%)	1,515.58 (99.10%)	✅ (view plot)	2,279.00 (+0.09%)	2,300.05 (99.08%)	✅ (view plot)	8.00 (-10.55%)	13.12 (60.95%)	✅ (view plot)	68.00 (+0.11%)	69.60 (97.70%)
client_sv2_setup_connection_serialize	✅ (view plot)	16,338.00 (+0.40%)	16,482.22 (99.12%)	✅ (view plot)	5,963.00 (+0.01%)	5,976.58 (99.77%)	✅ (view plot)	8,653.00 (-0.02%)	8,677.56 (99.72%)	✅ (view plot)	46.00 (+3.22%)	48.77 (94.32%)	✅ (view plot)	213.00 (+0.81%)	217.16 (98.08%)
client_sv2_setup_connection_serialize_deserialize	✅ (view plot)	35,638.00 (+0.28%)	35,752.88 (99.68%)	✅ (view plot)	14,814.00 (+0.03%)	14,828.24 (99.90%)	✅ (view plot)	21,748.00 (-0.01%)	21,772.47 (99.89%)	✅ (view plot)	104.00 (+4.49%)	112.50 (92.44%)	✅ (view plot)	382.00 (+0.59%)	384.49 (99.35%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

plebhash · 2024-05-24T16:57:59Z

test/message-generator/mock/tp-mock-quick-template.json

+                }
+            ],
+            "actiondoc": "This action checks that new template2 and new prev hash2 is received"
+        }


todo: missing actions for RequestTransactionData (2x)

plebhash · 2024-05-24T16:59:30Z

test/message-generator/test/jdc-quick-template-update/jdc-quick-template-update.json

+        "Launch real JDC",
+        "Mock TP",
+        "Mock JDS",
+        "todo: describe message sequence"


todo: we need to finish the doc description of the main test

plebhash · 2024-05-24T17:00:58Z

test/message-generator/test/jdc-quick-template-update/jdc-quick-template-update.json

+                }
+            ],
+          "actiondoc": "This action sends SetupConnection.Success and check that a open_extended_mining_channel is received"
+        }


todo: action asserting SubmitShares.Success from pool (maybe after waiting for some time, so JDS+JDC+TP can do all their stuff

rrybarczyk · 2024-07-17T13:28:36Z

Replaced with #1026.

plebhash force-pushed the mg-test-jdc-quick-templates branch 2 times, most recently from 03a209f to b9ef44d Compare May 24, 2024 16:54

plebhash commented May 24, 2024

View reviewed changes

plebhash force-pushed the mg-test-jdc-quick-templates branch 3 times, most recently from 508fdfe to 6c72fb1 Compare May 24, 2024 20:42

plebhash linked an issue Jun 4, 2024 that may be closed by this pull request

Add MG test for false fallback activation #923

Open

bootstrap MG test for issue stratum-mining#923

66799d8

plebhash force-pushed the mg-test-jdc-quick-templates branch from 6c72fb1 to 66799d8 Compare June 6, 2024 23:53

plebhash mentioned this pull request Jun 29, 2024

Add MG test for false fallback activation #923

Open

rrybarczyk closed this Jul 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MG test for JDC on quick templates #931

MG test for JDC on quick templates #931

plebhash commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

plebhash May 24, 2024

plebhash May 24, 2024

plebhash May 24, 2024

rrybarczyk commented Jul 17, 2024

MG test for JDC on quick templates #931

MG test for JDC on quick templates #931

Conversation

plebhash commented May 24, 2024 • edited Loading

github-actions bot commented May 24, 2024 • edited Loading

Bencher

github-actions bot commented May 24, 2024 • edited Loading

Bencher

github-actions bot commented May 24, 2024 • edited Loading

Bencher

github-actions bot commented May 24, 2024 • edited Loading

Bencher

plebhash May 24, 2024

Choose a reason for hiding this comment

plebhash May 24, 2024

Choose a reason for hiding this comment

plebhash May 24, 2024

Choose a reason for hiding this comment

rrybarczyk commented Jul 17, 2024

plebhash commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading

github-actions bot commented May 24, 2024 •

edited

Loading