-
Notifications
You must be signed in to change notification settings - Fork 177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add EpochExtended
and EpochFallbackModeExited
events
#6062
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature/efm-recovery #6062 +/- ##
=====================================================
Coverage 41.68% 41.69%
=====================================================
Files 1971 1971
Lines 139114 139140 +26
=====================================================
+ Hits 57991 58010 +19
- Misses 75069 75080 +11
+ Partials 6054 6050 -4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
it is covered by new tests under TestEmergencyEpochFallback
EpochExtended
and EpochFallbackModeExited
events
I have similar feelings. Additionally there is a related problem which I am not so sure how to handle. When we are in EFM but next epoch has been committed and we enter that epoch then the finalization logic detects it entering new epoch and emits all relevant events such as starting staking phase which should be tweaked when in EFM. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice, really like improvements in naming, code structure and new events on finalization. 🎸
I am planning to handle this in #6092, since we'll have the new phase that represents this state. Mainly I was planning to emit an event for starting |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very nice. Largely stylistic suggestions. Only algorithmic change would be my request to strongly follow the pattern of information-driven systems in the epoch notifications (👉 comment).
Co-authored-by: Alexander Hentschel <[email protected]>
commit 45f80fe022a645b6247564a177883ecd8c954d8b Merge: 08929bba55 c5123c9487 Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 07:57:31 2024 -0700 Merge pull request #6062 from onflow/jord/6013-events-metrics Add `EpochExtended` and `EpochFallbackModeExited` events commit c5123c9487e7bcdeaf26a04fe7cda87813fc3c18 Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 07:28:43 2024 -0700 specify precise expectations for epoch extensions in tests commit 5fc589bdd3252e4e8a4863ab77e55474b325637d Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 07:07:38 2024 -0700 update tests for interface change commit 5d65a0773c0cc1b53310aba888a5ccb92d5634fb Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 06:58:33 2024 -0700 update mocks commit 5d91f06665a6aa4055398df8addc5b823a1164c0 Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 06:52:56 2024 -0700 add epoch/header context to new events commit 19ccd4e9389f13320aa069f543bfef9826af728f Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 06:40:02 2024 -0700 update phase/finalview metrics on epoch transition commit 0f0022e1608768d78c16532da881fda4f923feb5 Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 06:29:42 2024 -0700 fix typo commit 8a36120af51ea7e40330b10807740e5b3d268427 Author: Jordan Schalm <[email protected]> Date: Thu Jun 20 06:29:03 2024 -0700 Apply suggestions from code review Co-authored-by: Alexander Hentschel <[email protected]> commit a9b2df064dae419aa562b1b640c9e83567f8b22d Author: Jordan Schalm <[email protected]> Date: Wed Jun 19 14:34:32 2024 -0700 add todos for issue #6123 commit b30dc2dfc68541308d7779d1e2c642e89f5a44e9 Merge: 02e9d9b6a5 08929bba55 Author: Jordan Schalm <[email protected]> Date: Mon Jun 17 11:41:00 2024 -0700 Merge branch 'feature/efm-recovery' into jord/6013-events-metrics commit 08929bba5582b5ad874f141e0178d86a4e7ca1c6 Merge: 29a79d06ed 6fd2ec4b44 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 12:05:24 2024 -0400 Merge pull request #5991 from onflow/khalil/5733-qc-stop-voting Stop in progress QC voting when network goes into EFM commit 6fd2ec4b443211dd422543009e5f23d8653e1173 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 11:26:06 2024 -0400 Update qc_voter_test.go commit 01d41d01b24c416878cde4d346649c21c3936166 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 11:19:33 2024 -0400 fix for loop commit a3679318c2c66c0a10a08b7a67cf8075f03a95ae Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:56:19 2024 -0400 update godoc commit baf45b3e997f525382dec90d2a4e3e32b19bdf04 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:53:31 2024 -0400 store cancel func directly commit f086cc9c0d5e1a89e9847d67d7a6c92f06d78288 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:43:48 2024 -0400 defer cancel commit 87ddec46fb8c724e1254203f45fe78eea2a34379 Merge: 089c141bc8 61e0eb1b27 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:34:51 2024 -0400 Merge branch 'khalil/5733-qc-stop-voting' of github.com:onflow/flow-go into khalil/5733-qc-stop-voting commit 61e0eb1b27385147e2cb200776bcf6a36c839f38 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:16:07 2024 -0400 Update module/epochs/qc_voter.go Co-authored-by: Jordan Schalm <[email protected]> commit 91e024b99f774372df0792ed842b168ef449f575 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:15:49 2024 -0400 Update module/epochs/qc_voter_test.go Co-authored-by: Jordan Schalm <[email protected]> commit 3c22d4e3a1a6a87b8a40279d12263a0e6acd9d5f Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:15:39 2024 -0400 Update module/epochs/qc_voter_test.go Co-authored-by: Jordan Schalm <[email protected]> commit 9af389626883432148797752b20766d4430c8607 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:15:17 2024 -0400 Update engine/collection/epochmgr/engine_test.go Co-authored-by: Jordan Schalm <[email protected]> commit 75cb96af54d9537c5d426ac73bf67b98a35a650c Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:14:33 2024 -0400 Update module/epochs/qc_voter.go Co-authored-by: Jordan Schalm <[email protected]> commit ab10efd8d7ada1db2f3bf81a915c50d4b9fa7494 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:14:24 2024 -0400 Update engine/collection/epochmgr/engine_test.go Co-authored-by: Jordan Schalm <[email protected]> commit 6ced75809460e03c41a894370797b2e8b5938335 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:14:12 2024 -0400 Update engine/collection/epochmgr/engine_test.go Co-authored-by: Jordan Schalm <[email protected]> commit a7ae53057733913978ee7f390977e5d7b4fe4ce9 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:13:08 2024 -0400 Update engine/collection/epochmgr/engine_test.go Co-authored-by: Jordan Schalm <[email protected]> commit 1c0e395e14b4f37e0cf4962fc38d7a776c74554e Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:12:16 2024 -0400 Update engine/collection/epochmgr/engine.go Co-authored-by: Jordan Schalm <[email protected]> commit cd0dfcd7e4728d2712266323529a30d49802332e Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:11:24 2024 -0400 Update engine/collection/epochmgr/engine.go Co-authored-by: Jordan Schalm <[email protected]> commit d71ef84e9c1bb21d6eb1393f83ce13c1be002fa7 Author: Khalil Claybon <[email protected]> Date: Mon Jun 17 10:10:35 2024 -0400 Update engine/collection/epochmgr/engine.go Co-authored-by: Jordan Schalm <[email protected]> commit 8a21a541c541cdf738b5f45f01970334b397d9f9 Merge: 535ed31535 29a79d06ed Author: Khalil Claybon <[email protected]> Date: Fri Jun 14 16:26:43 2024 -0400 Merge branch 'feature/efm-recovery' into khalil/5733-qc-stop-voting commit 535ed31535da8555e7dca9f225ffc859e16efc4e Merge: 091264c110 fda365e6ee Author: Khalil Claybon <[email protected]> Date: Fri Jun 14 16:26:17 2024 -0400 Merge branch 'khalil/5733-qc-stop-voting' of github.com:onflow/flow-go into khalil/5733-qc-stop-voting commit 091264c110f69ab77396c226cf45b98e9df02ba0 Author: Khalil Claybon <[email protected]> Date: Fri Jun 14 16:25:57 2024 -0400 use atomic pointer commit 02e9d9b6a5ab0686a47cf2d7d5cae997ec0c54aa Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 13:23:41 2024 -0700 consolidate DKG phase metrics, update godoc commit f9748fbf1a0a4c7ae12ef1223531e4db71d52fec Merge: b03776b5cd 97f16db05d Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 13:07:01 2024 -0700 Merge branch 'jord/6013-events-metrics' of github.com:onflow/flow-go into jord/6013-events-metrics commit b03776b5cdae581a3adf021d49413642472f5413 Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 13:06:26 2024 -0700 tolerate multiple epoch extensions in a block commit 97f16db05dec08dee472523bed9d667726145cd3 Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 13:02:06 2024 -0700 Update state/protocol/badger/mutator.go commit fda365e6ee90883431bf17c2b966627fc50ed1fd Author: Khalil Claybon <[email protected]> Date: Fri Jun 14 16:01:39 2024 -0400 Update engine/collection/epochmgr/engine.go Co-authored-by: Alexander Hentschel <[email protected]> commit 5af958363ad8a9b6635fe283743c57fb028384e9 Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 13:01:28 2024 -0700 update metric/event godocs commit 176b7a7313fbaf3b522d71fb3b3a8ecbce2f5136 Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 12:55:36 2024 -0700 lint imports commit 7d315a481144bd67de8f96cfb0deb8c6da56d984 Merge: f015fd1329 29a79d06ed Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 10:35:47 2024 -0700 Merge branch 'feature/efm-recovery' into jord/6013-events-metrics commit 29a79d06ed9ec0287863da752fc40ec94de33cbd Merge: 3cccef6d14 42426f1416 Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 10:35:11 2024 -0700 Merge pull request #6101 from onflow/jord/sync-master-efm-2024-06-14 Sync master/efm-recovery 2024 06 14 commit 42426f1416ba3a769e5dab525cfe02d7c50c6527 Merge: 339e9ef3ee d32b10ebda Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 09:44:18 2024 -0700 Merge branch 'master' into jord/sync-master-efm-2024-06-14 commit 339e9ef3ee115780be1527faa24be443ba40651c Merge: 3cccef6d14 b4e2ee254a Author: Jordan Schalm <[email protected]> Date: Fri Jun 14 09:39:29 2024 -0700 Merge branch 'jord/sync-master-efm' into jord/sync-master-efm-2024-06-14 commit f015fd1329aa619e25f211fd0690977a03f1a695 Author: Jordan Schalm <[email protected]> Date: Thu Jun 13 12:14:26 2024 -0700 consistent efm terminology commit d32b10ebdaea67c0c42a069b7e78f1efe64c87a3 Merge: 2818724f3c eb76ea497f Author: Ramtin M. Seraj <[email protected]> Date: Thu Jun 13 18:05:14 2024 +0000 Merge pull request #6089 from onflow/ramtin/fix-the-issue-with-the-tx-executed-error-message [Flow EVM] Add error message to the Tx executed event (bug fix) commit eb76ea497f3938b2ea8292082b641cf75e5c6c08 Author: ramtinms <[email protected]> Date: Thu Jun 13 09:20:07 2024 -0700 update state commitment commit dfbb133c6fce06c480615f32e5b556837e47b28e Author: Jordan Schalm <[email protected]> Date: Thu Jun 13 08:52:42 2024 -0700 add efm-exited events commit 02feb0fabd65896fa931b0398e93e9fee8e07cc3 Author: ramtinms <[email protected]> Date: Wed Jun 12 23:42:01 2024 -0700 fix typo commit 44fc4dc1b9227df3e49fdc7a51f677caa2182c58 Author: ramtinms <[email protected]> Date: Wed Jun 12 23:22:15 2024 -0700 update doc commit 11bdad354cdca6c0a75b1d23fd26d46896b5ec72 Author: ramtinms <[email protected]> Date: Wed Jun 12 23:21:53 2024 -0700 . commit 6b328a1d42c2653edd938678b997f1d39e53dcae Author: ramtinms <[email protected]> Date: Wed Jun 12 23:21:04 2024 -0700 hot fix commit 341b455ede9355f5a8d7b6ef1015a702983e3317 Author: ramtinms <[email protected]> Date: Wed Jun 12 18:35:51 2024 -0700 add returned data commit b89580eb53939039875d1219d16db5568e427175 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 16:46:29 2024 -0700 update mocks commit 019049b1ed4d8d60c6149f3710e387f80ddd1c5d Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 16:45:07 2024 -0700 add event for exiting efm commit c89e77fb8292d6358d325f6c4290571dd5426ed7 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 16:37:14 2024 -0700 rename: epoch emergency fallback -> epoch fallback mode commit 80c8557206a5913ca632142f62676ec7dee98828 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 16:34:57 2024 -0700 remove skipped test TestExtendEpochTransitionWithoutCommit it is covered by new tests under TestEmergencyEpochFallback commit 4e9a95299931aa37af279d11baa2c806f570570d Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 16:32:02 2024 -0700 re-enable skipped tests commit 0e5e254fa589c1a60e6807ec01fca8b08729e355 Author: ramtinms <[email protected]> Date: Wed Jun 12 14:58:23 2024 -0700 move error message closer to error code commit a48d27a824f8c9e959d18c59adc22591ef9c504f Author: ramtinms <[email protected]> Date: Wed Jun 12 14:56:36 2024 -0700 Add error message to the Tx executed event commit 2818724f3c61deccb4e2edf1da9e13422ddbf1f3 Merge: be5509d515 e5632072b2 Author: Joshua Hannan <[email protected]> Date: Wed Jun 12 21:02:38 2024 +0000 Merge pull request #6085 from onflow/update-core-contracts Updates core contracts commit e5632072b290ed7dc690ae52494da79e11b5a8ca Author: Bastian Müller <[email protected]> Date: Wed Jun 12 13:42:27 2024 -0700 go mod tidy commit c9c1de2e8fe6cd4b8be3fd84d1d46f17e7d370f5 Author: Josh Hannan <[email protected]> Date: Wed Jun 12 14:14:27 2024 -0500 update core contracts versions commit be5509d5151a0f9ec1805e469020ee66b4c287ef Merge: 53c970a0ef 058e3c4e76 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 20:13:59 2024 +0000 Merge pull request #6061 from onflow/jord/integration-cruise-ctl-config [Flaky Tests] Testing adjusting safety threshold and block time variables commit 53c970a0efc93fad07a45d0e78b2d860e5ecdcca Merge: 27bb2d1bc6 053e77ccfd Author: Faye Amacker <[email protected]> Date: Wed Jun 12 19:39:44 2024 +0000 Merge pull request #6081 from onflow/auto-update-onflow-cadence-v1.0.0-preview.34 Update to Cadence v1.0.0-preview.34 commit 27bb2d1bc6c9cca4e9e0a3721a440692ed046c3d Merge: ab907b9b44 b86ca1c87b Author: Bastian Müller <[email protected]> Date: Wed Jun 12 19:08:21 2024 +0000 Merge pull request #6082 from onflow/petera/fix-flaky-access-tests [CI] Remove cruise control overrides in Access integration tests commit b86ca1c87b0a63b0a41a9a8260bb5f4efc14c65b Author: Peter Argue <[email protected]> Date: Wed Jun 12 10:41:09 2024 -0700 [CI] Remove cruise control overrides in Access integration tests commit 053e77ccfd828e4592a164c569c7bab2540e9555 Author: Bastian Müller <[email protected]> Date: Wed Jun 12 10:27:37 2024 -0700 adjust tests: add new visits, account for them in metrics commit e5cc4766de0ee7967b35a0ad88489387bdedea82 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 10:14:14 2024 -0700 update tests for metric changes commit 6bd2e01996777d10c3b61c61bbd652d63776af62 Author: Bastian Müller <[email protected]> Date: Wed Jun 12 09:55:57 2024 -0700 Update to Cadence v1.0.0-preview.34 commit ab907b9b44892b94c7739208d8908b929eb37dc7 Merge: 7a6d8a7807 22daffdb72 Author: Leo Zhang <[email protected]> Date: Wed Jun 12 16:24:44 2024 +0000 Merge pull request #6042 from onflow/leo/pebble-chunk-data-packs Pebble based chunk data packs storage commit 7a6d8a7807a4511698fcbc7622229b829a7efa5d Merge: da4f52ccf1 6a430397fe Author: Gregor G <[email protected]> Date: Wed Jun 12 15:05:41 2024 +0000 Merge pull request #6059 from m-Peter/evm-dry-run-compute-gas-refunds [EVM] Take into account gas refunds in `EVM.dryRun` commit cd852c62833da0c9ba51a7f2f0b5ae22b938ff6c Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 08:00:12 2024 -0700 update event godoc commit 3a2c757b3b998594129718bbb192e37e14cbd897 Author: Jordan Schalm <[email protected]> Date: Wed Jun 12 07:55:45 2024 -0700 rename EFM event and metric commit da4f52ccf188e73da5fb2be18b13524a30377c95 Merge: 09b0870897 9e5913c020 Author: Janez Podhostnik <[email protected]> Date: Wed Jun 12 14:32:53 2024 +0000 Merge pull request #6044 from onflow/janez/change-metrics-collection-in-computer Change metrics collection in computer commit 09b08708974c04bec3e5287dac28c0a41826baf7 Merge: 216b3ce086 f1a3a3999d Author: Faye Amacker <[email protected]> Date: Tue Jun 11 22:48:23 2024 +0000 Merge pull request #6070 from onflow/fxamacker/fix-migration-missing-path-cap-domain Fix Cadence 1.0 migration missing PathCapabilityStorageDomain commit 216b3ce08691d396d2cd746051e20a25f1ee7374 Merge: 36575f8a38 b145fe1be8 Author: Faye Amacker <[email protected]> Date: Tue Jun 11 22:33:26 2024 +0000 Merge pull request #6063 from onflow/fxamacker/add-util-check-atree-inlined-status Add atree-inlined-status command to util program to check migration results commit 118ffebb52923e1cd9c02764d36a31947b835ae8 Author: Jordan Schalm <[email protected]> Date: Tue Jun 11 14:57:36 2024 -0700 use Noop in FinalizedReader commit f6a249e6accfb691f5c1b2200cca47a8a52a5562 Author: Jordan Schalm <[email protected]> Date: Tue Jun 11 14:52:15 2024 -0700 update tests commit f1a3a3999db9bc0bdb94f5ef05f465cfbf7513c5 Merge: 3075a06413 bbae923116 Author: Faye Amacker <[email protected]> Date: Tue Jun 11 16:35:39 2024 -0500 Merge pull request #6074 from onflow/fxamacker/fix-migration-missing-acc-cap-domain Fix migrations missing AccountCapabilityStorageDomain commit bbae9231164254d52b485de7b1906302b1040bdc Author: Faye Amacker <[email protected]> Date: Tue Jun 11 16:15:19 2024 -0500 Fix migrations missing AccountCapabilityStorageDomain Currently, migrations don't include AccountCapabilityStorageDomain during traversal and storage health check. This causes payloads in the domain to not be migrated. This affects atree migration and maybe also Cadence 1.0 migration. This commit adds AccountCapabilityStorageDomain to migrations. commit 22daffdb72b8fbd1f7434d0e1aa44a1b5e909f13 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Tue Jun 11 11:50:24 2024 -0700 address review comments commit cde5e81fa433856afaee118f8cda6e4115dd4eee Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Mon Jun 10 16:35:24 2024 -0700 add comments commit 466f9a6e1cac5b3f8650cea8651aca4f1f181557 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Mon Jun 10 09:30:10 2024 -0700 fix chunk data pack codec commit 7e8d0d6bde956ed33a36bd6839ce8667759a573c Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Mon Jun 10 09:06:53 2024 -0700 update mock commit ad06637cee71a5fe41f6c1121fe78238c24b2636 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Fri Jun 7 15:02:51 2024 -0700 fix lint commit 03fdb3924837fa73e282c680d96a950ad9cb535b Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Fri Jun 7 12:57:59 2024 -0700 extract batch commit 86e6267666475ca12bac231c47bfe9a39591fceb Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Fri Jun 7 11:46:57 2024 -0700 move modules to operations package commit 7efc768e95421a810a7a89ce2dd5a204d5e0f2c7 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Fri Jun 7 11:41:23 2024 -0700 implement pebble chunk data pack with cache commit b511b0a6f1097139d138ad706c040cfb4ccd326c Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Thu Jun 6 15:44:48 2024 -0700 tmp commit fc774476f32d8d095cbc74faa33dd7bb07035a00 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Wed Jun 5 16:13:48 2024 -0700 use pebble based chunk data pack storage commit 14dcad5e8f45044d75cfc410462ccaf98846dd07 Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Wed Jun 5 15:55:16 2024 -0700 implement pebble chunk data pack commit d907833726e22b27bbf234231163362f427b8eda Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Wed Jun 5 11:59:46 2024 -0700 move StoredChunkDataPack to storage package commit a96af388b551e3b7211d7d0d3d79bd839798d6fb Author: Leo Zhang (zhangchiqing) <[email protected]> Date: Wed Jun 5 10:56:48 2024 -0700 update chunk data pack storage interface commit 3075a064135d80d2ce5a81dd95b89b8223d8548f Author: Faye Amacker <[email protected]> Date: Tue Jun 11 14:45:16 2024 -0500 Fix migrations missing PathCapabilityStorageDomain Currently, migrations don't include PathCapabilityStorageDomain during traversal and storage health check. This causes payloads in the domain to not be migrated. This commit adds PathCapabilityStorageDomain to migrations. commit b145fe1be83e48739360ecc8d97606cfb953998d Author: Faye Amacker <[email protected]> Date: Tue Jun 11 12:18:49 2024 -0500 Extract readTrie and parseStateCommitment funcs commit 6a430397fe191d5a0eb5c8c89cb5004cfb693aef Merge: b6fc9e1ea3 36575f8a38 Author: Gregor G <[email protected]> Date: Tue Jun 11 14:53:45 2024 +0200 Merge branch 'master' into evm-dry-run-compute-gas-refunds commit b6fc9e1ea349062b7fbbb9d575421f9d213518e2 Author: Ardit Marku <[email protected]> Date: Tue Jun 11 13:00:52 2024 +0300 Add the Successful() method on Result and improve tests for EVM.dryRun commit 8d9b45ab91d9fac1b0c4a1bad8774bcb8fec4538 Merge: 8c59995dc8 36575f8a38 Author: Faye Amacker <[email protected]> Date: Mon Jun 10 20:41:16 2024 -0500 Merge branch 'master' into fxamacker/add-util-check-atree-inlined-status commit 8c59995dc8f4317c320964a52a63abf82255026e Author: Faye Amacker <[email protected]> Date: Mon Jun 10 20:14:08 2024 -0500 Add atree-inlined-status command to util program atree-inlined-status command examines atree payloads and outputs number of payloads inlined, not inlined, etc. in JSON format. Among several other flags, the program accepts - n-workers (default=8) - n-payloads (default=all, number of payloads to sample) This can be used for testing and debugging migration results. commit d9b9d3a8a083b43b9fa0ba60a50b67d652369e44 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 17:21:45 2024 -0700 mock new metrics/events commit 5aa5d343fde943b81dad27531fea2e30284ed454 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 17:04:06 2024 -0700 update mocks commit 8760e36b834eafd57fe69b16f536eafb215416ef Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 16:58:34 2024 -0700 add EpochExtended event commit 36575f8a38b579a9f08e6a7855523080a78476df Merge: 952cbe49c1 2d76ad360e Author: Peter Argue <[email protected]> Date: Mon Jun 10 21:57:40 2024 +0000 Merge pull request #5969 from AndriiDiachuk/choose-execution-nodes-preferred-EN-ids-fix [Access] chooseExecutionNodes fix commit e2460c7b51853deef2b01b8a673269f5c1af3a25 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 14:28:16 2024 -0700 add godoc to ComplianceMetrics commit 786a7733f2bf7d79844270bf86d3f1150e92431f Merge: 5ed22e7ae2 3cccef6d14 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 14:14:37 2024 -0700 Merge branch 'feature/efm-recovery' into jord/6013-events-metrics commit 3cccef6d143f7a45e1540dd4c9efcdd8cc7507dd Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 13:55:40 2024 -0700 Sync `master` into `feature/efm-recovery` (#6058) * Update cmd/util/cmd/diff-states/cmd.go * adjust test * merge cleanup * Update engine/consensus/matching/core.go Co-authored-by: Jordan Schalm <[email protected]> * added logging key for byzantine protocol violation * Update cmd/util/ledger/migrations/cadence_values_migration.go Co-authored-by: Faye Amacker <[email protected]> * fix suggestion * fix error handling / logging * Removed replace * Added check for uninitialized error * Apply suggestions from code review Co-authored-by: Peter Argue <[email protected]> * set HighestIndexedHeight to 0 on index not initialize error. * mod tidy * temp tracer * test for tracer * use errgroup for concurrent account migration * Apply suggestions from code review Co-authored-by: Alexander Hentschel <[email protected]> * rename RichEpochProtocolStateEntry comments, constructor, etc. * EpochProtocolState interface updates * upgrade emulator * rename EpochProtocolStateFromServiceEvents * rename RootEpochProtocolStateFixture * Add migration metrics collection * update epoch protocol state naming in storage layer * EpochProtocolStateEntries godoc * rename EpochProtocolStateEntries in all.go * Collect total values/objects for each contract * add get full collection api * add access fetcher * fix assigning collection requester * add log * add retry * fix retry * use tls * convert port * go mod tidy * add validation * Update to Cadence v1.0.0-preview.30 * remove caching, it is no longer available * adjust tests to fixed String() * Add flag for reporting metrics * update mocks * update mock * Update storage/badger/epoch_protocol_state.go * Also include attachments * add weight for new memory kind * Fix tests * lint * fix lint * handle panics while traversing * Read-lock metric generation * change batch size * TPS fixes * reduce evm batc size * change evm batch size * adjust batch size * experiment with reseting tps after setup * fix adjuster * test * fix set tps bug * test 2 * getting there * 50 per batch * batch size of 100 * next measurement * next measurement * next measurement * next measurement * Update state/protocol/inmem/epoch_protocol_state.go Co-authored-by: Yurii Oleksyshyn <[email protected]> * fixes * fixes * lint fix * disable script checking by default * Report storage traversing errors during metrics collection * add transaction tracer * add context evm debug params * create a tracer and trace with upload * add with transaction tracer option in config * add evm debug params in fvm context * add tracer on block context * wip test for trying out tracer * uploader first draft * Updated TestOnFinalizedBlockSeveralBlocksAhead and TestOnFinalizedBlockSingle tests * Removed unnecessary mock call * refactor to evm tracer * refactor to usage of interface tracer * implement evm tracer and nop tracer * add basic uploader * use evm tracer * rename of tracer tx * not needed * rename * add default client * add tracer test * pass in transaction id * add import for side-effects * add simple tracer test * add nop tracer test * move needed const * add gcp upload test * rename * Apply suggestions from code review Co-authored-by: Yurii Oleksyshyn <[email protected]> * Fixed remarks * log fetching collection * update buffer * add debug log * add uploader test * add integration upload tracer test * use logger * update test logger * remove local replace * revert emulator test * change nop tracer * update handler with nop tracer * Updated test by adding seals to block correctly * mock uploader * mock uploader reuse * test collection of contract call trace * Add the GetErrorForCode helper method This method performs is the inverse mapping of ExecutionErrorCode and ValidationErrorCode. It returns the corresponding error object, given an ErrorCode. * Rename GetErrorForCode to ErrorFromCode * Calculate TotalGasUsed for Tx batch runs and COA calls * add more tests for batch run and run tx * add collection of metrics in all the methods * Update test assertions for TotalGasUsed * add contract deploy at test * Updated test * Linted * add test for handler batch run with trace * remove check due to cd * bucket name configurable * configuring of execution node evm tracing * change logger name * Report additional logs * improve error handling of coa proof verification * add test for invalid proofs * add tracer to deploy at * add trace failure execution test * add retry logic * move collect at the bottom * extract constant values * Fix error message * don't trace scripts * Update to Cadence v1.0.0-preview.32 * add new type key migration * add comment * wip finalized block queue * remove unneeded code * remove map * wip changes on the subscription to the finalized blocks * Revert "wip changes on the subscription to the finalized blocks" This reverts commit 1b737bfe31e075d40a237f6fb746c9e7be1796fb. * Revert "remove map" This reverts commit 38ac28d5ea3ac82e5e30a0b270d28d33e5d0ec06. * Revert "remove unneeded code" This reverts commit 646bc47432d22fde0a90b734ea14fd1fcbc1cb45. * Revert "wip finalized block queue" This reverts commit eaa022bb0a92dfad081ff1cada694c168f0b0ab5. * remove retry complexity * limit to 5 min upload * add block id to evm setup * collect with block id * update tests * exposing evm error msg as part of event * update event decoding * update to Go 1.22 * run Go generate on all packages * go mod tidy * update mockery * change block id name * use nop tracer * tidy * update tests * add collector panic test * fix tests * expose error message to EVM.Result * update state commitments * more state commitment updates * remove unused * check checkpoint has only 1 trie before importing * fix tests * add default nop tracer * update error message name * make helper for trace id * set block id * fix tests with block id * commitment change due to evm contract change * update state commitments * fix lint * disable cruisectl by default in integration tests also remove unneeded hotstuff startup time flag * add test for custom error * capture ret data when tx is failed (evm revert) * add returned value to tx events * emit ret data as part of tx event * rename * unify return data name * update state commitments * Update to Cadence v1.0.0-preview.33 * update new instances of changed naming * update metrics to accomodate current state of EFM --------- Co-authored-by: Bastian Müller <[email protected]> Co-authored-by: Alexander Hentschel <[email protected]> Co-authored-by: Alexander Hentschel <[email protected]> Co-authored-by: Faye Amacker <[email protected]> Co-authored-by: Andrii Slisarchuk <[email protected]> Co-authored-by: Andrii Slisarchuk <[email protected]> Co-authored-by: Andrii Diachuk <[email protected]> Co-authored-by: Peter Argue <[email protected]> Co-authored-by: sideninja <[email protected]> Co-authored-by: Leo Zhang (zhangchiqing) <[email protected]> Co-authored-by: Supun Setunga <[email protected]> Co-authored-by: Janez Podhostnik <[email protected]> Co-authored-by: Janez Podhostnik <[email protected]> Co-authored-by: Yurii Oleksyshyn <[email protected]> Co-authored-by: UlyanaAndrukhiv <[email protected]> Co-authored-by: j pimmel <[email protected]> Co-authored-by: Ardit Marku <[email protected]> Co-authored-by: j pimmel <[email protected]> Co-authored-by: ramtinms <[email protected]> Co-authored-by: Ramtin M. Seraj <[email protected]> commit 058e3c4e76f292c060957b7a6111fa2a25f3dc57 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 13:03:22 2024 -0700 use 250ms block time across all epoch tests commit 5ed22e7ae2e7c1fe7ca377b7f0529f5d825f0c60 Author: Jordan Schalm <[email protected]> Date: Mon Jun 10 08:45:42 2024 -0700 sketch new events location commit d3af3c6cbde79df90abae09a66197bccb133b050 Author: Ardit Marku <[email protected]> Date: Mon Jun 10 15:24:52 2024 +0300 Take into account gas refunds in EVM.dryRun To correctly compute the gas estimation with EVM.dryRun, we need to add any gas refunds to the used gas. Any potential gas refunds are required during transaction execution, and only after they are refunded to the transaction author. commit 2d76ad360ea8c00a7d00206260523e143f7b8657 Author: Andrii <[email protected]> Date: Mon Jun 10 13:32:10 2024 +0300 Fixed tests commit 1c62819191b14de48c521f0d2f38efc7583c5dae Merge: ef71002708 8e6ef660a5 Author: Andrii <[email protected]> Date: Mon Jun 10 12:59:15 2024 +0300 Merge branch 'choose-execution-nodes-preferred-EN-ids-fix' of github.com:AndriiDiachuk/flow-go into choose-execution-nodes-preferred-EN-ids-fix commit ef71002708c2ccc75aa18b226764cc6a9001cd94 Merge: 6df9278ae5 952cbe49c1 Author: Andrii <[email protected]> Date: Mon Jun 10 12:59:00 2024 +0300 Merge branch 'master' of github.com:AndriiDiachuk/flow-go into choose-execution-nodes-preferred-EN-ids-fix commit b4e2ee254ac6ebfc0a0ea0b292f0eae5f6bd75a4 Author: Jordan Schalm <[email protected]> Date: Fri Jun 7 15:58:01 2024 -0700 update metrics to accomodate current state of EFM commit 5347f36d5d5c0fadd4b19850ec789c486b08a5dc Author: Jordan Schalm <[email protected]> Date: Fri Jun 7 15:09:00 2024 -0700 update new instances of changed naming commit 9501352ee877db7865902a460406b3533d6d0b22 Merge: 0dddc9e10f 61b1f2679a Author: Jordan Schalm <[email protected]> Date: Fri Jun 7 14:59:33 2024 -0700 Merge branch 'master' into jord/sync-master-efm address conflicts, including re-generating mocks Text of conflicts below: diff --cc model/flow/protocol_state.go index 4c8bba49bb,53798a1353..0000000000 --- a/model/flow/protocol_state.go +++ b/model/flow/protocol_state.go @@@ -162,20 -147,13 +162,28 @@@ func NewRichEpochProtocolStateEntry NextEpochIdentityTable: IdentityList{}, } - // If previous epoch is specified: ensure respective epoch service events are not nil and consistent with commitments in `ProtocolStateEntry.PreviousEpoch` + // If previous epoch is specified: ensure respective epoch service events are not nil and consistent with commitments in `EpochProtocolStateEntry.PreviousEpoch` if protocolState.PreviousEpoch != nil { ++<<<<<<< HEAD + if protocolState.PreviousEpoch.SetupID != previousEpochSetup.ID() { // calling ID() will panic if EpochSetup event is nil + return nil, fmt.Errorf("supplied previous epoch's setup event (%x) does not match commitment (%x) in ProtocolStateEntry", previousEpochSetup.ID(), protocolState.PreviousEpoch.SetupID) + } + if protocolState.PreviousEpoch.CommitID != previousEpochCommit.ID() { // calling ID() will panic if EpochCommit event is nil + return nil, fmt.Errorf("supplied previous epoch's commit event (%x) does not match commitment (%x) in ProtocolStateEntry", previousEpochCommit.ID(), protocolState.PreviousEpoch.CommitID) ++======= + if protocolState.PreviousEpoch.SetupID != previousEpochSetup.ID() { // calling ID() will panic is EpochSetup event is nil + return nil, fmt.Errorf("supplied previous epoch's setup event (%x) does not match commitment (%x) in EpochProtocolStateEntry", previousEpochSetup.ID(), protocolState.PreviousEpoch.SetupID) + } + if protocolState.PreviousEpoch.CommitID != previousEpochCommit.ID() { // calling ID() will panic is EpochCommit event is nil + return nil, fmt.Errorf("supplied previous epoch's commit event (%x) does not match commitment (%x) in EpochProtocolStateEntry", previousEpochCommit.ID(), protocolState.PreviousEpoch.CommitID) ++>>>>>>> master + } + } else { + if previousEpochSetup != nil { + return nil, fmt.Errorf("no previous epoch but gotten non-nil EpochSetup event") + } + if previousEpochCommit != nil { + return nil, fmt.Errorf("no previous epoch but gotten non-nil EpochCommit event") } } @@@ -227,12 -198,8 +235,12 @@@ } if nextEpoch.CommitID != ZeroID { if nextEpoch.CommitID != nextEpochCommit.ID() { - return nil, fmt.Errorf("supplied next epoch's commit event (%x) does not match commitment (%x) in ProtocolStateEntry", nextEpoch.CommitID, nextEpochCommit.ID()) + return nil, fmt.Errorf("supplied next epoch's commit event (%x) does not match commitment (%x) in EpochProtocolStateEntry", nextEpoch.CommitID, nextEpochCommit.ID()) } + } else { + if nextEpochCommit != nil { + return nil, fmt.Errorf("next epoch not yet committed but got EpochCommit event") + } } result.CurrentEpochIdentityTable, err = BuildIdentityTable( @@@ -285,11 -252,11 +293,19 @@@ func (e *EpochProtocolStateEntry) Copy( if e == nil { return nil } ++<<<<<<< HEAD + return &ProtocolStateEntry{ + PreviousEpoch: e.PreviousEpoch.Copy(), + CurrentEpoch: *e.CurrentEpoch.Copy(), + NextEpoch: e.NextEpoch.Copy(), + EpochFallbackTriggered: e.EpochFallbackTriggered, ++======= + return &EpochProtocolStateEntry{ + PreviousEpoch: e.PreviousEpoch.Copy(), + CurrentEpoch: *e.CurrentEpoch.Copy(), + NextEpoch: e.NextEpoch.Copy(), + InvalidEpochTransitionAttempted: e.InvalidEpochTransitionAttempted, ++>>>>>>> master } } @@@ -313,20 -280,9 +329,20 @@@ func (e *RichEpochProtocolStateEntry) C } } +// CurrentEpochFinalView returns the final view of the current epoch, taking into account possible epoch extensions. +// If there are no epoch extensions, the final view is the final view of the current epoch setup, +// otherwise it is the final view of the last epoch extension. +func (e *RichProtocolStateEntry) CurrentEpochFinalView() uint64 { + l := len(e.CurrentEpoch.EpochExtensions) + if l > 0 { + return e.CurrentEpoch.EpochExtensions[l-1].FinalView + } + return e.CurrentEpochSetup.FinalView +} + // EpochPhase returns the current epoch phase. - // The receiver ProtocolStateEntry must be properly constructed. - func (e *ProtocolStateEntry) EpochPhase() EpochPhase { + // The receiver EpochProtocolStateEntry must be properly constructed. + func (e *EpochProtocolStateEntry) EpochPhase() EpochPhase { // The epoch phase is determined by how much information we have about the next epoch if e.NextEpoch == nil { return EpochPhaseStaking // if no information about the next epoch is known, we are in the Staking Phase diff --cc state/protocol/badger/mutator.go index 230bd73227,395aa197ce..0000000000 --- a/state/protocol/badger/mutator.go +++ b/state/protocol/badger/mutator.go @@@ -662,49 -691,36 +691,77 @@@ func (m *FollowerState) Finalize(ctx co // We update metrics and emit protocol events for epoch state changes when // the block corresponding to the state change is finalized ++<<<<<<< HEAD + parentEpochStateSnapshot, err := m.protocolState.AtBlockID(header.ParentID) + if err != nil { + return fmt.Errorf("could not retrieve parent protocol state snapshot: %w", err) + } + epochStateSnapshot, err := m.protocolState.AtBlockID(blockID) + if err != nil { + return fmt.Errorf("could not retrieve protocol state snapshot: %w", err) + } + currentEpochSetup := epochStateSnapshot.EpochSetup() + epochFallbackTriggered := epochStateSnapshot.EpochFallbackTriggered() + + // if epoch fallback was not previously triggered, check whether this block triggers it + if epochFallbackTriggered && !parentEpochStateSnapshot.EpochFallbackTriggered() { ++======= + parentEpochState, err := m.protocolState.EpochStateAtBlockID(block.Header.ParentID) + if err != nil { + return fmt.Errorf("could not retrieve protocol state snapshot for parent: %w", err) + } + finalizingEpochState, err := m.protocolState.EpochStateAtBlockID(blockID) + if err != nil { + return fmt.Errorf("could not retrieve protocol state snapshot: %w", err) + } + currentEpochSetup := finalizingEpochState.EpochSetup() + epochFallbackTriggered, err := m.isEpochEmergencyFallbackTriggered() + if err != nil { + return fmt.Errorf("could not check persisted epoch emergency fallback flag: %w", err) + } + + // if epoch fallback was not previously triggered, check whether this block triggers it + // TODO(efm-recovery): remove separate global EFM flag + if !epochFallbackTriggered && finalizingEpochState.InvalidEpochTransitionAttempted() { + epochFallbackTriggered = true ++>>>>>>> master // emit the protocol event only the first time epoch fallback is triggered events = append(events, m.consumer.EpochEmergencyFallbackTriggered) metrics = append(metrics, m.metrics.EpochEmergencyFallbackTriggered) } - isFirstBlockOfEpoch, err := m.isFirstBlockOfEpoch(header, currentEpochSetup) + // Determine metric updates and protocol events related to epoch phase changes and epoch transitions. + epochPhaseMetrics, epochPhaseEvents, err := m.epochMetricsAndEventsOnBlockFinalized(parentEpochState, finalizingEpochState, header) if err != nil { ++<<<<<<< HEAD + return fmt.Errorf("could not check if block is first of epoch: %w", err) + } + if isFirstBlockOfEpoch { + epochTransitionMetrics, epochTransitionEvents := m.epochTransitionMetricsAndEventsOnBlockFinalized(header, currentEpochSetup) + if err != nil { + return fmt.Errorf("could not determine epoch transition metrics/events for finalized block: %w", err) + } + metrics = append(metrics, epochTransitionMetrics...) + events = append(events, epochTransitionEvents...) + } + + // Determine metric updates and protocol events related to epoch phase changes and epoch transitions. + // If epoch emergency fallback is triggered, the current epoch continues until + // the next spork - so skip these updates. + // TODO(EFM, #5732, #6013): needs update for EFM recovery + if !epochFallbackTriggered { + epochPhaseMetrics, epochPhaseEvents, err := m.epochPhaseMetricsAndEventsOnBlockFinalized(block) + if err != nil { + return fmt.Errorf("could not determine epoch phase metrics/events for finalized block: %w", err) + } + metrics = append(metrics, epochPhaseMetrics...) + events = append(events, epochPhaseEvents...) ++======= + return fmt.Errorf("could not determine epoch phase metrics/events for finalized block: %w", err) ++>>>>>>> master } + metrics = append(metrics, epochPhaseMetrics...) + events = append(events, epochPhaseEvents...) // Extract and validate version beacon events from the block seals. versionBeacons, err := m.versionBeaconOnBlockFinalized(block) @@@ -732,7 -748,14 +789,18 @@@ if err != nil { return fmt.Errorf("could not update sealed height: %w", err) } ++<<<<<<< HEAD + if isFirstBlockOfEpoch { ++======= + if epochFallbackTriggered { + err = operation.SetEpochEmergencyFallbackTriggered(blockID)(tx) + if err != nil { + return fmt.Errorf("could not set epoch fallback flag: %w", err) + } + } + // TODO(efm-recovery): we should be able to omit the `!epochFallbackTriggered` check here. + if isFirstBlockOfEpoch(parentEpochState, finalizingEpochState) && !epochFallbackTriggered { ++>>>>>>> master err = operation.InsertEpochFirstHeight(currentEpochSetup.Counter, header.Height)(tx) if err != nil { return fmt.Errorf("could not insert epoch first block height: %w", err) diff --cc state/protocol/badger/state.go index beef6343ec,298f0f5c03..0000000000 --- a/state/protocol/badger/state.go +++ b/state/protocol/badger/state.go @@@ -968,3 -969,16 +968,19 @@@ func (state *State) populateCache() err return nil } ++<<<<<<< HEAD ++======= + + // isEpochEmergencyFallbackTriggered checks whether epoch fallback has been globally triggered. + // TODO(efm-recovery): Stop storing a global EFM flag, use parentState.EFMTriggered instead + // + // Returns: + // * (true, nil) if epoch fallback is triggered + // * (false, nil) if epoch fallback is not triggered (including if the flag is not set) + // * (false, err) if an unexpected error occurs + func (state *State) isEpochEmergencyFallbackTriggered() (bool, error) { + var triggered bool + err := state.db.View(operation.CheckEpochEmergencyFallbackTriggered(&triggered)) + return triggered, err + } ++>>>>>>> master diff --cc state/protocol/mock/epoch_protocol_state.go index a8685f222f,2ad2840d11..0000000000 --- a/state/protocol/mock/epoch_protocol_state.go +++ b/state/protocol/mock/epoch_protocol_state.go @@@ -112,24 -132,14 +132,28 @@@ func (_m *EpochProtocolState) EpochComm return r0 } +// EpochFallbackTriggered provides a mock function with given fields: +func (_m *DynamicProtocolState) EpochFallbackTriggered() bool { + ret := _m.Called() + + var r0 bool + if rf, ok := ret.Get(0).(func() bool); ok { + r0 = rf() + } else { + r0 = ret.Get(0).(bool) + } + + return r0 +} + // EpochPhase provides a mock function with given fields: - func (_m *DynamicProtocolState) EpochPhase() flow.EpochPhase { + func (_m *EpochProtocolState) EpochPhase() flow.EpochPhase { ret := _m.Called() + if len(ret) == 0 { + panic("no return value specified for EpochPhase") + } + var r0 flow.EpochPhase if rf, ok := ret.Get(0).(func() flow.EpochPhase); ok { r0 = rf() @@@ -188,10 -210,32 +224,35 @@@ func (_m *EpochProtocolState) Identitie return r0 } ++<<<<<<< HEAD:state/protocol/mock/dynamic_protocol_state.go ++======= + // InvalidEpochTransitionAttempted provides a mock function with given fields: + func (_m *EpochProtocolState) InvalidEpochTransitionAttempted() bool { + ret := _m.Called() + + if len(ret) == 0 { + panic("no return value specified for InvalidEpochTransitionAttempted") + } + + var r0 bool + if rf, ok := ret.Get(0).(func() bool); ok { + r0 = rf() + } else { + r0 = ret.Get(0).(bool) + } + + return r0 + } + ++>>>>>>> master:state/protocol/mock/epoch_protocol_state.go // PreviousEpochExists provides a mock function with given fields: - func (_m *DynamicProtocolState) PreviousEpochExists() bool { + func (_m *EpochProtocolState) PreviousEpochExists() bool { ret := _m.Called() + if len(ret) == 0 { + panic("no return value specified for PreviousEpochExists") + } + var r0 bool if rf, ok := ret.Get(0).(func() bool); ok { r0 = rf() diff --cc state/protocol/mock/instance_params.go index 7b42147d2d,0ca3db5c8c..0000000000 --- a/state/protocol/mock/instance_params.go +++ b/state/protocol/mock/instance_params.go @@@ -12,6 -12,34 +12,37 @@@ type InstanceParams struct mock.Mock } ++<<<<<<< HEAD ++======= + // EpochFallbackTriggered provides a mock function with given fields: + func (_m *InstanceParams) EpochFallbackTriggered() (bool, error) { + ret := _m.Called() + + if len(ret) == 0 { + panic("no return value specified for EpochFallbackTriggered") + } + + var r0 bool + var r1 error + if rf, ok := ret.Get(0).(func() (bool, error)); ok { + return rf() + } + if rf, ok := ret.Get(0).(func() bool); ok { + r0 = rf() + } else { + r0 = ret.Get(0).(bool) + } + + if rf, ok := ret.Get(1).(func() error); ok { + r1 = rf() + } else { + r1 = ret.Error(1) + } + + return r0, r1 + } + ++>>>>>>> master // FinalizedRoot provides a mock function with given fields: func (_m *InstanceParams) FinalizedRoot() *flow.Header { ret := _m.Called() diff --cc state/protocol/mock/params.go index 677ba3d9ff,56cad5a925..0000000000 --- a/state/protocol/mock/params.go +++ b/state/protocol/mock/params.go @@@ -40,6 -48,34 +48,37 @@@ func (_m *Params) EpochCommitSafetyThre return r0 } ++<<<<<<< HEAD ++======= + // EpochFallbackTriggered provides a mock function with given fields: + func (_m *Params) EpochFallbackTriggered() (bool, error) { + ret := _m.Called() + + if len(ret) == 0 { + panic("no return value specified for EpochFallbackTriggered") + } + + var r0 bool + var r1 error + if rf, ok := ret.Get(0).(func() (bool, error)); ok { + return rf() + } + if rf, ok := ret.Get(0).(func() bool); ok { + r0 = rf() + } else { + r0 = ret.Get(0).(bool) + } + + if rf, ok := ret.Get(1).(func() error); ok { + r1 = rf() + } else { + r1 = ret.Error(1) + } + + return r0, r1 + } + ++>>>>>>> master // FinalizedRoot provides a mock function with given fields: func (_m *Params) FinalizedRoot() *flow.Header { ret := _m.Called() diff --cc state/protocol/protocol_state.go index 8dd9877b24,2dbbb457ab..0000000000 --- a/state/protocol/protocol_state.go +++ b/state/protocol/protocol_state.go @@@ -5,37 -5,46 +5,54 @@@ import "github.com/onflow/flow-go/storage/badger/transaction" ) - // InitialProtocolState returns constant data for given epoch. - // This interface can be only obtained for epochs that have progressed to epoch commit event. - type InitialProtocolState interface { - // Epoch returns counter of epoch. + // EpochProtocolState represents the subset of the Protocol State KVStore related to epochs: + // the Identity Table, DKG, cluster assignment, etc. + // EpochProtocolState is fork-aware and can change on a block-by-block basis. + // Each EpochProtocolState instance refers to the state with respect to some reference block. + type EpochProtocolState interface { + // Epoch returns the current epoch counter. Epoch() uint64 + // Clustering returns initial clustering from epoch setup. + // CAUTION: This describes the initial epoch configuration from the view point of the Epoch + // Smart Contract. It does _not_ account for subsequent node ejections. For Byzantine Fault + // Tolerance, the calling code must account for ejections! // No errors are expected during normal operations. Clustering() (flow.ClusterList, error) + // EpochSetup returns original epoch setup event that was used to initialize the protocol state. + // CAUTION: This describes the initial epoch configuration from the view point of the Epoch + // Smart Contract. It does _not_ account for subsequent node ejections. For Byzantine Fault + // Tolerance, the calling code must account for ejections! EpochSetup() *flow.EpochSetup + // EpochCommit returns original epoch commit event that was used to update the protocol state. + // CAUTION: This describes the initial epoch configuration from the view point of the Epoch + // Smart Contract. It does _not_ account for subsequent node ejections. For Byzantine Fault + // Tolerance, the calling code must account for ejections! EpochCommit() *flow.EpochCommit + // DKG returns information about DKG that was obtained from EpochCommit event. + // CAUTION: This describes the initial epoch configuration from the view point of the Epoch + // Smart Contract. It does _not_ account for subsequent node ejections. For Byzantine Fault + // Tolerance, the calling code must account for ejections! // No errors are expected during normal operations. DKG() (DKG, error) - // Entry Returns low-level protocol state entry that was used to initialize this object. - // It shouldn't be used by high-level logic, it is useful for some cases such as bootstrapping. - // Prefer using other methods to access protocol state. - Entry() *flow.RichProtocolStateEntry - } - - // DynamicProtocolState extends the InitialProtocolState with data that can change from block to block. - // It can be used to access the identity table at given block. - type DynamicProtocolState interface { - InitialProtocolState ++<<<<<<< HEAD + // EpochFallbackTriggered denotes whether an invalid epoch state transition was attempted + // on the fork ending in this block. Once the first block where this flag is true is finalized, epoch + // fallback mode is triggered. This flag is reset to false when finalizing a block that seals + // a valid EpochRecover service event. + EpochFallbackTriggered() bool ++======= + // InvalidEpochTransitionAttempted denotes whether an invalid epoch state transition was attempted + // on the fork ending this block. Once the first block where this flag is true is finalized, epoch + // fallback mode is triggered. + // TODO for 'leaving Epoch Fallback via special service event': at the moment, this is a one-way transition and requires a spork to recover - need to revisit for sporkless EFM recovery + InvalidEpochTransitionAttempted() bool + ++>>>>>>> master // PreviousEpochExists returns true if a previous epoch exists. This is true for all epoch // except those immediately following a spork. PreviousEpochExists() bool diff --cc state/protocol/protocol_state/epochs/factory.go index 6dd9884e85,4abce702ad..0000000000 --- a/state/protocol/protocol_state/epochs/factory.go +++ b/state/protocol/protocol_state/epochs/factory.go @@@ -44,11 -44,11 +44,16 @@@ func (f *EpochStateMachineFactory) Crea f.epochProtocolStateDB, parentState, mutator, - func(candidateView uint64, parentState *flow.RichProtocolStateEntry) (StateMachine, error) { + func(candidateView uint64, parentState *flow.RichEpochProtocolStateEntry) (StateMachine, error) { return NewHappyPathStateMachine(candidateView, parentState) }, ++<<<<<<< HEAD + func(candidateView uint64, parentState *flow.RichProtocolStateEntry) (StateMachine, error) { + return NewFallbackStateMachine(f.params, candidateView, parentState) ++======= + func(candidateView uint64, parentState *flow.RichEpochProtocolStateEntry) (StateMachine, error) { + return NewFallbackStateMachine(candidateView, parentState), nil ++>>>>>>> master }, ) } diff --cc state/protocol/protocol_state/epochs/fallback_statemachine.go index 8237488a90,e8719615c5..0000000000 --- a/state/protocol/protocol_state/epochs/fallback_statemachine.go +++ b/state/protocol/protocol_state/epochs/fallback_statemachine.go @@@ -24,22 -16,12 +24,31 @@@ type FallbackStateMachine struct var _ StateMachine = (*FallbackStateMachine)(nil) ++<<<<<<< HEAD +// NewFallbackStateMachine constructs a state machine for epoch fallback. It automatically sets +// EpochFallbackTriggered to true, thereby recording that we have entered epoch fallback mode. +// No errors are expected during normal operations. +func NewFallbackStateMachine(params protocol.GlobalParams, view uint64, parentState *flow.RichProtocolStateEntry) (*FallbackStateMachine, error) { + state := parentState.ProtocolStateEntry.Copy() + nextEpochCommitted := state.EpochPhase() == flow.EpochPhaseCommitted + // we are entering fallback mode, this logic needs to be executed only once + if !state.EpochFallbackTriggered { + // the next epoch has not been committed, but possibly setup, make sure it is cleared + if !nextEpochCommitted { + state.NextEpoch = nil + } + state.EpochFallbackTriggered = true + } + + sm := &FallbackStateMachine{ ++======= + // NewFallbackStateMachine constructs a state machine for epoch fallback, it automatically sets + // InvalidEpochTransitionAttempted to true, thereby recording that we have entered epoch fallback mode. + func NewFallbackStateMachine(view uint64, parentState *flow.RichEpochProtocolStateEntry) *FallbackStateMachine { + state := parentState.EpochProtocolStateEntry.Copy() + state.InvalidEpochTransitionAttempted = true + return &FallbackStateMachine{ ++>>>>>>> master baseStateMachine: baseStateMachine{ parentState: parentState, state: state, diff --cc state/protocol/protocol_state/epochs/happy_path_statemachine.go index 00467d518e,fdd2518338..0000000000 --- a/state/protocol/protocol_state/epochs/happy_path_statemachine.go +++ b/state/protocol/protocol_state/epochs/happy_path_statemachine.go @@@ -29,10 -28,10 +29,15 @@@ type HappyPathStateMachine struct var _ StateMachine = (*HappyPathStateMachine)(nil) // NewHappyPathStateMachine creates a new HappyPathStateMachine. -// An exception is returned in case the `InvalidEpochTransitionAttempted` flag is set in the `parentState`. This means that +// An exception is returned in case the `EpochFallbackTriggered` flag is set in the `parentState`. This means that // the protocol state evolution has reached an undefined state from the perspective of the happy path state machine. ++<<<<<<< HEAD +func NewHappyPathStateMachine(view uint64, parentState *flow.RichProtocolStateEntry) (*HappyPathStateMachine, error) { + if parentState.EpochFallbackTriggered { ++======= + func NewHappyPathStateMachine(view uint64, parentState *flow.RichEpochProtocolStateEntry) (*HappyPathStateMachine, error) { + if parentState.InvalidEpochTransitionAttempted { ++>>>>>>> master return nil, irrecoverable.NewExceptionf("cannot create happy path protocol state machine at view (%d) for a parent state"+ "which is in Epoch Fallback Mode", view) } @@@ -58,7 -57,7 +63,11 @@@ // CAUTION: the HappyPathStateMachine is left with a potentially dysfunctional state when this error occurs. Do NOT call the Build method // after such error and discard the HappyPathStateMachine! func (u *HappyPathStateMachine) ProcessEpochSetup(epochSetup *flow.EpochSetup) (bool, error) { ++<<<<<<< HEAD + err := protocol.IsValidExtendingEpochSetup(epochSetup, u.parentState) ++======= + err := protocol.IsValidExtendingEpochSetup(epochSetup, u.parentState.EpochProtocolStateEntry, u.parentState.CurrentEpochSetup) ++>>>>>>> master if err != nil { return false, fmt.Errorf("invalid epoch setup event: %w", err) } @@@ -139,60 -153,32 +148,91 @@@ func (u *HappyPathStateMachine) Process return true, nil } ++<<<<<<< HEAD +// ProcessEpochRecover returns the sentinel error `protocol.InvalidServiceEventError`, which +// indicates that `EpochRecover` are not expected on the happy path of epoch lifecycle. +func (u *HappyPathStateMachine) ProcessEpochRecover(*flow.EpochRecover) (bool, error) { + return false, protocol.NewInvalidServiceEventErrorf("epoch recover event received while on happy path") +} + +// When observing setup event for subsequent epoch, construct the EpochStateContainer for `ProtocolStateEntry.NextEpoch`. +// Context: +// Note that the `EpochStateContainer.ActiveIdentities` only contains the nodes that are *active* in the next epoch. Active means +// that these nodes are authorized to contribute to extending the chain. Nodes are listed in `ActiveIdentities` if and only if +// they are part of the EpochSetup event for the respective epoch. +// +// sanity checking SAFETY-CRITICAL INVARIANT (I): +// - Per convention, the `flow.EpochSetup` event should list the IdentitySkeletons in canonical order. This is useful +// for most efficient construction of the full active Identities for an epoch. We enforce this here at the gateway +// to the protocol state, when we incorporate new information from the EpochSetup event. +// - Note that the system smart contracts manage the identity table as an unordered set! For the protocol state, we desire a fixed +// ordering to simplify various implementation details, like the DKG. Therefore, we order identities in `flow.EpochSetup` during +// conversion from cadence to Go in the function `convert.ServiceEvent(flow.ChainID, flow.Event)` in package `model/convert` +// sanity checking SAFETY-CRITICAL INVARIANT (II): +// While ejection status and dynamic weight are not part of the EpochSetup event, we can supplement this information as follows: +// - Per convention, service events are delivered (asynchronously) in an *order-preserving* manner. Furthermore, weight changes or +// node ejection is entirely mediated by system smart contracts and delivered via service events. +// - Therefore, the EpochSetup event contains the up-to-date snapshot of the epoch participants. Any weight changes or node ejection +// that happened before should be reflected in the EpochSetup event. Specifically, the initial weight should be reduced and ejected +// nodes should be no longer listed in the EpochSetup event. +// - Hence, the following invariant must be satisfied by the system smart contracts for all active nodes in the upcoming epoch: +// (i) The Ejected flag is false. Node X being ejected in epoch N (necessarily via a service event emitted by the system +// smart contracts earlier) but also being listed in the setup event for the subsequent epoch (service event emitted by +// the system smart contracts later) is illegal. +// (ii) When the EpochSetup event is emitted / processed, the weight of all active nodes equals their InitialWeight and + +// For collector clusters, we rely on invariants (I) and (II) holding. See `committees.Cluster` for details, specifically function +// `constructInitialClusterIdentities(..)`. While the system smart contract must satisfy this invariant, we run a sanity check below. +func buildNextEpochActiveParticipants(activeIdentitiesLookup map[flow.Identifier]*flow.DynamicIdentityEntry, currentEpochSetup, nextEpochSetup *flow.EpochSetup) (flow.DynamicIdentityEntryList, error) { + nextEpochActiveIdentities := make(flow.DynamicIdentityEntryList, 0, len(nextEpochSetup.Participants)) + prevNodeID := nextEpochSetup.Participants[0].NodeID + for idx, nextEpochIdentitySkeleton := range nextEpochSetup.Participants { + // sanity checking invariant (I): + if idx > 0 && !flow.IsIdentifierCanonical(prevNodeID, nextEpochIdentitySkeleton.NodeID) { + return nil, protocol.NewInvalidServiceEventErrorf("epoch setup event lists active participants not in canonical ordering") + } + prevNodeID = nextEpochIdentitySkeleton.NodeID + + // sanity checking invariant (II.i): + currentEpochDynamicProperties, found := activeIdentitiesLookup[nextEpochIdentitySkeleton.NodeID] + if found && currentEpochDynamicProperties.Ejected { // invariant violated + return nil, protocol.NewInvalidServiceEventErrorf("node %v is ejected in current epoch %d but readmitted by EpochSetup event for epoch %d", nextEpochIdentitySkeleton.NodeID, currentEpochSetup.Counter, nextEpochSetup.Counter) + } + + nextEpochActiveIdentities = append(nextEpochActiveIdentities, &flow.DynamicIdentityEntry{ + NodeID: nextEpochIdentitySkeleton.NodeID, + Ejected: false, // according to invariant (II.i) + }) + } + return nextEpochActiveIdentities, nil ++======= + // TransitionToNextEpoch updates the notion of 'current epoch', 'previous' and 'next epoch' in the protocol + // state. An epoch transition is only allowed when: + // - next epoch has been set up, + // - next epoch has been committed, + // - invalid state transition has not been attempted (this is ensured by constructor), + // - candidate block is in the next epoch. + // No errors are expected during normal operations. + func (u *HappyPathStateMachine) TransitionToNextEpoch() error { + nextEpoch := u.state.NextEpoch + // Check i…
Changes
TestBootstrapAndOpen_EpochCommitted
andTestExtendReceiptsNotSorted
TestExtendEpochTransitionWithoutCommit
, as it is covered by newerTestEpochFallbackMode
testsEpochExtended
andEpochFallbackModeExited
events and metricsNotes
1 -
EpochFallbackModeExited
vsEpochCommittedPhaseStarted
The
EpochFallbackModeExited
event here is simpler than what is in the design doc (no arguments). The reason for this is that, in the current implementation,EpochCommittedPhaseStarted
is also emitted when exiting EFM. For theEpochRecovered
event consumers we have specified in the design doc, the consumer is identical to an existingEpochCommittedPhaseStarted
consumer.My thinking is we can rely on
EpochCommittedPhaseStarted
only for both happy-path and recovery-path, rather than adding theEpochRecovered
event and duplicate handlers.2 - Handling of epoch phase in EFM
I've created a separate issue to address adding the epoch fallback phase: #6092.