Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DOCUMENT][DEMO] Block by block state sync #579

Open
10 tasks
jessicadaugherty opened this issue Mar 13, 2023 · 0 comments · May be fixed by #812
Open
10 tasks

[DOCUMENT][DEMO] Block by block state sync #579

jessicadaugherty opened this issue Mar 13, 2023 · 0 comments · May be fixed by #812
Assignees
@red-0ne @Olshansk
Labels
consensus Consensus specific changes documentation Improvements or additions to documentation testing Defining, adding, automating or modifying tests
Milestone
M1: Pocket PoS (P...

Comments

@jessicadaugherty
Copy link
Contributor

jessicadaugherty commented Mar 13, 2023 •
edited by Olshansk
Loading

Objective

Productionize an E2E demo of state sync using localnet to show the functionality of the state sync server, state machines, and the download and application of blocks on a node by node basis.

Origin Document

Iteration Demos

Goals

  • Demo the state sync functionality and provide instructions on how other contributors or users can recreate the demo on their local envs

Deliverable

  • Bonus: A PR that adds documentation on how a developer can effectively debug & develop state sync
  • A PR that includes a markdown file with demo goals and instructions on how to use state sync locally
  • Updates to the demo diagram

Non-goals / Non-deliverables

  • Demo'ing additional localnet or V1 functionality

General issue deliverables

  • Update the appropriate CHANGELOG
  • Update any relevant READMEs (local and/or global)
  • Update any relevant global documentation & references
  • If applicable, update the source code tree explanation
  • If applicable, add or update a state, sequence or flowchart diagram using mermaid

Testing Methodology

  • All tests: make test_all
  • LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md

Creator: @jessicadaugherty
@jessicadaugherty jessicadaugherty converted this from a draft issue Mar 13, 2023
@jessicadaugherty jessicadaugherty added triage It requires some decision-making at team level (it can't be worked on as it stands) documentation Improvements or additions to documentation consensus Consensus specific changes labels Mar 13, 2023
@jessicadaugherty jessicadaugherty moved this to Backlog in V1 Dashboard Mar 13, 2023
@jessicadaugherty jessicadaugherty changed the title [WIP][DEMO] Block by block state sync [WIP][DOCUMENT][DEMO] Block by block state sync Mar 13, 2023
@jessicadaugherty jessicadaugherty moved this from Backlog to Up Next in V1 Dashboard Mar 13, 2023
@jessicadaugherty jessicadaugherty changed the title [WIP][DOCUMENT][DEMO] Block by block state sync [DOCUMENT][DEMO] Block by block state sync Apr 3, 2023
@Olshansk Olshansk moved this from Up Next to In Progress in V1 Dashboard Apr 10, 2023
@Olshansk Olshansk moved this from In Progress to Up Next in V1 Dashboard Jun 14, 2023
@Olshansk Olshansk linked a pull request Jun 15, 2023 that will close this issue
Open
20 tasks
@Olshansk Olshansk mentioned this issue Jul 28, 2023
Merged
20 tasks
@Olshansk Olshansk removed the status in V1 Dashboard Jul 30, 2023
@Olshansk Olshansk added testing Defining, adding, automating or modifying tests and removed triage It requires some decision-making at team level (it can't be worked on as it stands) labels Jul 30, 2023
@Olshansk Olshansk moved this to In Progress in V1 Dashboard Jul 30, 2023
Olshansk added a commit that referenced this issue Aug 3, 2023
@Olshansk @dylanlott
[E2E Test] Utilities for State Sync Test (#874)
fd09f2c 
## Description

<!-- reviewpad:summarize:start -->
### Summary generated by Reviewpad on 03 Aug 23 20:11 UTC
This pull request introduces several changes across multiple files. 

Here is a summary of the changes:

1. `go.mod`:
   - The `golang.org/x/text` dependency is now listed explicitly instead of being an indirect dependency.

2. `scenario_test.go`:
   - Added a new feature called "State Sync Namespace" that includes various commands and waits for specific amounts of time.
   - Code improvements and TODO comments have been added.

3. `FAQ.md`:
   - Updated an issue related to starting LocalNet with SELinux on an operating system. Replaced the command `make compose_and_watch` with `make lightweight_localnet` to avoid permission denied errors.

4. `.gitignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.
   - Added the entry "**/gomock_reflect_*/" to ignore mock temporary files.

5. `e2e/README.md`:
   - Added a new section on `Keywords`.
   - Modified scenario descriptions and code examples to replace instances of "Validator" with "Node".
   - Included a flowchart depicting the E2E scenarios with updated terminology.

6. Consensus module files:
   - Added a new logging statement in the `HandleDebugMessage` function.
   - Simplified the handling of the `DEBUG_CONSENSUS_RESET_TO_GENESIS` action.

7. `.tiltignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.

8. `CHANGELOG.md`:
   - Updated build commands and added a new section on `Keywords`.

9. `persistence/docs/CHANGELOG.md`:
   - Several changes related to deprecation, addition, and fixing of functions and issues.

10. `validator.feature`:
    - Renamed file from "valdator.feature" to "validator.feature".
    - Updated scenario titles and step descriptions to use more descriptive terminology.
    - Replaced references to "validator" with "node".

11. `tilt_helpers.go`:
    - Added a new file containing functions related to syncing network configuration and checking package installation.

12. `debug.go`:
    - Added new debug commands and subcommands.
    - Updated existing functions and added new functions for debug actions.
    
13. `account.feature`:
    - Added a new file containing scenarios for testing node account functionalities.

14. `README.md` files:
    - Updated sections, titles, dependencies, and instructions in various README.md files.

15. `build/config/README.md`:
    - Updated usage instructions, changing the command `make compose_and_watch` to `make lightweight_localnet`.

16. `iteration_3_end_to_end_tx.md`:
    - Updated commands to start LocalNet and consensus debugger.

17. Deleted files:
   - `validator.go`
   - `watch_build.sh`

18. New files added:
   - `debug.feature`
   - `tilt_helpers.go`
   - `account.feature`

Please let me know if you need more information about any specific change.
<!-- reviewpad:summarize:end -->

## Issue

Fixes par of #579

## Type of change

Please mark the relevant option(s):

- [x] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [ ] Major breaking change
- [ ] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- `s/compose_and_watch/lightweight_localnet` and all related helpers
- `s/validator/node` in e2e tests for clarity
- Add fire-and-forget `Debug` CLI w/ several useful initial subcommands
- Add `keywords` to the `e2e` document
- Add an `e2e debug` test to trigger views and track the blockchain increasing
- Avoid rebuilding the actors if the CLI changes
- Small miscellaneous improvements & code cleanup

## Testing

- [x] `make develop_test`; if any code changes were made
- [ ] `make test_e2e` on [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any code changes were made
- [ ] `e2e-devnet-test` passes tests on [DevNet](https://pocketnetwork.notion.site/How-to-DevNet-ff1598f27efe44c09f34e2aa0051f0dd); if any code was changed
- [ ] [Docker Compose LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md); if any major functionality was changed or introduced
- [ ] [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any infrastructure or configuration changes were made

## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment))
- [ ] I have tested my changes using the available tooling
- [ ] I have updated the corresponding CHANGELOG

### If Applicable Checklist

- [x] I have updated the corresponding README(s); local and/or global
- [x] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s)
- [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s)

---

Co-authored-by: d7t <[email protected]>
red-0ne pushed a commit that referenced this issue Aug 4, 2023
@Olshansk @dylanlott @red-0ne
[E2E Test] Utilities for State Sync Test (#874)
fef4217 
## Description

<!-- reviewpad:summarize:start -->
### Summary generated by Reviewpad on 03 Aug 23 20:11 UTC
This pull request introduces several changes across multiple files. 

Here is a summary of the changes:

1. `go.mod`:
   - The `golang.org/x/text` dependency is now listed explicitly instead of being an indirect dependency.

2. `scenario_test.go`:
   - Added a new feature called "State Sync Namespace" that includes various commands and waits for specific amounts of time.
   - Code improvements and TODO comments have been added.

3. `FAQ.md`:
   - Updated an issue related to starting LocalNet with SELinux on an operating system. Replaced the command `make compose_and_watch` with `make lightweight_localnet` to avoid permission denied errors.

4. `.gitignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.
   - Added the entry "**/gomock_reflect_*/" to ignore mock temporary files.

5. `e2e/README.md`:
   - Added a new section on `Keywords`.
   - Modified scenario descriptions and code examples to replace instances of "Validator" with "Node".
   - Included a flowchart depicting the E2E scenarios with updated terminology.

6. Consensus module files:
   - Added a new logging statement in the `HandleDebugMessage` function.
   - Simplified the handling of the `DEBUG_CONSENSUS_RESET_TO_GENESIS` action.

7. `.tiltignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.

8. `CHANGELOG.md`:
   - Updated build commands and added a new section on `Keywords`.

9. `persistence/docs/CHANGELOG.md`:
   - Several changes related to deprecation, addition, and fixing of functions and issues.

10. `validator.feature`:
    - Renamed file from "valdator.feature" to "validator.feature".
    - Updated scenario titles and step descriptions to use more descriptive terminology.
    - Replaced references to "validator" with "node".

11. `tilt_helpers.go`:
    - Added a new file containing functions related to syncing network configuration and checking package installation.

12. `debug.go`:
    - Added new debug commands and subcommands.
    - Updated existing functions and added new functions for debug actions.
    
13. `account.feature`:
    - Added a new file containing scenarios for testing node account functionalities.

14. `README.md` files:
    - Updated sections, titles, dependencies, and instructions in various README.md files.

15. `build/config/README.md`:
    - Updated usage instructions, changing the command `make compose_and_watch` to `make lightweight_localnet`.

16. `iteration_3_end_to_end_tx.md`:
    - Updated commands to start LocalNet and consensus debugger.

17. Deleted files:
   - `validator.go`
   - `watch_build.sh`

18. New files added:
   - `debug.feature`
   - `tilt_helpers.go`
   - `account.feature`

Please let me know if you need more information about any specific change.
<!-- reviewpad:summarize:end -->

## Issue

Fixes par of #579

## Type of change

Please mark the relevant option(s):

- [x] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [ ] Major breaking change
- [ ] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- `s/compose_and_watch/lightweight_localnet` and all related helpers
- `s/validator/node` in e2e tests for clarity
- Add fire-and-forget `Debug` CLI w/ several useful initial subcommands
- Add `keywords` to the `e2e` document
- Add an `e2e debug` test to trigger views and track the blockchain increasing
- Avoid rebuilding the actors if the CLI changes
- Small miscellaneous improvements & code cleanup

## Testing

- [x] `make develop_test`; if any code changes were made
- [ ] `make test_e2e` on [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any code changes were made
- [ ] `e2e-devnet-test` passes tests on [DevNet](https://pocketnetwork.notion.site/How-to-DevNet-ff1598f27efe44c09f34e2aa0051f0dd); if any code was changed
- [ ] [Docker Compose LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md); if any major functionality was changed or introduced
- [ ] [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any infrastructure or configuration changes were made

## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment))
- [ ] I have tested my changes using the available tooling
- [ ] I have updated the corresponding CHANGELOG

### If Applicable Checklist

- [x] I have updated the corresponding README(s); local and/or global
- [x] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s)
- [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s)

---

Co-authored-by: d7t <[email protected]>
@Olshansk Olshansk moved this from In Progress to Up Next in V1 Dashboard Aug 7, 2023
dylanlott added a commit that referenced this issue Aug 7, 2023
@Olshansk @dylanlott
[E2E Test] Utilities for State Sync Test (#874)
a6aac61 
## Description

<!-- reviewpad:summarize:start -->
### Summary generated by Reviewpad on 03 Aug 23 20:11 UTC
This pull request introduces several changes across multiple files. 

Here is a summary of the changes:

1. `go.mod`:
   - The `golang.org/x/text` dependency is now listed explicitly instead of being an indirect dependency.

2. `scenario_test.go`:
   - Added a new feature called "State Sync Namespace" that includes various commands and waits for specific amounts of time.
   - Code improvements and TODO comments have been added.

3. `FAQ.md`:
   - Updated an issue related to starting LocalNet with SELinux on an operating system. Replaced the command `make compose_and_watch` with `make lightweight_localnet` to avoid permission denied errors.

4. `.gitignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.
   - Added the entry "**/gomock_reflect_*/" to ignore mock temporary files.

5. `e2e/README.md`:
   - Added a new section on `Keywords`.
   - Modified scenario descriptions and code examples to replace instances of "Validator" with "Node".
   - Included a flowchart depicting the E2E scenarios with updated terminology.

6. Consensus module files:
   - Added a new logging statement in the `HandleDebugMessage` function.
   - Simplified the handling of the `DEBUG_CONSENSUS_RESET_TO_GENESIS` action.

7. `.tiltignore`:
   - Removed the entry "main" from the list of ignored files.
   - Removed the entries "rpc/server.gen.go" and "rpc/client.gen.go" from the list of ignored files.

8. `CHANGELOG.md`:
   - Updated build commands and added a new section on `Keywords`.

9. `persistence/docs/CHANGELOG.md`:
   - Several changes related to deprecation, addition, and fixing of functions and issues.

10. `validator.feature`:
    - Renamed file from "valdator.feature" to "validator.feature".
    - Updated scenario titles and step descriptions to use more descriptive terminology.
    - Replaced references to "validator" with "node".

11. `tilt_helpers.go`:
    - Added a new file containing functions related to syncing network configuration and checking package installation.

12. `debug.go`:
    - Added new debug commands and subcommands.
    - Updated existing functions and added new functions for debug actions.
    
13. `account.feature`:
    - Added a new file containing scenarios for testing node account functionalities.

14. `README.md` files:
    - Updated sections, titles, dependencies, and instructions in various README.md files.

15. `build/config/README.md`:
    - Updated usage instructions, changing the command `make compose_and_watch` to `make lightweight_localnet`.

16. `iteration_3_end_to_end_tx.md`:
    - Updated commands to start LocalNet and consensus debugger.

17. Deleted files:
   - `validator.go`
   - `watch_build.sh`

18. New files added:
   - `debug.feature`
   - `tilt_helpers.go`
   - `account.feature`

Please let me know if you need more information about any specific change.
<!-- reviewpad:summarize:end -->

## Issue

Fixes par of #579

## Type of change

Please mark the relevant option(s):

- [x] New feature, functionality or library
- [ ] Bug fix
- [ ] Code health or cleanup
- [ ] Major breaking change
- [ ] Documentation
- [ ] Other <!-- add details here if it a different type of change -->

## List of changes

- `s/compose_and_watch/lightweight_localnet` and all related helpers
- `s/validator/node` in e2e tests for clarity
- Add fire-and-forget `Debug` CLI w/ several useful initial subcommands
- Add `keywords` to the `e2e` document
- Add an `e2e debug` test to trigger views and track the blockchain increasing
- Avoid rebuilding the actors if the CLI changes
- Small miscellaneous improvements & code cleanup

## Testing

- [x] `make develop_test`; if any code changes were made
- [ ] `make test_e2e` on [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any code changes were made
- [ ] `e2e-devnet-test` passes tests on [DevNet](https://pocketnetwork.notion.site/How-to-DevNet-ff1598f27efe44c09f34e2aa0051f0dd); if any code was changed
- [ ] [Docker Compose LocalNet](https://github.com/pokt-network/pocket/blob/main/docs/development/README.md); if any major functionality was changed or introduced
- [ ] [k8s LocalNet](https://github.com/pokt-network/pocket/blob/main/build/localnet/README.md); if any infrastructure or configuration changes were made

## Required Checklist

- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have added, or updated, [`godoc` format comments](https://go.dev/blog/godoc) on touched members (see: [tip.golang.org/doc/comment](https://tip.golang.org/doc/comment))
- [ ] I have tested my changes using the available tooling
- [ ] I have updated the corresponding CHANGELOG

### If Applicable Checklist

- [x] I have updated the corresponding README(s); local and/or global
- [x] I have added tests that prove my fix is effective or that my feature works
- [ ] I have added, or updated, [mermaid.js](https://mermaid-js.github.io) diagrams in the corresponding README(s)
- [ ] I have added, or updated, documentation and [mermaid.js](https://mermaid-js.github.io) diagrams in `shared/docs/*` if I updated `shared/*`README(s)

---

Co-authored-by: d7t <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@red-0ne red-0ne

@Olshansk Olshansk

Labels
consensus Consensus specific changes documentation Improvements or additions to documentation testing Defining, adding, automating or modifying tests
Projects
V1 Dashboard
Status: Up Next
Milestone
M1: Pocket PoS (Proof of Stake)
Development

Successfully merging a pull request may close this issue.

[WIP] Block by Block Synchronous Sate Sync pokt-network/pocket
4 participants
@red-0ne @Olshansk @jessicadaugherty @gokutheengineer