Test Coverage / Quality Umbrella

[HalosGhost]

There are a great deal of tests that should be added, and many additional branches that should be covered. To that end, this issue is meant to be a centralized place to keep track of tests that have been identified as good opportunities for improvement. We will try to keep the task list below up-to-date; so if you think another item should be added, feel free to comment it below and we will add to the list as makes sense!

Note: task lists allow for each item to easily be converted into a stand-alone issue; If a particular test/improvement merits deeper discussion as someone seeks to implement it, we can always make that change!

• Add tests to check the thread-safety of any/all classes which are expected to be thread-safe
• Add unit test for cbdc::from_buffer(nuraft::buffer&) #111
• Add unit test to cover non-error cases of cbdc::nuraft_serializer::read() #117
• Add unit tests for cbdc::raft::rpc::server
• Add unit test to cover error and some non-error cases of cbdc::raft::node #133
• Add integration test to check Atomizer snapshotting #108
• Add replication test to specifically check that Atomizers handle node failure gracefully
• Parallel Shard Replication/Partitioning Integration Tests #2
• Add integration test to check that Shards always attest at the correct block height
• Add integration tests for Locking Shards
• Add integration tests for Coordinators
• Add integration tests for 2PC Sentinels
• Add integration tests for Atomizer Sentinels
• Craft More Intricate Config Files #136
• Add support for OS-assigned ephemeral port numbers and leverage them in-tests
• Fix network::tcp_listener::listen() to fail when binding to an invalid endpoint #11
• Add github action to check that new PRs don't decrease test coverage
• Refactor test suite to support starting components asynchronously #127

[mszulcz-mitre]

I'd like to contribute to the project and it seems like writing a few tests is a good entry point. I've started looking into the 2nd task on the list, 'Add unit test for cbdc::from_buffer(nuraft::buffer&)'. Is anyone working on this yet?

[HalosGhost]

@mszulcz-mitre Not that I'm aware of; Feel free to hit the Convert to issue button, and dig in!

[mszulcz-mitre]

@HalosGhost Thanks for the quick reply! Ready to dig in! I'm having some trouble converting the task to an issue though. According to the GitHub page about task lists, you're supposed to be able to hover over a task and then click a circle with a dot in it. I've opened this page in 3 different browsers, but I don't see any pop up when hovering on a task. If I manually open an issue, could I ask you to link it to the task? I don't think I have permission to edit the original issue post.

[HalosGhost]

@mszulcz-mitre happily!

I wondered about that; I think you're not the first one who's not been able to do that, but GitHub's documentation is unclear on who is allowed to hit that button. Please feel free to open the new issue; I'll edit the original task-list appropriately.

[ykaravas]

Hello, my name is Yiannis Karavas (with MITRE) and I'd like to start contribute to the project. I have begun work on "Improve test for cbdc::nuraft_serializer::read() to cover non-error cases". I cannot click the "Convert to issue" button either, so should i also just create a new issue?

[HalosGhost]

@ykaravas indeed; feel free to just reference this issue in the one you create and I'll update/add it to the list as-needed. :)

[ykaravas]

@HalosGhost I would like to take a crack at the task that reads " Craft a config that can be used as a basis for most tests". I think this will be helpful in furthering my understanding of the code and will obviously be useful for testing etc... I was wondering if you could elaborate a bit on what is meant by "as a basis for more tests"?

• Are we looking for larger and more complex config files?
• Are we looking for config files that will test specific scenarios?
• Should these new config file(s) target specific integration tests?
• Also, what do you think about creating some sort of simple tool (maybe in the tools folder or tests folder) that could either be a simple C++ program or a bash script that could autogenerate a large config file from a few parameters? For example, one could put number of wallets, shards, atomizers, sentinels etc.... Eventually, we could have parameters like average transactions per second and maybe even parameters that could be something like probability of failure for each type of component that could, in an integration test that would ingest these, simulate such failures and such volumes of transactions and many more dynamics as well I'm sure. I am not very familiar with the existing testing framework or the benchmarking tools you have used but let me know if you think this sounds useful. I suppose, however, I am thinking about this mainly, in a non-docker context. But, I can envision an integration test that could parse the autogenerated configuration file, use docker-cli spin up all the docker containers required in the proper network and issue commands such as transactions, minting and simulating outages based on parameters in the file.

[HalosGhost]

@ykaravas I think it probably makes sense for that discussion to happen under that issue (feel free to open it and repost your questions), and I'll answer there.

However, in broad strokes, I can tell you my original motivation for adding that task: I thought (and still think) that having a more complex and varied configuration would be helpful. Namely, I think it might serve as a solid test bed for a system-test suite, as well as potentially being useful for many of our integration and unit tests (depending on what's being tests). Further, I think it might serve as a nice diving-in point for people who want to experiment with the system on a larger scale.

Additionally, I love the idea of a configuration generator (though, I'd wonder if it might start to have overlap with #81 and I'm unsure of which would end up being the right path to explore).