Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templates testing #370

Open
dkazanc opened this issue Jun 17, 2024 · 1 comment
Open

Templates testing #370

dkazanc opened this issue Jun 17, 2024 · 1 comment

Comments

@dkazanc
Copy link
Collaborator

dkazanc commented Jun 17, 2024

Don't think that we've got an issue on that, @jessicavers ?

I believe that for that work we need to create a simulated test data. We need to have a ground truth projection data and also a phantom. I'll create a separate issue on this #371. The main reason for that is that later we can also check the what the method actually does make sense.

The templates tests should probably do the following:

  • Each test automatically appends the loader which loads the synthetic dataset.
  • HTTomo executes the resulting pipeline.
  • Tests the result based on the type of the method using the initial input and ground truth datasets. For the majority of the methods that are not reconstruction algorithms, we need to use the synthetic projection data for testing. For reconstruction methods we can use the synthetic phantom instead.
  • Using some quantitative measures for the result of the test we can calculate the RMS error for the input compared to the output (it should be above a very small tolerance value, so that there has been a change to the input by the method). We cal also calculate RMSE of output compared to the ground truth data, it should be also non-zero value and larger than the value of RMSE for input and output. Then also the result shouldn't contain Nan's and Inf's. To conclude, here is a list of measures:
  1. RMSE(input, output) > tolerance
  2. RMSE(output, groundtruth) > tolerance AND RMSE(output, groundtruth) > RMSE(input, output)
  3. No Inf's and Nan's
  4. max(output) < large tolerance value
@jessicavers
Copy link
Collaborator

Thank you for detailing it here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants