Benchmarking and validation functionality to evaluate cellfinder, as semi-automated high-level integration tests

[alessandrofelder]

Follows on from #536

Initial plan:

• a combination of brainglobe workflow benchmarks to evaluate performance
• regression tests against test data agreed in Document a list of real-life, diverse(CFOS and rabies-tracing) test datasets that allow us to evaluate correctness and performance of cellfinder #536, maybe re-using functionality from Add cell matching support brainglobe-utils#74

• some scripts to allow iterating over commit SHAs and datasets? (doesn't have to be user-facing, running these scripts would constitute the manual part)

(@IgorTatarnikov and @alessandrofelder - and maybe @sfmig - plan to have a go at this on 27th June.)

[adamltyson]

Could we rename this issue? It doesn't mean much out of context.

[matham]

There's some testing already with existing datasets. Namely we use this dataset to test it here end-to-end that the 2d/3d filters result in the same cells being detected.

Similarly we use the same dataset to test the 2d filters alone here.

I can add cfos datasets, once that dataset is detected well. But the problem here is the speed because these all take a while. The dataset above is fairly small. Perhaps we can have some larger cfos and serial2p datasets that is run weekly? Then we can generate output for all the steps in 2d and 3d and test that, like in the linked tests above?

[alessandrofelder]

Perhaps we can have some larger cfos and serial2p datasets that is run weekly?

Yes, something like this would be good, I think