Skip to content

Optimisations required for stage IV simulations#10

Open
jlvdb wants to merge 239 commits intomasterfrom
abandon-FITS-support
Open

Optimisations required for stage IV simulations#10
jlvdb wants to merge 239 commits intomasterfrom
abandon-FITS-support

Conversation

@jlvdb
Copy link
Contributor

@jlvdb jlvdb commented Sep 15, 2020

  • Added support for Flagship.
  • Switched from FITS files to memory mapped binary files as intermediate storage for the pipeline.
    • Great performance gains, improved parallelism.
    • Reduced disk space requirements by optimizing how galaxy sample selection information is stored (using bit masks).
  • Improved documentation.
  • Bug fixes.

This restructured pipeline reproduces the KV450 MICE2 mock catalogues and spec-z samples from van den Busch et al. 2020 within the statistical sampling uncertainty (modulo bug fixes).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant