ENH: Parallel mode for monte-carlo simulations #619
Conversation
brunosorban
changed the title
|
Benchmark of the results. A machine with 6 cores(12 threads) was used.
|
There was a problem hiding this comment.
Amazing feature, as the results show the MonteCarlo class has great potential for parallelization.
The only blocking issue I see with this PR is the serialization code. It still does not support all of rocketpy features and requires a lot of maintanance and updates on our end.
Do you see any other option for performing the serialization of inputs?
@phmbressan we should make all the classes json serializable, it's an open issue at #522 . In the meantime, maybe we could still use the _encoders module to serialize inputs. I agree with you that implementing flight class serialization within this PR may conflict create maintenance issues for us. The simplest solution would be to delete the |
|
Converted to draft until you solve the remaining issues, specially the random number generation problem, |
TST: Add slow testing for different ode solvers. MNT: Move ode solver validation to separate method.
* BLD: add a flight example to rocketpy "Defiance" rocket flight example was added to doc\examples as my (Arthur Hwang) challenge submission for the Team Recruitment * DOC: Add Defiance launch to flight examples graph. * Updates CHANGELOG --------- Co-authored-by: Pedro Bressan <[email protected]> Co-authored-by: Gui-FernandesBR <[email protected]> DOC: add Defiance flight example (#742) * BLD: add a flight example to rocketpy "Defiance" rocket flight example was added to doc\examples as my (Arthur Hwang) challenge submission for the Team Recruitment * DOC: Add Defiance launch to flight examples graph. * Updates CHANGELOG --------- Co-authored-by: Pedro Bressan <[email protected]> Co-authored-by: Gui-FernandesBR <[email protected]>
* ENH: Create a dataset of pre-registered motors. See #664 I followed the recommendation "Download and save several .eng files in the repo so we can install it along with the rocketpy package itself". The website thrustcurve.org was very useful to search for some .eng files. I mainly focused in some of the main brands on the market: Cesaroni, Aero Tech, Animal Motors and Loki. And also focused on classes K to M, because this is the main range of total impulse that I'm used to seeing in rocketry. I tried to pick motors with a difference of about 300~600Ns in total impulse. Some more improvements than can also be made following this issue are expanding the dataset for whole SolidMotor objects, more than only thrust curves. I think this would be what the recommendation "Save .json files with all the information we may find available on internet" could mean. I decided to go for the simple for now, but having the thrust curves is a good first step to implementing that in the future, which I would totally be able to do! * Update CHANGELOG.md * MNT: git rename motor eng files in data folder * ENH: Create a dataset of pre-registered motors. See #664 I followed the recommendation "Download and save several .eng files in the repo so we can install it along with the rocketpy package itself". The website thrustcurve.org was very useful to search for some .eng files. I mainly focused in some of the main brands on the market: Cesaroni, Aero Tech, Animal Motors and Loki. And also focused on classes K to M, because this is the main range of total impulse that I'm used to seeing in rocketry. I tried to pick motors with a difference of about 300~600Ns in total impulse. Some more improvements than can also be made following this issue are expanding the dataset for whole SolidMotor objects, more than only thrust curves. I think this would be what the recommendation "Save .json files with all the information we may find available on internet" could mean. I decided to go for the simple for now, but having the thrust curves is a good first step to implementing that in the future, which I would totally be able to do! Update CHANGELOG.md MNT: git rename motor eng files in data folder --------- Co-authored-by: Gui-FernandesBR <[email protected]>
* DOC : Add weather file for simulation * DOC : Add data for simulation example * DOC: Add flight simulation example * DOC: Update index * DOC: Update Changelog * Update docs/examples/index.rst --------- Co-authored-by: Gui-FernandesBR <[email protected]>
* DOC: Add data for lince example * DOC: create simulation file * DOC: Update simulation file * DOC: Add euroc_2023 weather data * DOC: Add flight data * DOC: Update flight sim * DOC: Changelog update & Run black * DOC: Small fixes to fligh sim * DOC : Run black * DOC: Simulation fixes and index sim add * DOC: Update environments in flight documentation * DOC: fix Lince values in the rst file --------- Co-authored-by: Gui-FernandesBR <[email protected]> Co-authored-by: Gui-FernandesBR <[email protected]>
* DOC: Improve flight examples documentation Update flight simulation documentation with improved markdown headers and replace matplotlib with Plotly for enhanced visualizations * DEV: update changelog * DOC: Fix title in camoes simul * DOC: Update docs/requirements.txt to include new dependencies --------- Co-authored-by: LUCKIN13 <[email protected]>
* DOCS: add data for bme suborbitals flight example * DOCS: add simulation file for bme suborbitals flight example * DOCS: Error in motor fixed * DOC: Improve flight examples documentation Update flight simulation documentation with improved markdown headers and replace matplotlib with Plotly for enhanced visualizations * DEV: update changelog * DOC: Update simulation and flight data * DOC: Add weather file for simulations * DOC: Updates to erebus flight sim * DOC: Update changelog * DOC: Fix title in camoes simul * DOC:Update flight data * DOC: Comparison plots * MNT: Delete unnecessary file * DOC: Update index * DOC: run Black * DEV: update changelog * DOC: small fix index --------- Co-authored-by: Gui-FernandesBR <[email protected]> Co-authored-by: Gui-FernandesBR <[email protected]>
* MNT: move piecewise functions to separate file closes #667 * improved import for linting * MNT: applying code formaters * ENH: simplifying and optimizing the function, implementing tests. * MNT: update changelog and apply changes suggested in review --------- Co-authored-by: Lucas Prates <[email protected]> Co-authored-by: Lucas de Oliveira Prates <[email protected]> Co-authored-by: Gui-FernandesBR <[email protected]>
phmbressan
force-pushed
the
enh/parallel_montecarlo
branch
from
53ba8ed to
00d9d02
Compare
…s inside methods of Components
Lucas-Prates
requested review from
phmbressan,
MateusStano and
Gui-FernandesBR
|
I believe this PR is ready again for another round of review. These are the changes since the previous review:
Overall, it seems that the time per iteration is even faster now, at least by my local measurements. @phmbressan might want to complement the information provided here, he knows this PR much better than I do! Please, make sure to take a careful look at the Monte Carlo .input file to check that there is indeed no dependency on the generated random variables. |
|
Another important issue: I currently can not interrupt the |
Gui-FernandesBR
force-pushed
the
develop
branch
from
2931c30 to
7864590
Compare
|
Please follow this one: #768 |
This pull request implements the option to run simulations in parallel to the
MonteCarloclass. The feature is using a context manager namedMonteCarloManagerto centralize all workers and shared objects, ensuring proper termination of the sub-processes.A second feature is the possibility to export (close to) all simulation inputs and outputs to an .h5 file. The file can be visualized via HDF View (or similar) software. Since it's a not so conventional file, method to read and a structure to post-process multiple simulations was also added under
rocketpy/stochastic/post_processing. There's a cache handling the data manipulation where a 3D numpy array is returned with all simulations, the shape corresponds to (simulation_index, time_index, column).columnis reserved for vector data, where x,y and z, for example, may be available under the same data. For example, undercache.read_inputs('motors/thrust_source')time and thrust will be found.Pull request type
Checklist
black rocketpy/ tests/) has passed locallypytest tests -m slow --runslow) have passed locallyCHANGELOG.mdhas been updated (if relevant)Current behavior
In the current moment, montecarlo simulations must run in parallel and all outputs a txt file
New behavior
The montecarlo simulations may now be executed in parallel and all outputs may be exported to a txt or an h5 file, saving some key data or everything.
Breaking change
Additional information
None