Multiple slurm jobs writting to one file is bad practice

[avivajpeyi]

Hi Avi,

the below file is being appended to from jobs across 14 nodes. please don't do that.

14 = /fred/oz200/avajpeyi/projects/atlas_runs/oct9_out/0.2.1.dev64+gc7fa3a0/tess_atlas_runner.log = 14 j27 14 j34 14 j49 14 j50 14 j64 14 j7 14 j71 14 j75 28 j44 28 j48 28 j70 28 j79 70 j51 84 j52
https://supercomputing.swin.edu.au/docs/1-getting_started/Filesystems.html#other-things-to-avoid

cheers,
robin

woops 😅

[avivajpeyi]

Fixed at some point with the changes to the slurm+logging system.

[avivajpeyi]

I still have the all jobs appending one line to a common file at the end of the job

tess-atlas/src/tess_atlas/notebook_controllers/controllers/toi_notebook_controller/toi_notebook_core.py

Line 124 in 2a5da69

TOIRunStatsRecorder.save_stats(

This should be changed such that all jobs write their own file and these files are merged at the end.