profiling (don't merge!) #25
Conversation
WITHOUT simplification/checking/computing tree sequence or other - about 60% of time is spend in `argrecorder.py` - most of this (46%) is in `argrecorder.py:add_record` and 3/4 of that in stuff that is not `msprime` (which takes up total of 10% of the 46% for argrecorder.py in the flame graph from command below) --- so mostly the logic in `argrecorder.py:merge_records` and buildin logic of OrderedDict. you can re-run (from project root after first `pip install vprof`): `vprof -c cmp tests/test_recomb_collector.py`
Codecov Report
@@ Coverage Diff @@
## master #25 +/- ##
=========================================
- Coverage 99.36% 4.4% -94.97%
=========================================
Files 3 3
Lines 158 159 +1
=========================================
- Hits 157 7 -150
- Misses 1 152 +151
Continue to review full report at Codecov.
|
To re-run ```sh pip install line_profiler kernprof -v -l tests/test_arg_recorder_with_wf.py > devel/prof.txt kernprof -v -l tests/test_recomb_collector.py > devel/prof_rc.txt ```
devel/prof_rc.txt
Outdated
| 54 559608 242579 0.4 0.8 children=children, | ||
| 55 559608 238853 0.4 0.8 left=left, | ||
| 56 559608 1555702 2.8 5.4 right=right) | ||
| 57 559608 24781697 44.3 85.9 merge_records(new_rec, self[parent][1]) |
There was a problem hiding this comment.
The merge records call is taking most of the time in add_record
|
Results of line profiling arg_recorder are in |
| 74 1 146 146.0 0.0 matingScheme=mating_scheme_factory(recombinator, popsize), | ||
| 75 1 45696018 45696018.0 100.0 gen=generations | ||
| 76 ) | ||
| 77 1 1 1.0 0.0 return pop |
There was a problem hiding this comment.
pretty slow vs just simupop, as 100% of the time is spent in recombinator
There was a problem hiding this comment.
Nevermind. I think I'm misinterpreting this
ashander
changed the title
vprof for running w/ RecombCollector
| 147 Keeping them in sorted order simplifies the procedure | ||
| 148 (makes it so we don't have to split the new record). | ||
| 149 ''' | ||
| 150 559608 1563370 2.8 9.6 k = 0 |
There was a problem hiding this comment.
strange. makes me doubt the other results here
There was a problem hiding this comment.
huh, yes. it would hit this without continuing to the rest of the function quite a bit, when we add offspring to new individuals (so len(existing) is zero). maybe percentages are "average percent of the time spent on this line for each time the function is called" rather than "percent of total time spent in this function that was spent on this line"?
WITHOUT simplification/checking/computing tree sequence or other
argrecorder.pyargrecorder.py:add_recordand 3/4 of that in stuffthat is not
msprime(which takes up total of 10% of the 46% forargrecorder.py in the flame graph from command below) --- so mostly the logic
in
argrecorder.py:merge_recordsand buildin logic of OrderedDict.you can re-run (from project root after first
pip install vprof):vprof -c cmp tests/test_recomb_collector.py