Full Scale Performance: Sharrow Off

[dhensle]

This is the issue to report on memory usage and runtime performance without using sharrow...

[dhensle]

Ran with a 100 percent sample and single process. Machine has 1TB of RAM and a 2.79 GHz CPU. Memory peaks at about 240GB. Run time was approximately 21 hours.

logs.zip

[dhensle]

Multiprocessing:

households_sample_size: 0 (100 percent sample)
sharrow: False
multiprocessing: True
num_processors: 24

Completed in 173 minutes, or 2.9 hours
log_no_sh_full_mp.zip

[dhensle]

Varying the number of cores on the RSG machine with 500 GB of RAM and 24 cores. (Same exercise as performed for the abm3 model here)

Observations:

• Same sort of diminishing returns as seen in all the other profiling runs when adding more and more cores.
• Interestingly, there was no inflection point for the MTC model -- more cores means lower runtime. This was different with sharrow on and different compared to both sharrow on and off ABM3 benchmarking runs.

[asiripanich-dtp]

@dhensle – Thanks for sharing these benchmark results. Do you have similar benchmarks using ActivitySim v1.2? It would be really interesting to see how much improvement in both runtime performance and memory usage the new version shows compared to v1.2.

[dhensle]

@asiripanich-dtp -- I do not directly have those comparisons, but you can find some of that work / discussion in this issue: ActivitySim/activitysim#904