Adds performance annotations using the new Hatchet annotation interface#195
Adds performance annotations using the new Hatchet annotation interface#195ilumsden wants to merge 1 commit intollnl:developfrom
Conversation
There was a problem hiding this comment.
@ilumsden Can you remind me - is there anything important about the order of the decorators or are they applied independently? Just wondering what the outcome will be for some functions with the cache_stats_op and perf annotations.
Otherwise looks good, since this is just annotations and the functionality is in hatchet.
michaelmckinsey1
left a comment
There was a problem hiding this comment.
A couple of comments:
- Is it possible to add metadata with the pycaliper interface?
- I think the amount of annotations is causing high overhead. See following:
Time to create Thicket with X amount of files.
| files | pycaliper | develop |
|---|---|---|
| 1,000 | 36s | 8s |
| 10,000 | 15m36s | 1m26s |
I notice Calls for Node.__eq__ is 96,174,300 for the thicket with 10,000 files. By removing all of the node annotations in Hatchet for pycaliper time goes from 15m36s to 3m. So I think we may have to be more selective with what we annotate or maybe provide some kind of different levels for profiling.
|
Thanks for taking a look @michaelmckinsey1. That timing info is very interesting. I definitely agree that the annotations need to be paired down given the timing. |
This PR adds performance measurement annotations to Thicket using the APIs added to Hatchet in llnl/hatchet#142.