You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the runtime, if an option is set, we should profile memory usage.
It is useful to know the maximum/peak memory used by a step (and indeed by a whole workflow)
At the moment we log the final heap memory at the end of each step. But this doesn't tell us much really - GC has probably run after the most expensive operations. I don't actually think we log a "workflow total"?
For this to work we need to manually poll process.memoryUsage() and save the highest/peak memory used while the workflow is running. We might want a count per step and per workflow. We should poll often - ever 5 or 2 s. We can take an argument to set the poll rate.
Polling is cheap but even so, the more you do it the more it'll slow down runs.
ALSO! Rename "final memory usage" log to "step memory usage", because the final thing is super confusing
We need to decide whether to default this in the worker. It would be good to compare some jobs to assess performance impact.
Activity