Echo Server: Simplify benchmarking setup - #757
Conversation
This commit removes a #define that was guarding utilisation information from being captured on x86. From my perspective x86 seems to have the same support as risc-v (pmu not yet supported) thus we make the #defines match. This commit removes the idle thread setup. seL4 already has an idle thread, and in the benchmark configuration the kernel tracks its utilisation like any other normal thread. Thus we can retrieve identical (if not more accurate) information about idle utilisation from seL4. This commit removes the total cycles and idle cycles print by ip bench, since we log this in the benchmark pd now. This also simplifies the implementation & reduces the sloc to achieve the same thing. Signed-off-by: Callum <c.berry@student.unsw.edu.au>
370e94a to
8f84f94
Compare
|
link to one article discussing how idle time can be calculated, With respect to the accuracy of this accounting the kernel provides as an example: |
|
Those should definitely be three separate commits, and maybe separate PRs. I'm pretty sure we have an explicit idle thread for a reason - for one, on certain platforms (mostly ARM32 ones to be fair) the kernel WFI turns off the cycle counter's clocks to save power, which affects measurement - for two, I believe it was mentioned as feedback for one of the papers we submitted, that measuring idle cycles in a user thread represents a more accurate measurement of idle time because it represents actually useable time by other threads; whereas using the kernel idle thread has less overhead compared to someone doing real work that requires an extra mode switch to userspace. |
|
Yes, we discussed this with Peter yesterday, we don't want to remove the idle thread (it is indeed there for a reason).
This should definitely be a separate PR. Please make one. |
| strcat(buffer, ibuf); | ||
| strcat(buffer, ","); | ||
| strcat(buffer, tbuf); | ||
| strcpy(buffer, RES(5, 0, 0)); |
There was a problem hiding this comment.
BTW you are supposed to be reporting your CPU utilisation to IPbench here, which it seems that you no longer do...
I included this in ixgbe driver PR, which reads TSC in idld thread. I can make a separate commit/PR once the ixgbe driver code is fine to get merged. |
|
I see, maybe there is no choice but to compute idle time based off a user-level idle thread given this was requested in feedback. It's just unfortunate because based off of what I've read about computing idle time in networking benchmarks online, counting the CPUs true idle cycle count is the preferred method. For instance the netperf docs go into a decent amount of detail in how they prioritise different ways of computing idle time, and they only use the user-level idle thread when there is not a more accurate alternative such as an accurate kernel idle thread counter. But, oh well, I guess we are just being conservative about any idle time reported. If we do want to compute the cycles spent in an idle user-level thread could the current implementation be simplified (reducing a significant amount of C and python code) and potentially made more accurate by using the kernels counters? More specifically, since the kernel counters provide the total utilisation and the kernel utilisation of each TCB, can't we take the difference, which should yield almost the exact number of cycles the idle thread runs in user level? This could avoid inaccuracies associated with not counting cycles to get into the user level loop, assuming idle time is only possible when a loop iteration takes less than a constant #defined amount of cycles, (for example pipeline stalls, cache misses etc could also occur in the idle thread resulting in a longer loop iteration but we might not count these cycles). |
|
As a side note, the reported kernel utilisation under smp is not accurate AFAIK (see issue #570). |
|
Oh jeez, it seems like your solution of making the benchmark info core local is clean though? And then the user can combine the data from each core themself? Or is there some edge case where you can't do this? |
|
I'm not convinced that my fixes are complete; there could be some other holes. Further careful comparisons are desired. |
Something like this could potentially work, but you would need to arrange something so the echo server client also has access to these values as well as the benchmark PD. Which is possible but would require more code. |
5 participants
This commit removes a #define that was guarding utilisation information from being captured on x86. From my perspective x86 seems to have the same support as risc-v (pmu not yet supported) thus we make the #defines match.
This commit removes the idle thread setup. seL4 already has an idle thread, and in the benchmark configuration the kernel tracks its utilisation like any other normal thread. Thus we can retrieve identical (if not more accurate) information about idle utilisation from seL4.
This commit removes the total cycles and idle cycles print by ip bench, since we log this in the benchmark pd now. This also simplifies the implementation & reduces the sloc to achieve the same thing.