-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a profiler leveraging the contextual logging #3888
Comments
I would start very coarse grained, and only make it more fine-grained as we discover things. I would start with:
Then, whichever of those cost-centers is the biggest, break down into smaller cost-centers. I guess the granularity can (or should) be tunable in the tool which reads and processes the logs, rather than having to re-run the proofs. So that's good because you can have very fast iteration on looking at cost-centers even for a long-running proof (if we store the log file). |
Also, maybe visualizing the cost-centers and transit between them is useful. So building a graph where each type of log event we care about is a node, and edges between nodes are weighted with how much cumulative time is spent on that edge. That can give us an overall idea of how control flow is broken down in a more fine-grained manner. @jberthold made some diagrams like that for the old backend a long time ago. |
The code to process the event logs has moved to https://github.com/runtimeverification/kore-prof/tree/master/src/TSM (currently unmaintained). The legacy I found an old file showing such a graph for the rule application during a test (evm-semantics, IIRC): |
* Introduces a JSON-enabled type for the log context * Uses the new type in all booster logging * Modifies `count-aborts` to parse the json data using the new type. Performance is on par after switching to TH-derived `FromJSON` instances. Conversion of `kore-rpc` logging is left for future work. Part of #3888 --------- Co-authored-by: github-actions <github-actions@github.com>
Motivation
When we try to understand why proofs are slow, we only have either very coarse-grained or very find-grained timing data. The coarse-grained data is either wall-clock time of running
kontrol
and other tools, or, at the level ofkore-rpc-booster
, the--log-stats
option which times the requests. The fine-grained tool is the GHC profiler, which gives us Haskell-level cost-centres.To better inform our decisions on what to work on to improve performance, we need something that sits in between.
Use cases
booster
aborts on rewrite rules (falling back tokore-rpc
)execute
andsimplify
requests (internal and external)Functionality
We would like to have a proof-level profiling tool that would allow us to find out, for both Booster and Kore:
Additionally, for Booster we would like to know:
Data source
We have implemented the contextual JSON logging, which, in conjunction with timestamps, should give us all the information we need. We may need to disable the buffering of the timestamps in
fast-logger
or, of that is not possible, figure out another to provide timestamps.Tool workflow and design
The tool's input will be the JSON log file, with every line being a json object. We should design the tool to support streaming the JSONs from the file so that we do not load it all into memory.
The log file will be generated by running the server with context logging enabled. This can be either done directly by the downstream tooling, i.e.
kontrol prove --profile
or by a script that consumes bug reports and runs the server with logging enabled.The output of the tool will likely be a Markdown/HTML document.
The text was updated successfully, but these errors were encountered: