Add HTML instrumentation report types, registry & template generation#196
Open
AlexJones0 wants to merge 6 commits into
Open
Add HTML instrumentation report types, registry & template generation#196AlexJones0 wants to merge 6 commits into
AlexJones0 wants to merge 6 commits into
Conversation
Define a common protocol for instrumentation report visualisations with a rendering interface. The idea is to produce a single HTML instrumentation report with various different visualisations (which themselves could be graphs, images, text, etc.) which can be switched between. The rendered outputs themselves are just embedded HTML fragments (or `None` on failure). This should be simple enough whilst allowing the level of extensibility and flexibility that is needed for the current level of instrumentation reporting. Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
Add a registry (like the existing registry/factory for the different instrumentation types) that we can use to add new instrumentation visualisations. In the same manner that we currently support plugins to extend the instrumentation functionality of DVSim, this will allow users extending DVSim to hook in and add their own custom instrumentation visualisations as well. Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
This will allow reports to add custom content to the HTML head if desired for e.g. styling purposes. Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
This commit introduces the instrumentation report template itself, and a function `render_html_report` for rendering the instrumentation report with a list of given instrumentation report visualization implementations. Some key things to note: - We use `plotly.offline` to get the version of the minified plotly.js that is packaged with the plotly Python package. Like vendored static files this will still work in an airgapped environment, but this way it should remain in sync even when the plotly dependency has its version updated. - We only include plotly JS (which is around 4MB minified) as a dependency when we actually have renders that might use it. - For now we include all visualizations as different tabs on the same page. This is to keep things simple as the goal is to keep all the report logic relatively small and self-contained, but this could be expanded upon in the future if it is deemed too restrictive of an interface. Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
Whenever a DVSim scheduler run completes and we flush the JSON instrumentation report, add additional logic to create the HTML instrumentation report as well - under the same report directory that is currently being used for the existing simulation HTML report outputs. It is not 100% clear whether this is the best place to integrate the instrumentation reporting yet. It seems like it might be more sensible to do it alongside the simulation reporting to keep common functionality grouped together, but this then means that we do not get the same benefits for other flows (e.g. linting, formal) that do not have their own custom HTML reports. So for now, we keep this in the generic scheduler running logic. Signed-off-by: Alex Jones <alex.jones@lowrisc.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR is the sixth of a series of PRs to introduce instrumentation reporting to DVSim.
This PR adds an initial basis for the instrumentation HTML report generation. It features:
A few important things to note:
plotlypython package (so we don't need to worry about vendoring & versions becoming out of sync). We only generate the staticplotly.min.jsfile if there are any visualizations available to actually use plotly.metrics.htmlanywhere from the main sim flowindex.htmlpage. This will be added later.See the commit messages for more info & context about each change.
Currently, the generated
metrics.htmlreport will look something like:With the
JSONbutton taking you to the existingmetrics.jsonfile.