Repository navigation
Conversation
The from-distribution and from-sources pipelines created SolrProvisioner without telemetry_devices, so --telemetry jfr/gc/jit added nothing to SOLR_OPTS. Build the enabled devices from telemetry.devices and telemetry.params and pass them in. Output goes to the test run's telemetry directory. Docker is left out on purpose: the flags point at host paths, which the JVM inside the container cannot open. That part needs the log storage change discussed in apache#138. Part of apache#138
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
--telemetry jfr,gcandjitdid nothing underrun.SolrProvisioneralready appends each device'sinstrument_java_opts()toSOLR_OPTS, butsolr_from_distributionandsolr_from_sourcesnever passed it anytelemetry_devices.This PR adds
_create_jvm_telemetry_devices(cfg)totest_run_orchestrator.py. It buildsFlightRecorder,GcandJitCompilerfromtelemetry.devices/telemetry.paramsand returns only the enabled ones. It has to filter because the provisioner adds flags from every device it gets, not just the enabled ones. Both pipelines now pass the result toSolrProvisioner. Output goes to<test-run-dir>/telemetry/, sogc.log,profile.jfrandjit.logsit next totest_run.json. That directory is also outside the install dir, whichclean()deletes on teardown.I left the
dockerpipeline unchanged on purpose. The device flags contain host paths, and the JVM in the container exits on startup if it can't open them. Withsolr:9I getError opening log file '...gc.log': No such file or directory, thenCould not create the Java Virtual Machine. Docker needs the log-storage change discussed in #138, so it comes in the second PR, together withheapdump.docs/reference/telemetry.mdnow listsfrom-distribution/from-sourcesfor these three devices and gives the output path. The multi-device example also usesfrom-distributioninstead ofdocker.Issues Resolved
Part of #138 (this covers from-distribution and from-sources; Docker and heapdump follow in a second PR)
Testing
New tests in
tests/test_execution_orchestrator_test.py:test_from_distribution_passes_jvm_telemetry_devicesandtest_from_sources_passes_jvm_telemetry_devicesrun each pipeline withSolrProvisionermocked and check thetelemetry_devicesit receives. Onmainthey fail withKeyError: 'telemetry_devices'.test_jvm_telemetry_devices_add_their_flags_to_solr_optspasses the devices to a realSolrProvisionerand checks that_build_env()["SOLR_OPTS"]contains the-Xlog:gc...:file=.../gc.log,StartFlightRecording...filename=.../profile.jfrand-XX:LogFile=.../jit.logflags. It also checks thatgc-log-configfrom--telemetry-paramsis respected.segment-statsin the list is ignored) and nothing when--telemetryis empty.The full
tests/suite passes locally on Python 3.13 (1180 passed, 5 skipped), and so doesruff check .. I haven't run the integration suite (it/).By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.