Repository navigation
Conversation
The agenda and callback decorators parsed function bodies with inspect.getsource, which fails when a file is executed as a string: matplotlib's plot directive runs example files that way, and those code objects carry the filename <string> with no linecache entry. Add utility._source_and_location(), returning source, filename and first line number in one step. It tries inspect first - a single getsourcelines walk instead of getsource, getfile and getsourcelines - and otherwise reads candidate files (co_filename, __file__, sys.argv[0]), locating the function at co_firstlineno. A located block is accepted only if recompiling it matches the live code object in argument counts, variable names, used names and constants, plus bytecode when no free variables are involved, so a decoy definition in an unrelated file cannot silently produce a wrong agenda. Functions are unwrapped first, or functools.wraps would hand out the wrapper's source, which matches the wrapper's own code object exactly. _arts_agenda, in_parallel and _callback_operator use the new helper; _arts_agenda now takes the location from its caller instead of looking it up a second time. Add tests/core/agenda/source_from_exec.py covering exec-string agendas, callbacks and in_parallel, indented functions, decoy rejection, fast/slow path agreement and wrapped functions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
@arts_agenda,@callback_operatorand@in_parallelparse the decoratedfunction's body with
inspect.getsource. That only works throughlinecache, so it fails for any function defined by executing a file as astring — exactly how matplotlib's plot directive (and Sphinx-Gallery style
tooling) runs ARTS example files. The code objects of such functions carry
the filename
<string>and no linecache entry, and the decorators raisedOSError: could not get source code.What changes
Adds
pyarts3.workspace.utility._source_and_location(func), returning(source, filename, first_line)in one step, and switches the threedecorators to it.
inspect.getsourcelines+getfile— strictlyfewer
inspectwalks than the old code, which didgetsource+getfile+getsourcelines(two full source searches per agenda).(
co_filename,__file__,sys.argv[0]— the latter is where the plotdirective keeps the original example path), locates the definition at
co_firstlineno, and takes the block viainspect.getblock.matches the live code object — argument counts, local variable names,
used names, bytecode (when no free variables are involved) and
constants. The constants check (
_consts_match) is needed becausebytecode alone pins the position of literals but not their values:
two same-line agendas differing only in
species=["H2O"]vsspecies=["O2"]compile to identicalco_code.inspect.unwrapfirst, matchinggetsourcelines.Without it, a
functools.wrapswrapper below the decorator would handout the wrapper's source — and the validation would not notice, since a
wrapper's own source matches the wrapper's code object perfectly.
_arts_agendanow accepts an already-resolved(source, fn, ln)triple,so
in_parallelno longer looks the source up twice.Adds
tests/core/agenda/source_from_exec.py: exec-string agendas,callbacks and
in_parallel; indented (nested) functions; decoy rejectionfor an unrelated file (same name, and separately same lines with different
literals); fast/slow path agreement for decorated and wraps-decorated
functions in readable files.
Advantages
that breaks agenda creation in doc/example builds.
function, an
OSErrornaming the qualname is raised (fromthe originalinspect error) rather than silently parsing an unrelated same-named
function that happens to sit at that line of some candidate file.
inspectwalks per decoration than before (one lookup instead ofthree), and no behavioral change for normal file-based functions: the
fast path returns exactly what the old code returned.
Disadvantages / limitations
only accepted after re-compilation matching; but for functions with
free variables the bytecode comparison is skipped (free variables
resolve as globals in the stand-alone block), so for closures the
guarantee reduces to names/args/locals/consts — a contrived unrelated
closure at the same line could still be accepted. Documented in
_block_compiles_to.code objects (lambdas/inner defs compare by identity across
compilations); those functions keep a weaker guarantee. Documented.
block if the executed string has the same line layout as the candidate
file. Misaligned sources fail loudly (validation rejects), not silently.
sys.argv[0]is consulted as a candidate, which is only meaningful whilethe code is being executed by such a tool; the validation bounds the risk
(see above).
_source_and_location).Performance
Measured microbenchmarks (µs, best-of-3 loops, two processes each, Python
3.14.7, HEAD baseline vs this branch):
@arts_agendaapplied (full parse)_arts_agendainternals@callback_operatorappliedgetfile)OSError)Net: normal (file-based) decoration is neutral-to-slightly-faster; the only
measurable regression is ~0.3 µs on
callback_operator. The ~1.4 msfallback cost applies only where the old code raised outright, so it is a
new capability's cost, not a fast-path tax. Full suite timing unchanged
(
tests.core: 237 s, 155/155 passed).