[shoumikhin]

Core ML lowering crashes, or gives wrong values, for several common patterns. This change fixes them in the Core ML partitioner and backend.

Problems and fixes

Lowering crashes, or the program fails to load:

• uint8 and int8 inputs. coremltools cannot take them as program inputs, so converting a uint8 camera frame to float fails with KeyError: torch.uint8 (issue CoreML partitioner errors out with 8-bit integer types #11686). Ops that read such an input now run on the CPU up to the conversion to float, and the ops after it still go to Core ML. So does an op that makes an int8 tensor used outside Core ML, which the Core ML runtime cannot return, and a partition left with only size inputs, or no inputs below iOS 18. With lower_full_graph=True, lowering fails and names the tensor.
• State. If a CPU op splits the users of a state buffer that Core ML takes over, lowering now fails, names the ops and where they run, and points to take_over_mutable_buffer=False, instead of leaving one side with a stale copy. State that only CPU ops read stays on the CPU, also when Core ML computes its new value.
• bool min and max. min, max, amin, amax, minimum or maximum over a bool tensor, such as the check torch.distributions.MultivariateNormal exports, crashes lowering. These ops now run on the portable CPU kernels.
• Kept ops rejected after decomposition. When a float32 matmul keeps aten.matmul whole, a bfloat16 linear decomposes into another matmul that also stays whole, on the CPU, and loading fails with "kernel 'aten::matmul.out' not found". The partitioner now decomposes a copy of the program the way lowering will and runs the same search there, so an op that ends up on the CPU is not kept whole (an op skipped by name still is). This dry run only happens with a uint8 or int8 tensor, or when an op Core ML rejects gets decomposed, and once per program. On a 48-block transformer it took 0.06 s when it did not need to decompose and 1.45 s when it did, against 11.5 s for the whole lowering.
• Fills. coremltools makes full without a dtype, zeros and new_zeros as float, ones_like as float from iOS 16, zeros_like with the dtype of its input, and full_like, new_full, fill and scalar_tensor with the type of their fill value. So torch.full((1, 16), 0) (int64 in torch) is made as float, and index math on it fails to lower, as in the SmolVLM vision embeddings, or loses integers above 2048 at float16 precision. The backend now converts these ops, and empty, new_empty and empty_like, with the dtype torch gives them, so they stay in Core ML. A fill value known only at run time keeps the coremltools conversion.
• RMSNorm without eps. torch.nn.RMSNorm (root mean square normalization) leaves out eps (the small number added before the square root) by default, and coremltools requires it: "got 3 input(s), expected [4]". The backend now passes the float32 machine epsilon, which is what PyTorch uses. A model whose tensors are float16 still fails inside coremltools, with or without eps.
• index_put with a whole slice. coremltools converts y[:, idx] = v only when idx has a single entry. With idx = torch.tensor([0, 2]), as in the X-VLA policy, lowering fails with "Input squeeze_0 has rank 1 != other inputs rank 0". accumulate=True, a bool mask, or, below iOS 18, a bool tensor next to a slice fail too, and so do index tensors that broadcast. These forms now run on the portable kernels, while forms coremltools handles, such as a single-entry index for a cache update, stay in Core ML. So a cache update at several positions, as in a prefill, now lowers, with the update on the CPU. A negative index next to a slice still stays in Core ML, since the partitioner cannot see index values: it fails, or, for an index computed at run time, can give wrong values. An index buffer that such an index_put shares with a Core ML op goes to Core ML as an input, since to_backend does not copy a constant that an op reads from a list.

Wrong values without any error:

• tanh GELU. gelu(approximate="tanh") ran as the exact GELU (Gaussian Error Linear Unit), because torch.export passes approximate as a keyword and coremltools reads it only from the positional arguments. The two differ by up to 4.7e-4, which in pi0 and pi0.5 was the whole float32 error of the policy. The backend now reads the keyword.
• An output that repeats an input or another output. coremltools turns ops such as clone into nothing, so a partition can return its own input, as in return x.clone(), x * 2, or one value twice. Next to other outputs, such an output came back as zeros or as the bytes of another output. The backend now gives each one a copy of its own.
• index_put with a 0-d index or several indices next to a slice. coremltools lays out the written part differently from torch, so values land in the wrong place unless they vary only in the dims after the last index. Those forms now run on the portable kernels.
• nn.MultiheadAttention with a bool mask. It turns the mask into a float one with torch.zeros_like(mask, dtype=...), which coremltools kept bool, so attention came out wrong. The fill fix above covers this too.

With lower_full_graph=True, an op the partitioner refuses now fails with the reason, such as "aten.index_put.default with these indices is not supported by CoreML. Set lower_full_graph=False in the CoreMLPartitioner to run it outside of CoreML.", instead of saying that Core ML does not support the op.

A uint8 buffer that the model casts still fails with the default settings. The docs now suggest storing it as float16 or int32, or setting take_over_constant_data=False, and say what that setting costs.

How to review

Start with coreml_partitioner.py: the uint8 and int8 search in _propose_partitions, the state check, and the dry run in _drop_ops_left_outside. The refusals are small predicates near the top of the file, one per problem. The fixes in torch_ops.py and coreml_preprocess.py do not depend on the partitioner changes.

Tests

With coremltools 9.0, the tests below cover these cases. On macOS 15 or later they also run the program with the Core ML runtime and compare it with eager. With the Core ML code of main, the fill, index_put, state and compiler tests fail (the nn.MultiheadAttention case only with the runtime). The cases:

• uint8 frames read by pad, reflect pad, max_pool2d, roll, zeros_like, masked_fill, a channel swap, max(dim=1) or split, an int8 frame read by a permute, and a roll with a dynamic height.
• uint8 tensors next to ops left out of Core ML, such as a skipped flip or a diagonal, int8 outputs, state split by a CPU read or update, CPU-only state, state read only on the CPU, bool min/amax/maximum, and a bfloat16 linear next to a float32 matmul.
• Every fill op above, with an int or float fill value or a dtype argument, a float16 fill, and nn.MultiheadAttention with a bool mask; RMSNorm without eps and tanh GELU, also checked in the converted program; a returned clone and a value returned twice.
• index_put with a two-entry index, broadcast index tensors, accumulate=True or a bool tensor written next to a slice, and 0-d indices whose values do not fit or would land in the wrong place, next to a 0-d index whose values vary only after it, which stays in Core ML. From iOS 18, a bool write that stays in Core ML shares its index buffer with a write on the CPU. With lower_full_graph=True, the error gives the reason.

test_coreml_partitioner.py, test_torch_ops.py and test_coreml_recipes.py pass with the Core ML runtime on macOS. For a transformer block and conv models, with and without skipped ops, the partitions match main.

LeRobot policies, measured on an earlier version of this change with wheels built from the 1.6 release branch: without it, ACT, Diffusion, VQ-BeT, Multi-Task DiT and pi0.5 (bfloat16) fail to lower with KeyError: torch.uint8. With it, ACT, Diffusion, VQ-BeT and Multi-Task DiT export, run without loading torch, and match eager within 4.2e-06 over 16 frames. pi0.5 exports and passes its one-chunk check, but runs very slowly, since its bfloat16 linears run on the portable CPU kernels.

Not tested: Core ML state at run time, and a model that builds torch.distributions.MultivariateNormal itself (the test copies its bool check).

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/24317

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 Cancelled Jobs, 1 Pending

As of commit 3efab3a with merge base 681e493 ():

CANCELLED JOBS - The following jobs were cancelled. Please retry:

• Build Presets / apple (ios-simulator) / build (gh)
• Build Presets / apple (ios) / build (gh)
• Build Presets / apple (llm) / build (gh)
• Build Presets / apple (profiling) / build (gh)
• Build Presets / apple (pybind) / build (gh)
• Test CoreML Backend / test-coreml / test-backend-macos (coreml, models) / macos-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[shoumikhin]

@claude /pr-review

[claude]

Claude finished @shoumikhin's task in 28m 3s —— View job

---

Summary: This fixes the reported uint8 camera-frame case, but some uint8/int8 inputs still fail and the new tests leave most of the claimed behaviour unchecked. A split/unbind of the frame followed by F.pad now fails at to_executorch(), reflect/replicate pad on the frame probably still can't lower, and the new eager-match check always compares all-zero tensors. Recommendation: Request Changes.

Detailed review

PR Review: #24317

Summary

The PR keeps ops that read a uint8/int8 tensor from outside their Core ML partition on the CPU.

• partition() re-runs CapabilityBasedPartitioner until no partition reads such a tensor.
• ops_to_not_decompose predicts the same set of ops on the ATen graph, so that those ops are kept whole only when they have an out variant.

The problems:

• The prediction misses cases that partition() catches.
• Decomposing a reader can still leave a partition Core ML rejects.
• The new tests leave most of the claimed behaviour unchecked.

Nothing was run for this review, because this job is not allowed to run python. The failure claims below come from reading the code. The two marked low confidence depend on coremltools behaviour that could not be checked.

Code Quality

• ops_to_not_decompose does not follow tensors through getitem. backends/apple/coreml/partition/coreml_partitioner.py:549-550, 564-567

  • Problem: the loop only visits nodes whose target is an OpOverload. operator.getitem nodes never join outside, so the prediction stops at any op with several outputs (split, split_with_sizes, unbind, chunk, topk, sort).
  • Trigger: r, g, b = frame_u8.unbind(-1); return F.pad(r, (1, 1)).float() / 255.
    • aten.pad.default is predicted to stay in Core ML, so it is kept whole.
    • On the edge graph, partition() moves it to the CPU.
    • to_executorch() then fails in ToOutVarPass with Missing out variants (exir/passes/__init__.py:461-462). The message says nothing about the uint8 input.
  • Why the existing check misses it: the check after lowering (exir/program/_program.py:1418-1436) calls ops_to_not_decompose again on the edge program. Its targets are EdgeOpOverload, which is not an OpOverload subclass (exir/dialects/edge/_ops.py:291), so the call returns an empty list.
  • Second trigger: a uint8 tensor made in one Core ML partition and read by another. This happens when partitions are split to avoid a cycle through a CPU op. _find_readers_of_unsupported_inputs flags the reader (arg not in partition.nodes), but the prediction does not.
  • Fix:
    • Propagate outside through getitem.
    • Add a split/unbind test.
    • Consider raising a clear error after the loop in partition() when a node it moved is kept whole but has no out variant.

  Fix this →

• Decomposing a uint8 reader can still leave a Core ML partition with no inputs. coreml_partitioner.py:570-573. Low confidence, not run.

  • Trigger: F.pad(frame_u8, (1, 1, 1, 1), mode="reflect"), or mode="replicate". Reflect-padding a camera frame is common.
    • aten.pad.default has no out variant, so the pad is rejected and decomposed.
    • test_uint8_made_inside_the_delegate_keeps_ops_whole shows the decomposition is an aten.index.Tensor gather, with indices built from arange.
    • The gather reads the frame, so it moves to the CPU.
    • The index computation reads no runtime input, and it cannot join the float partition after the gather without making a cycle.
  • Result: a Core ML partition with no inputs, the case the roll docstring (test file 566-568) describes. model_metadata_from_spec rejects such a partition when the deployment target is below iOS18 (backends/apple/coreml/compiler/coreml_preprocess.py:439-442).
  • Even at iOS18+: ops are rejected by target, not by node (line 580). So every float reflect pad elsewhere in the network also becomes a gather inside Core ML.
  • Possible fixes: stop the decomposition at aten.reflection_pad2d/aten.replication_pad2d, which have portable kernels (kernels/portable/functions.yaml:823, 838), or don't delegate partitions that read no runtime input. Either way, add reflect/replicate pad to the uint8 test.

• _has_out_variant checks for an ATen .out schema, not for an ExecuTorch kernel. coreml_partitioner.py:196-201

  • Problem: to_out_variant only looks up torch's JIT schemas (exir/operator/convert.py:229-231).
  • Effect: an op that Core ML supports and that has an autogen .out but no portable or optimized kernel is kept whole on the CPU. It passes to_executorch() and then fails at load_method. A possible case is rot90, which has no entry in kernels/portable/functions.yaml (whether coremltools supports it is unverified).
  • The comment at 561-563, the test docstring at 565-566 and the PR description all treat "has an out variant" and "has a CPU kernel" as the same thing.
  • Narrow exception handling: to_out_variant can also raise AssertionError (convert.py:213-226, 308-311), or AttributeError when _get_overload_schema returns None (convert.py:303-304). Only RuntimeError is caught, and this call sits outside the try/except at 552-560. Such an op stops lowering instead of being decomposed.

  Fix this →

• unsupported_input_readers is passed through a side channel. coreml_partitioner.py:226, 337, 475

  • Problem: partition() adds nodes to a public set on the support object between rounds. So is_node_supported answers differently for the same node depending on hidden state, and the instance in ops_to_not_decompose never sees the set.
  • Cost: each round re-runs CapabilityBasedPartitioner and coremltools' is_torch_fx_node_supported on every node, once per op along the uint8 chain.
  • Fix: pass the set in explicitly, for example as a constructor argument with a new support object each round, instead of changing it from outside the class.

• Only partition inputs are checked, not partition outputs of these dtypes. coreml_partitioner.py:174-193. Low confidence.

  • Trigger: the loop can create such an output itself. In torch.maximum(frame_u8, (x > 0).to(torch.uint8)), moving maximum to the CPU turns the uint8 operand made inside Core ML into a delegate output.
  • Runtime side:
    • get_data_type(ScalarType) has no Char (int8) case (backends/apple/coreml/runtime/delegate/coreml_backend_delegate.mm:130-151).
    • The MLMultiArray output mapping covers only Float16/32/64 and Int32 (ETCoreMLModelManager.mm:137-155).
  • How coremltools types such outputs could not be checked here.

Backward Compatibility

• "Models that already lowered keep the same Core ML program" is not true in general. coreml_partitioner.py:568-569
  • Problem: the prediction runs on the ATen graph and adds every node unsupported there to outside, even when the pieces it decomposes into are delegated on the edge graph.
  • Example:
    • _HIGH_RANK_LAYOUT_TARGETS (70-77) lists only edge ops, so an ATen view/permute above rank 5 is unsupported at 110-115 but delegated after decomposition.
    • Model: (x * 255).to(torch.uint8) → rank-6 reshape/permute/reshape → F.pad → .float().
    • Before this PR it lowered entirely into Core ML.
    • Now the pad reads an "outside" uint8 tensor, so aten.pad.default is rejected and every pad in the model is decomposed.
  • The trigger is contrived. At minimum, narrow the claim in the PR description.

Testing

• test_uint8_camera_frame_is_read_outside_the_delegate (test_coreml_partitioner.py:533, 538, 556):
  • With randint(0, 100), x / 255 - 0.5 < -0.11, so relu returns all zeros. The allclose passes even with a wrong permute, conversion or delegate value.
  • A wider range alone is not enough for int8: 127/255 − 0.5 < 0 still gives all zeros, so the model has to change too.
• Same test, 558-561: the lower_full_graph=True check runs after the loop, with the ep from the last pass. Only int8 is covered, never uint8 from CoreML partitioner errors out with 8-bit integer types #11686. Move the check into the subTest.
• test_uint8_input_read_by_ops_with_and_without_cpu_kernels (563-596):
  • Off macOS it only checks that lowering and to_executorch() don't raise.
  • Nothing asserts that a delegate exists, that aten.roll.default stays whole, or that pad and max_pool2d were decomposed.
  • The neighbouring placement tests assert on op_names on every platform (e.g. 415-422, 512-518).
• _placeholders_kept_inside (coreml_partitioner.py:503-516) has no test. Nothing reads a uint8/int8 parameter, buffer, lifted constant or mutable buffer inside Core ML with take_over_constant_data / take_over_mutable_buffer on and off.

Fix the tests →

Recommendation

Request Changes

• The getitem gap is a crash in the new logic, and the fix is small.
• The main test's end-to-end check cannot detect wrong output.
• The reflect/replicate-pad case needs either a fix or a documented limitation.

---

• Gather context (diff, commit, repo conventions)
• Run independent reviews (3 reviewers)
• Consolidate findings
• Fact-check findings against the cited code
• Post review

[shoumikhin]

@metascroy @cymbalrush @YifanShenSZ could you take a look? This changes the Core ML backend.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[shoumikhin]

@claude /pr-review

[claude]

Claude finished @shoumikhin's task in 42m 31s —— View job

---

This keeps ops that read uint8/int8 tensors on the CPU as intended, but at the default deployment target it can also move a state update to the CPU while a Core ML delegate keeps reading a frozen copy of that state, so the recommendation is Request Changes.

Detailed review

PR Review: #24317

Summary

The PR makes the Core ML partitioner run on the CPU two kinds of ops: those that read a uint8/int8 tensor from outside their partition, and those in any partition this leaves with no tensor input. It also makes ops_to_not_decompose decompose the non-core ops among them. The main problem is that at the default deployment target, a model with a uint8/int8 input and state now lowers without error but then computes with stale state. The other findings are smaller gaps in what the new partition boundaries allow, in the messages, and in the tests. Nothing was run, because this job can't run python or git fetch or read the coremltools/torch sources. The runtime effects below are traced from the ExecuTorch code.

Code Quality

• Stale state at the default deployment target (major). In backends/apple/coreml/partition/coreml_partitioner.py, line 222 doesn't count kept-inside placeholders as inputs. So once the loop leaves anything out (line 251), a partition whose only outside input is a taken-over mutable buffer counts as having no inputs.

  • Without an iOS 18 minimum_deployment_target (_allows_partitions_without_inputs, 554-559), line 225 moves that partition to the CPU, while the buffer's other readers stay delegated.
  • tag_mutated_buffer ignores the untagged CPU users and gives the buffer to the reading delegate (exir/backend/utils.py:416-430). _maybe_duplicate_constant_nodes then gives the CPU ops a copy (utils.py:225-237).
  • The CPU updates its copy, and from the second call on the delegate keeps reading the initial value.
  • The model in test_uint8_input_keeps_state_in_the_delegate_from_ios18 (test file 669-679) does exactly this at the default target, which is the stale copy its docstring describes. The test only covers iOS 18, only checks op placement, and never calls to_executorch().
  • Suggested fix: don't move a partition that writes taken-over state. It is stateful, so it already needs iOS 18 (log at 536-542), and the no-input check reads the converted model's specificationVersion (backends/apple/coreml/compiler/coreml_preprocess.py:439-442), which such a model should already meet. Wherever the users of a taken-over buffer would still be split between Core ML and the CPU, raise an error that points to take_over_mutable_buffer=False or an iOS 18 target.
  • Low confidence: in the same test at iOS 18, the reader and the writer are separate delegates, and the same duplication looks like it gives the reader a frozen copy there too. Running forward twice on macOS and comparing with eager would show both cases.

  Fix this →

• Values other than input tensors can cross the new boundaries unchecked (low confidence). The loop only checks the dtype of tensors a partition reads.

  • With a dynamic dim on the uint8 input, the sym_size that reads that input ends up on the CPU. A delegated op whose size comes from it, such as a reshape computed from image.shape[2], then reads a SymInt from outside. Its partition also reads a tensor, so line 225 keeps it. The delegate then gets the symbolic input that the docstring at 213 and the TODO at 357-372 say Core ML can't take.
  • A uint8/int8 tensor made inside a partition and read by a left-out op becomes a delegate output, for example (x > 0.5).to(torch.uint8) * image with a uint8 image. I couldn't check what coremltools does with that output dtype. The runtime's dtype map has no int8 (Char) case (backends/apple/coreml/runtime/delegate/coreml_backend_delegate.mm:130-151).

• Skipped ops have different names in the two graphs that compute left_out (minor; the mismatch predates this PR). skip_ops_for_coreml_delegation is matched against node.target.__name__ (line 383).

  • That name is mm.default for the ATen op ops_to_not_decompose sees, but aten.mm.default for the edge op partition() sees (exir/dialects/edge/_ops.py:303). The tests pass the edge form (test file 73, 100, 433).
  • So a skipped non-core op is still preserved, then lands whole on the CPU, where it may have no kernel. Separately, the new ATen-side left_out can disagree with the one partition() computes.
  • The post-lowering check (exir/program/_program.py:1418-1436) catches neither case: ops_to_not_decompose only looks at torch._ops.OpOverload targets (621-622), so it returns nothing for edge programs.

  Fix this →

• The new messages are hard to act on (minor).

  • The log at 523-526 joins str(node.target) once per node, and for edge ops that is the whole <EdgeOpOverload: …>: schema = … repr (exir/dialects/edge/_ops.py:334-339).
  • The lower_full_graph=True error (517-522) doesn't name the input or the dtype.
  • The log goes through CoreMLPartitioner.log_once (549-552), which uses root logging.info rather than the module logger used at 285-288, so ET_COREML_LOG_LEVEL doesn't control it.

  Fix this →

Testing

• test_uint8_input_with_dynamic_shape (633-660) and test_uint8_buffer_not_taken_over_is_read_outside_the_delegate (697-723) only assert under if _TEST_RUNTIME:, which is false on Linux (line 45). So the Linux run checks only that lowering doesn't raise, not the op placement their docstrings describe. Fix this →
• test_uint8_input_read_by_ops_with_and_without_cpu_kernels only lowers with the default CoreMLPartitioner() (618). At iOS 18, _allows_partitions_without_inputs() is true, so the gather index math stays in Core ML as a delegate with no inputs. The aten.arange.start_step expectations (594, 596) therefore hold only at the default target, and the iOS 18 path is never lowered or run. Fix this →

Performance

• Extra partitioning on every export (minor). ops_to_not_decompose (613-618) now runs _propose_partitions even when the graph has no tensor of a dtype Core ML can't take.

  • Each call gets an extra CapabilityBasedPartitioner pass and a second support check per node (624).
  • to_edge_transform_and_lower calls it four times per program (exir/program/_program.py:1193, 1196→1140, 1422, 1426→1140). So a float model pays for two extra full passes over the ATen graph that yield an empty left_out.
  • The two post-lowering calls throw the result away, since edge targets fail the check at 621-622.
  • Skip the call when no node satisfies _cannot_be_coreml_input.

  Fix this →

Recommendation

Request Changes

The stale-state case gives wrong results with no error. It happens with a uint8/int8 input, state and the default deployment target, a combination that crashed with a KeyError before this PR and now lowers. The new tests would not catch it. The remaining items are minor or low confidence.

• Gather context
• Run independent reviews
• Consolidate findings
• Fact-check
• Post review

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

[shoumikhin]

@claude /pr-review

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

[shoumikhin]

@claude /pr-review

[claude]

Claude finished @shoumikhin's task in 50m 52s —— View job

---

Needs discussion: this PR works around an exir bug (shared constants read from a list) inside the Core ML partitioner, its int8-output rule may be covering a missing case in this repo's Core ML delegate, and it has a wrong comment, missing upstream links and two fixes that only macOS tests check.

Detailed review

PR Review: #24317

Summary

The PR runs ops on the CPU when they read uint8 or int8 inputs, or make int8 outputs that are used outside Core ML. It refuses the index_put forms and the bool min/max that coremltools gets wrong. It also fixes fill dtypes, tanh GELU, RMSNorm without eps, and outputs that repeat a value or an input. Two design questions below need the author's input; the other problems are small. The checkout is shallow, so I reviewed git diff 681e493 HEAD: 681e493 is the parent of the first PR commit, and the diff stats match the PR. I ran nothing. The torch and coremltools sources weren't available, so anything said here about how coremltools behaves comes from the PR's own comments and tests.

Code Quality

• The shared-constant fix sits in the Core ML partitioner instead of exir.

  • _untag_constants_shared_through_lists (coreml_partitioner.py:413-437, called at 765) works around duplicate_constant_node. That function only swaps the copy into top-level args and kwargs (exir/backend/canonical_partitioners/duplicate_constant_node_pass.py:99-118), so any backend that calls tag_constant_data can hit the same failure.
  • Commit 6ba764d fixed it in exir, and 1d19533 reverted that: a CPU op that reads a delegate's mutated buffer from a list would then read a stale copy. But tag_constant_data never tags mutated buffers (exir/backend/utils.py:367-380). So an exir fix that skips mutated buffers avoids the stale copy and still covers every case this helper handles.
  • The helper also causes a problem of its own. It runs after _propose_partitions has already counted the constant as staying inside Core ML (693-704). An untagged uint8 or int8 constant then becomes exactly the kind of Core ML input this PR moves to the CPU everywhere else.
  • If the exir fix stays out of this PR, add a TODO here that links a tracking issue.

  Fix this →

• The int8-output rule may be working around this repo's delegate rather than Core ML (low confidence).

  • _find_makers_of_int8_outputs (coreml_partitioner.py:336-349) moves an op off Core ML because "the Core ML runtime cannot return int8".
  • The refusal I can see is in ExecuTorch's own delegate code. get_data_type (runtime/delegate/coreml_backend_delegate.mm:130-151) has no ScalarType::Char case, so get_multi_array rejects every int8 tensor (164-168). Meanwhile MultiArray already handles DataType::Char (multiarray.mm:69, 388, 440, 572).
  • If that missing case is the only gap, a one-line runtime change would let ops whose int8 result is used outside Core ML stay in Core ML. I couldn't check whether Core ML itself refuses int8 outputs.

  Fix this →

• The new coremltools workarounds have no upstream links.

  • This file links each refusal to its upstream issue "so future readers can tell when an entry is safe to drop" (coreml_partitioner.py:50-52; also 497 and 511-514). The new _is_min_max_of_bool (195-205) and _is_index_put_coremltools_cannot_convert (214-275) only say "coremltools 9.0".
  • Likewise, torch_ops.py:7-9 asks for a coremltools link for every op registered there, but the fill overrides' comment (91-96) only links the zeros_like fix.

  Fix this →

• The comment on the empty aliases is wrong for two of the three ops.

  • torch_ops.py:121-123 says lowering turns new_empty and empty_like into _empty_dim_order, so their aliases only matter when dim order ops are skipped.
  • But ops_to_not_decompose keeps every op coremltools supports whole (coreml_partitioner.py:897-906), and that includes these two. Because a kept empty_like is a preserved op, _generate_edge_program skips ReplaceEmptyLikePass (exir/program/_program.py:846-852). And DimOrderOpsMap only rewrites empty.memory_format (exir/passes/dim_order_ops_registry.py:81-85).
  • So the new_empty and empty_like aliases are used on the default path. Only empty becomes _empty_dim_order.

  Fix this →

Testing

• Two fixes are only checked on macOS. Linux CI runs these tests (pytest.ini:89, and pytest -n auto in .ci/scripts/unittest-linux-cmake.sh). But outputs are only compared where _TEST_RUNTIME or IS_VALID_TEST_RUNTIME is true, which means macOS.

  • In test_ops_coremltools_gets_wrong (test_coreml_partitioner.py:706-726), the Attention case only checks off macOS that a delegate exists and that no zeros_like runs on the CPU. Without the override, the mask would probably stay bool and still lower, so reverting the fix would leave Linux CI green. The milinternal dtype check at 727-729 covers only Counts.
  • test__clone_dim_order_contiguous (test_torch_ops.py:322-328) applies executorch::copy_reused_outputs by hand. The append_pass at coreml_preprocess.py:598, which is what users actually get, is only exercised by the macOS runtime comparison.

  Fix this →

Recommendation

Needs Discussion

Nothing here breaks the cases the tests cover. Two decisions need settling first:

• Should the shared-constant fix go into exir, limited to constants that aren't mutated buffers?
• Is the int8-output rule needed at all, or does the delegate just lack a Char case?

The comment fix, the upstream links and the two Linux-side test checks are small.

• Read the PR context and CLAUDE.md
• Get the diff
• Run three independent reviewers
• Consolidate findings (27 candidates → 21)
• Fact-check each finding against the code (5 of 21 kept)
• Post review

[shoumikhin]

I split two parts out of this PR so each can be reviewed on its own:

• Fix wrong dtypes and values in Core ML conversion #24424 fixes Core ML conversions that give wrong dtypes or values: fills, tanh GELU, RMSNorm without eps, repeated outputs, and a negative index next to a slice.
• Refuse ops coremltools cannot convert in the Core ML partitioner, and say why #24425 makes the partitioner refuse ops coremltools cannot convert, such as bool min and max and some index_put forms, and say why.

The rest stays here. It depends on #24404, so I will update this PR once that lands.

[metascroy]

Question about the uint8 part, for the camera-frame case (#11686 and the LeRobot policies): would exporting the model with a float input, and converting the uint8 frame with the image processor in extension/image, solve the problem?

ImageProcessor::process() takes the uint8 frame (RGB, BGRA or NV12, with resize, crop and orientation) and returns a normalized float tensor [1, 3, H, W], with per-channel scale, mean and std in ImageProcessorConfig::normalization. Then:

• No uint8 tensor reaches Core ML, so the whole graph is delegated. With this PR, the ops that touch the frame (flip, permute, pad, the cast) run on the portable CPU kernels and then hand off to Core ML.
• If you'd rather keep the / 255 and normalization in the graph, fp16 holds every integer up to 2048 exactly, so pixel values 0–255 lose nothing.

One caveat: process() currently returns a float32 tensor, so an fp16 model input would need a cast, or fp16 output support in the processor.

If that covers your use cases, most of the uint8 partitioning and the decomposition dry run could become a clear partitioner error for uint8/int8 inputs that points to the image processor. The general fixes here (the state check and the int8-output rule) could then land as their own change. Are there cases it doesn't cover, for example a uint8 input that isn't an image, or a policy that has to keep its eager uint8 signature?

[shoumikhin]

Thanks. That is a good fit for apps that read camera frames in C++, and it keeps the whole graph in Core ML. It does not cover everything this PR handles, though:

1. Programs whose exported signature takes uint8. For example, a policy whose forward takes the raw frame and converts it itself, so that the program matches eager on the same inputs. A program run from Python cannot use extension/image either.
2. uint8 and int8 tensors that are not program inputs. A uint8 tensor made in Core ML can come back to Core ML through a CPU op, such as a skipped op or one Core ML rejects, and an int8 tensor can cross from one Core ML partition to another. On main these fail with KeyError: torch.uint8 or KeyError: torch.int8, even though the model has no uint8 or int8 input, so an input check would not help. test_ops_after_a_cpu_op_on_a_tensor_made_inside covers the uint8 case, and the next push adds a test for the int8 one.
3. The dry run is shared. The int8-output rule and the symbolic-value rule also move ops to the CPU. The dry run is what keeps those moves from leaving an op such as pad whole on the CPU, where it has no portable kernel. So it stays even without the uint8 input part.

So I would keep the general handling, and the next push adds a line to the Core ML docs that points camera apps to ImageProcessor and a float input.

[shoumikhin]

On your point in #24404: the next push drops _check_state_is_used_in_one_place. With #24404, state used in more than one place stays in the program, and exir warns when that happens, so the partitioner no longer raises.