Skip to content

[SPFx Copilot Apps] blob_resource content is silently unavailable to Code Interpreter for files larger than 1 KB #11044

Description

@tiboldtomi

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

💥 SharePoint Framework

Developer environment

Windows

What browser(s) / client(s) have you tested

  • 💥 Internet Explorer
  • 💥 Microsoft Edge
  • 💥 Google Chrome
  • 💥 FireFox
  • 💥 Safari
  • mobile (iOS/iPadOS)
  • mobile (Android)
  • not applicable
  • other (enter in the "Additional environment details" area below)

Additional environment details

  • SharePoint Framework: 1.24.0-beta.3
  • Package: @microsoft/sp-copilot-component@1.24.0-beta.3
  • Node.js: 22.x
  • Component type: SPFx Copilot Component
  • Agent type: Microsoft 365 Copilot declarative agent
  • Agent capabilities:
    • CodeInterpreter
    • OneDriveAndSharePoint
  • Host: Microsoft 365 Copilot
  • Authentication: signed-in organizational user
  • Browser: Chromium-based browser
  • Microsoft 365 environment: commercial tenant

Describe the bug / error

When an SPFx Copilot Component adds binary content to the model context by using createCopilotBlobResourceContent or createCopilotBlobResourceContentFromBlobAsync, the binary resource is available to Code Interpreter only for very small payloads.

In our reproducible size tests, a deterministic 1 KB binary resource was successfully made available to Code Interpreter. Code Interpreter could read the file, report its exact byte size, calculate its SHA-256 hash, and verify its beginning and ending ASCII markers.

Starting with a 100 KB payload, only the structured metadata remained visible to the agent. The corresponding binary resource was not available as a readable file in the Code Interpreter environment.

Observed results:

Payload Structured metadata visible Binary resource readable
1 KB Yes Yes
100 KB Yes No
1 MB Yes No
5 MB Yes No
10 MB Yes No
20 MB Yes No

The copilotBridge.updateModelContextAsync() call completes successfully. No client-side exception, payload-size error, warning, or rejection indicates that the binary content was dropped.

A representative agent response for the failing cases was:

The structured metadata is visible, but the associated binary blob resource is not available in the Code Interpreter environment. Therefore, its actual size, SHA-256 hash, and markers cannot be verified.

We also reproduced the problem with real XLSX workbooks. The agent could see the filenames, sizes, and structured metadata, but the actual workbooks were not available as local or readable binary files in Code Interpreter.

We tested both:

  • createCopilotBlobResourceContent
  • createCopilotBlobResourceContentFromBlobAsync

Using createCopilotBlobResourceContentFromBlobAsync instead of the byte-based helper did not make the real XLSX files available to Code Interpreter.

This blocks document-processing scenarios where an SPFx Copilot App needs to provide an existing SharePoint or OneDrive document to Code Interpreter. Our real use case involves XLSX files of approximately 20 MB that contain worksheets, formatting, images, shapes, drawing relationships, and other OOXML content.

This is different from manually attaching a file in Microsoft 365 Copilot. Manually attached files can be processed by Code Interpreter, while files supplied by the SPFx Copilot Component as blob_resource content are not available in the same way.

Could you please clarify:

  1. What is the maximum supported size for createCopilotBlobResourceContent?
  2. What is the maximum supported size for createCopilotBlobResourceContentFromBlobAsync?
  3. Is a blob_resource expected to be materialized as a readable file in the Code Interpreter environment?
  4. Is blob_resource intended to support Office files such as XLSX and DOCX?
  5. Are there limits for individual resources, total context size, MIME types, or the number of resources?
  6. Is the difference between a manually attached Copilot file and an SPFx blob_resource intentional?
  7. Is there a supported SPFx API that provides the same file attachment behavior as manually attaching a file in Microsoft 365 Copilot?
  8. If the payload exceeds a supported limit, should updateModelContextAsync() return an explicit error instead of silently accepting the update?
  9. Is the observed 1 KB versus 100 KB behavior a known preview limitation or a product defect?
  10. Where are the supported limits and intended usage of these APIs documented?

Steps to reproduce

  1. Create an SPFx 1.24.0-beta.3 Copilot Component.

  2. Create deterministic binary content with:

    • a known filename;
    • an exact byte length;
    • a known SHA-256 hash;
    • an ASCII marker at the beginning;
    • an ASCII marker at the end.
  3. Convert the content into a Copilot blob resource by using createCopilotBlobResourceContent.

  4. Add the blob resource to the model context with copilotBridge.updateModelContextAsync().

  5. Include the expected filename, size, SHA-256 hash, and markers in structuredContent.

  6. Send a follow-up message asking Code Interpreter to:

    • locate the supplied binary resource;
    • read its bytes;
    • report its actual byte size;
    • calculate its SHA-256 hash;
    • report its beginning and ending markers.
  7. Run the test with a 1 KB payload.

  8. Verify that Code Interpreter can read the resource and returns the correct size, hash, and markers.

  9. Repeat the same test with:

    • 100 KB;
    • 1 MB;
    • 5 MB;
    • 10 MB;
    • 20 MB.
  10. Observe that for the larger payloads:

    • updateModelContextAsync() still completes successfully;
    • the structured metadata is visible to the agent;
    • the actual binary resource is not available to Code Interpreter.
  11. Repeat the test using createCopilotBlobResourceContentFromBlobAsync.

  12. Repeat the test with a real XLSX workbook.

  13. Observe that changing the helper or MIME type does not make the larger binary file available to Code Interpreter.

Expected behavior

The complete binary resource should be available to Code Interpreter as a readable file after updateModelContextAsync() completes successfully.

If the payload size, MIME type, or intended use is unsupported, the bridge should reject the operation with a clear and documented error instead of silently accepting the context update while dropping the binary content.

At minimum, the documentation should clearly specify:

  • the maximum supported size of an individual blob resource;
  • the maximum total size of a model-context update;
  • whether blob resources are materialized as files for Code Interpreter;
  • whether Office files such as XLSX and DOCX are supported;
  • whether multiple binary resources are supported;
  • the behavioral difference between an SPFx blob resource and a manually attached Copilot file;
  • the supported SPFx mechanism for programmatically attaching a file to Code Interpreter.

A successful API call should not result in structured metadata being delivered while the associated binary content is silently unavailable.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    AI-generated responsetype:bug-suspectedSuspected bug (not working as designed/expected). See “type:bug-confirmed” for confirmed bugs.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions