Repository navigation
What is the recommended way to add image to LLM chat history after calling a tool? #796
Description
If a tool calling returns an image instead of texts, how to add it properly to the chat history? My workaround is
def load_screenshot(tool_context: ToolContext):
""" Load the system screenshot from clipboard.
"""
if image_bytes := get_clipboard_image():
tool_context._invocation_context.user_content.parts.append(
types.Part.from_bytes(
data=image_bytes,
mime_type="image/png"
)
)
return "Successfully load screenshot from clipboard."
else:
return "Failed to load screenshot from clipboard."Considering _invocation_context is an internal variable, I guess it is not the traditional way to do this. Then what is the recommended way for this purpose?
Activity