Skip to content

Add Tzafon Northstar computer use template#134

Merged
dprevoznik merged 3 commits intokernel:mainfrom
mttias:tzafon-computer-use
Apr 10, 2026
Merged

Add Tzafon Northstar computer use template#134
dprevoznik merged 3 commits intokernel:mainfrom
mttias:tzafon-computer-use

Conversation

@mttias
Copy link
Copy Markdown
Contributor

@mttias mttias commented Mar 13, 2026

Adds TypeScript and Python templates for Tzafon's Northstar CUA Fast model using Kernel Computer Controls API. Tested end-to-end on Kernel.

Attaching video of the following task: "go to magnitasks.com and click on "Tasks" in the sidebar. Then, drag the task cards from "To Do" to "Done" column without clicking into them.".
https://github.com/user-attachments/assets/b42f816f-c0b8-4b32-a6c0-0fbf2c8d8f15


Note

Low Risk
Low risk because changes are additive and mostly isolated to new template scaffolding, with only a small update to template selection/sorting and sample commands.

Overview
Adds a new tzafon template option to pkg/create/templates.go, including display metadata, sort priority, and deploy/invoke sample commands for both TypeScript and Python.

Introduces new pkg/templates/{typescript,python}/tzafon reference apps that run a Northstar CUA loop via Tzafon Lightcone, including browser-session management (optional replay recording), computer-control tool implementations (click/type/key/scroll/drag), and corresponding .env.example/README/dependency configs.

Reviewed by Cursor Bugbot for commit 678ebf6. Bugbot is set up for automated code reviews on this repo. Configure here.

@mttias mttias force-pushed the tzafon-computer-use branch 2 times, most recently from f599a30 to 00b5338 Compare March 13, 2026 21:11
@dprevoznik dprevoznik requested review from Sayan- and dprevoznik March 17, 2026 04:39
@mttias mttias force-pushed the tzafon-computer-use branch from 7ae85f2 to 1aeabe5 Compare March 17, 2026 08:54
Copy link
Copy Markdown
Contributor

@Sayan- Sayan- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of small q's. overall looks good!

@mttias mttias force-pushed the tzafon-computer-use branch 2 times, most recently from 224c5d9 to 6770380 Compare April 10, 2026 00:38
@mttias mttias requested review from Sayan- and dprevoznik April 10, 2026 00:41
@mttias
Copy link
Copy Markdown
Contributor Author

mttias commented Apr 10, 2026

Made a recent update that fixed some of the minor issues such as drag n drop.

@mttias mttias force-pushed the tzafon-computer-use branch from 6770380 to 08870c0 Compare April 10, 2026 00:51
@dprevoznik
Copy link
Copy Markdown
Contributor

Suggestion: Add a system prompt via instructions

The Lightcone Responses API supports an instructions field that acts as a system prompt (docs). The tzafon template currently doesn't use it — the task is passed directly as the first user message with no system-level guidance.

The other templates (Anthropic, Gemini) all include system prompts with environment context and behavioral hints. For reference, here's what the Anthropic template uses:

<SYSTEM_CAPABILITY>
* You are utilising an Ubuntu virtual machine with internet access.
* When you connect to the display, CHROMIUM IS ALREADY OPEN. The url bar is not visible but it is there.
* If you need to navigate to a new page, use ctrl+l to focus the url bar and then enter the url.
* You won't be able to see the url bar from the screenshot but ctrl-l still works.
* As the initial step click on the search bar.
* When viewing a page it can be helpful to zoom out so that you can see everything on the page.
* Either that, or make sure you scroll down to see everything before deciding something isn't available.
* Scroll action: scroll_amount and the tool result are in wheel units (not pixels).
* When using your computer function calls, they take a while to run and send back to you.
* Where possible/feasible, try to chain multiple of these calls all into one function calls request.
* The current date is <date>.
* After each step, take a screenshot and carefully evaluate if you have achieved the right outcome.
* Explicitly show your thinking: "I have evaluated step X..." If not correct, try again.
* Only when you confirm a step was executed correctly should you move on to the next one.
</SYSTEM_CAPABILITY>

<IMPORTANT>
* When using Chromium, if a startup wizard appears, IGNORE IT. Do not even click "skip this step".
* Instead, click on the search bar on the center of the screen where it says "Search or enter address", and enter the appropriate search term or URL there.
</IMPORTANT>

Proposed addition — add an instructions field to the responses.create() call in both TS and Python loops. Things to include:

  • Environment context: Chromium is already open, url bar behavior, ctrl+l navigation
  • Scrolling guidance: Prefer key("pagedown") / key("pageup") for full-page scrolling (the key map already supports these) — the scroll tool is mouse-wheel-style and can be unreliable for large page traversals
  • Date: Current date for time-sensitive tasks
  • Step verification: Evaluate each step before moving on
  • Chromium startup wizard: Ignore it if it appears

Some of the Anthropic hints may not apply directly (e.g., scroll_amount units, chaining calls) since Northstar uses explicit function tools rather than Anthropic's computer_use tool type. We should adapt rather than copy verbatim.

@dprevoznik dprevoznik requested review from Sayan- and dprevoznik and removed request for Sayan- April 10, 2026 15:34
Add new CLI templates for Tzafon's Northstar CUA Fast model, enabling
users to scaffold browser automation projects using Kernel's infrastructure.

New templates:
- TypeScript: `kernel create --template ts-tzafon-cua`
- Python: `kernel create --template python-tzafon-cua`

Both templates include:
- Agentic sampling loop using Tzafon's Lightcone SDK responses API
- Computer tool mapping Northstar actions (click, type, scroll, drag,
  key, navigate, wait) to Kernel's Computer Controls API
- Session management with optional replay recording
- 1280x800 default viewport

Files changed:
- pkg/templates/typescript/tzafon-computer-use/ - TypeScript template
- pkg/templates/python/tzafon-computer-use/ - Python template
- pkg/create/templates.go - Template registration
@mttias mttias force-pushed the tzafon-computer-use branch from 08870c0 to 4db9163 Compare April 10, 2026 16:04
Copy link
Copy Markdown
Contributor

@Sayan- Sayan- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution!

@socket-security
Copy link
Copy Markdown

socket-security bot commented Apr 10, 2026

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​tzafon/​lightcone@​0.7.18010010094100
Addednpm/​@​types/​node@​22.19.151001008196100
Addedpypi/​tzafon@​2.35.099100100100100

View full report

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit ae96cc7. Configure here.

Copy link
Copy Markdown
Contributor

@dprevoznik dprevoznik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@dprevoznik dprevoznik merged commit 82d943f into kernel:main Apr 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants