UI-TARS Desktop: a visual desktop agent inside a multimodal stack

Everything worth knowing about bytedance/UI-TARS-desktop: a ByteDance monorepo that bundles the UI-TARS Desktop app and Agent TARS to operate graphical interfaces, browsers, and tools through multimodal models.

What UI-TARS Desktop is

UI-TARS Desktop started life as a native graphical-interface agent application for the local computer. The current README places it inside TARS, a multimodal agent stack that ships two products: UI-TARS Desktop and Agent TARS.

The desktop app uses UI-TARS models and the Seed-1.5-VL/1.6 series to interpret screenshots and execute mouse and keyboard actions. It declares support for Windows, macOS, and browser, visual recognition, real-time status feedback, and local processing. It should not be confused with a simple sequence automator: the flow starts from a natural-language instruction and a vision-language model.

Illustration of an AI agent interpreting a computer desktop through a neural network that analyzes on-screen icons, windows, and text.

The same repository incorporates Agent TARS, the terminal- and web-oriented evolution: a command-line interface and a web interface for combining vision, a graphical-interface agent, browser, and MCP servers. Its README illustrates tasks such as searching for flights, booking a hotel, or producing a chart using additional MCP tools.

The origin: from research model to an operating application

The repository was created by the ByteDance organization on January 19, 2025, according to the GitHub API. The timeline preserved in the README marks the release context:

On January 23, 2025, the cloud deployment guide for ModelScope was updated.
On April 17, 2025, UI-TARS Desktop v0.1.0 was announced, with a redesigned agent interface, browser operation, and support for UI-TARS-1.5.
On June 12, 2025, UI-TARS Desktop v0.2.0 was announced, adding free remote computer and browser operators, presented as usable without configuration.
On June 25, 2025, ByteDance announced the Agent TARS beta and its command-line interface.
On November 5, 2025, the README announced Agent TARS CLI v0.3.0.

Timeline showing the v0.1.0, v0.2.0, and Agent TARS Beta milestones connected by glowing nodes.

The project’s narrative shows a transition: the local application centered on UI-TARS came to share space with a more general agent stack. There is a practical tension between controlling an interface solely through pixels and using structured knowledge of the browser or external tools. Agent TARS answers it with a hybrid browser — visual agent, DOM, or combined strategy — and with MCP; UI-TARS Desktop keeps the native visual-operator use case.

The academic paper cited by the repository itself, “UI-TARS: Pioneering Automated GUI Interaction with Native Agents,” attributes the work to Yujia Qin, Yining Ye, Junjie Fang, Haoming Wang, Shihao Liang, Shizuo Tian, Junda Zhang, Jiahao Li, Yunxin Li, Shijue Huang, and other authors. This attribution identifies the model’s origin; it does not prove that every author maintains the current application.

Philosophy and principles

The documentation lets us identify four design principles, without turning them into promises of absolute reliability:

The interface as the agent’s work surface. UI-TARS Desktop proposes that a vision-language model see the screen and act on controls, instead of requiring a specific integration for each application.
Natural control, but observable. The feature list includes natural-language instructions, screenshots, visual recognition, and real-time status; Agent TARS adds an event sequence to observe and debug the flow.
A choice of browser strategy. Agent TARS does not limit automation to visual reasoning: it documents graphical-interface, DOM, and hybrid modes.

Hybrid browser split into two layers: a visual pixel-grid view and the underlying DOM tree, with a robotic cursor over a flight-booking button.

Extensibility toward real tools. Agent TARS’s core is described as built on MCP and able to mount MCP servers. This extends the agent beyond clicking buttons, but it also introduces the responsibility of reviewing the servers and credentials that get connected.

How it works

UI-TARS Desktop

The desktop path combines an instruction, an interface screenshot, and a vision-language model. From that perception, the operator executes precise mouse and keyboard actions. The official examples include changing VS Code’s auto-save setting and checking a GitHub issue from a browser.

The contribution documentation shows that the code is a monorepo. Its application lives in apps/ui-tars; the packages include browser agent infrastructure, browser-use, MCP client and servers, search and logging, plus UI-TARS packages for the action parser, command-line interface, Electron IPC, operators, SDK, and visualizer.

Agent TARS

The minimum documented usage of the command-line interface is:

npx @agent-tars/cli@latest
npm install @agent-tars/cli@latest -g
agent-tars --provider anthropic --model claude-3-7-sonnet-latest --apiKey your-api-key

Futuristic Agent TARS command center with a terminal floating above a holographic browser, command symbols, and graphs orbiting the combined CLI, browser, and MCP interface.

The README requires Node.js 22 or higher for the global installation of the command-line interface and gives examples with both Anthropic and Volcengine. It also declares execution with a visible web interface or on a headless server. The v0.3.0 update added streaming results for tools — including terminal commands and structured multi-file display — execution parameters with call and reasoning timings, an event viewer, and specific support for the AIO agent Sandbox isolated environment.

Technology and testing

The contribution file identifies Electron as the cross-platform framework; React and Vite for the interface; Zustand and @ui-tars/electron-ipc for state and communication; nut.js for automation; and Vitest and Playwright for testing. These are components declared by the project, not an independent evaluation of their quality or of the models’ accuracy.

Tech-stack architecture with glowing boxes for Electron, React, Vite, Nut.js, and MCP servers interconnected by data pathways.

Official and semi-official status

The repository is an official ByteDance project: the owning account is bytedance, the README publishes its announcements, links the UI-TARS models on Hugging Face and ModelScope, and presents Agent TARS as part of the same stack.

No evidence was retrieved that UI-TARS Desktop has been accepted into an official third-party plugin marketplace or received a certification from Anthropic, OpenAI, Microsoft, or another vendor. It is therefore not appropriate to call it an official integration of those providers. Its MCP support and its examples with model providers are documented technical compatibilities, not endorsements from those providers. The verified 38,440 stars reflect interest on GitHub, not a formal designation as a standard.

The ecosystem

bytedance/UI-TARS: the model’s research repository, described on GitHub as “Pioneering Automated GUI Interaction with Native Agents”; the GitHub search performed for this report showed 11,265 stars.
bytedance/UI-TARS-desktop: this monorepo bundles UI-TARS Desktop and Agent TARS; the search page showed 38,440 stars.
Midscene: the README links Midscene as the option for using the model in the browser. The consulted source establishes the product relationship, but no repository identifier or metrics were verified in this run; no figures are therefore attributed to it.
UI-TARS models on Hugging Face and ModelScope: these are official resources linked from the README for accessing models and deployment; they are not forks of the desktop code.

The GitHub search restricted to ByteDance and “TARS” also returned bytedance/tarsier and bytedance/tar-wasm, but their descriptions talk about video-language models and an experimental browser implementation of tar. They share the term, not a proven integration with UI-TARS Desktop; they should not be presented as components of this product.

Open-source ecosystem with a glowing central core orbited by satellite nodes representing forks, community extensions, and connected tools.

Forks, ports, and community extensions

The exact search for “UI-TARS Desktop” returned 37 repositories. Among the visible results were forks or mirrors with zero stars, and dreadheadlex/UI-TARS-desktop appeared with zero stars. No documentation was retrieved to demonstrate that it is a distinct port, so it is classified only as a fork or mirror, not as a derivative product.

yangzhuxinyzx/auto-openclaw also appeared, with 18 stars: its description says it gives OpenClaw graphical-interface automation driven by vision-language models and mentions UI-TARS. It is a related community extension, not an official ByteDance integration. No non-English translation or community port with sufficient documentation and verifiable relationship to describe as part of the ecosystem was found during this query.

Repo numbers

Measurement: August 5, 2026; GitHub API and pages consulted in this run.

Metric	Value
Stars	38,440
Forks	3,869
Real subscribers	277
Commits shown by the GitHub page	1,110
Open issues reported by the API	416
Branches	278
Tags	547
Main language	TypeScript
License	Apache License 2.0
Created	January 19, 2025
Last metadata update	August 5, 2026
Last push reported by the API	August 5, 2026
Latest release retrieved	`v0.3.0`, November 4, 2025

The API returns watchers_count with the same value as the stars; because of that known duplication, subscribers_count (277) is reported as the real number of subscribers. open_issues_count may include open pull requests, so 416 does not necessarily equal 416 issues exclusively. The separate GitHub page showed 320 open issues and 96 open pull requests in its interface at the time of the query; the difference from the API’s aggregate field is another reason not to interpret the counter as an exact breakdown.

The contributors API query was rejected due to the unauthenticated rate limit. This report therefore does not invent a contributor ranking. ycjcl868 was observed as the author of the latest commit shown on the root page, and ulivz as the author of the latest change shown in CONTRIBUTING.md; that data does not constitute a contribution ranking.

How to contribute

The repository documents a contribution process. It requires Node.js 20 or higher and pnpm 9 or higher. The documented dev startup is:

git clone https://github.com/bytedance/ui-tars-desktop.git
cd ui-tars-desktop
pnpm install
pnpm run dev:ui-tars

Developer workspace terminal running the pnpm install and git clone commands, with holographic testing and build-pipeline elements floating around it.

To also reload Electron’s main process during debugging, pnpm run dev:w is used. The document offers per-platform builds with pnpm run build or the commands publish:mac-x64, publish:mac-arm64, publish:win32, and publish:win32-arm64.

Contributions must go to a feature branch on a fork and arrive via pull request; the contributor must accept the CLA. Before opening an issue, contributors are asked to read the documentation, search existing issues, and provide context along with platform, Node.js, and npm versions where relevant. Husky and lint-staged run Prettier formatting and strict type checking; every pull request and every push to main trigger unit tests (pnpm run test) and end-to-end tests (pnpm run test:e2e).

How the community received it

The retrieved public reception is limited but contains concrete signals of interest and friction:

Holographic display of community reception showing streams of forum comments, bug reports, and feature requests, with some lines of code flickering with red error warnings.

The Hacker News submission 42806473, posted by thekevan on January 23, 2025, presented the project as an alternative to Anthropic’s Computer Use. It reached 8 points and 1 comment. The only comment, also by thekevan, specifically highlighted that it was Apache-licensed. That is concrete appreciation for the license, not a broad functional review.
In 46540439, mark_l_watson titled his submission as a local ByteDance agent one might feel safe running. It got 2 points and 1 comment. The title expresses an individual positive assessment of the local option; no discussion sufficient to generalize it was retrieved.
The search also found the submissions 42802185 by epaga (4 points, 0 comments) and 44876801 by prashantsengar (3 points, 0 comments). They document reach, not community praise or criticism, because they had no conversation.
On GitHub, NiksaSprinting opened issue #591 to report skipped clicks; it remains open on the page consulted. love01211 opened #587 about a maximum-context-length error in the UI-TARS-2B-SFT model. These are concrete technical objections from users, not a demonstration that the defects affect every use case.
groupsell opened issue #366 alleging that a Windows installer could not be found and that installation documentation was inconsistent; the page showed it closed as completed. This is a verifiable criticism about distribution and documentation at that time, not a claim about the installer’s current state.

The cautious conclusion is that there was interest in an open, local alternative, while the visible issues record click failures, context limits, and installation problems. Neither the few Hacker News threads nor the star count allow inferring consensus on accuracy, safety, or maturity.

UI-TARS Desktop versus other proposals

Proposal	Verifiable match	Verifiable difference
Anthropic Computer Use	Hacker News submission 42806473 presented UI-TARS Desktop as an alternative; both belong to the computer-interaction problem space.	The source retrieved for UI-TARS Desktop declares a local application, UI-TARS/Seed models, and Apache 2.0 code; this report did not retrieve a technical specification of Computer Use for a performance comparison.
`bytedance/UI-TARS`	It is the model’s research repository that the README links to the desktop application.	UI-TARS Desktop adds an Electron application, operators, SDK, and agent infrastructure; `UI-TARS` is presented on GitHub as a research repository.
Midscene	The README links it for browser use, and the Hacker News search picked up that UI-TARS can be used with Midscene.	The consulted source did not allow verifying Midscene’s architecture, license, or metrics in this run; a complete functional equivalence is not claimed.
`yangzhuxinyzx/auto-openclaw`	Its description relates it to graphical-interface automation guided by vision-language models and UI-TARS.	It is a community extension for OpenClaw with 18 stars in the search, not ByteDance’s official project.

Use cases and who this repository can help

People who need to operate a desktop application through an instruction can use UI-TARS Desktop for scenarios where the visual interface is the available surface, such as changing a VS Code preference; they should validate each action, especially if the flow can modify data or configurations.
Teams automating web tasks with varying degrees of structure can evaluate Agent TARS for its visual, DOM, or hybrid browser mode. The choice is concrete: the DOM can provide structure when it exists; the visual mode covers controls that are not sufficiently exposed; the combination tries to leverage both.
Developers already using MCP tools can connect Agent TARS to MCP servers and combine the graphical interface with real tool operations, for example fetching data for a chart. This benefits integrated flows, but requires controlling the permissions, servers, and keys of each connection.
Maintainers of cross-platform Electron-based applications can study the monorepo and its operator, IPC, SDK, and visualizer packages, or contribute following its unit and end-to-end tests. This is not a guarantee that the model will be appropriate for a critical production environment without independent evaluation.

Resources

Repository: https://github.com/bytedance/UI-TARS-desktop
Documentation and installation: https://github.com/bytedance/UI-TARS-desktop#quick-start
Contribution documentation: https://github.com/bytedance/UI-TARS-desktop/blob/main/CONTRIBUTING.md
Official model and research: https://github.com/bytedance/UI-TARS
Official models: https://huggingface.co/ByteDance-Seed and https://modelscope.cn/
Community: https://github.com/bytedance/UI-TARS-desktop/discussions
Discussions and references: https://news.ycombinator.com/item?id=42806473, https://news.ycombinator.com/item?id=46540439, https://github.com/bytedance/UI-TARS-desktop/issues/591, https://github.com/bytedance/UI-TARS-desktop/issues/366

Note: this article combines the project’s README and contribution guide, GitHub pages and API, and Hacker News, consulted on August 5, 2026. Figures change over time; when GitHub did not allow retrieving a data point, this is explicitly noted.