Mage-VL-Cowork

Research checked 6 November 2026. Publisher claims and third-party trials are identified separately; this page is not an AIDreamHub hands-on benchmark.

A model policy, not a ready-made office assistant

Mage-VL-Cowork is a Microsoft 4B vision-language checkpoint that selects between computer_use and bash. It can reason over screenshots and propose GUI actions or programmatic file operations. The model alone does not provide a desktop runtime, permissions, application setup or a reliable business workflow.

Its useful distinction is channel choice: use visual control for canvas or live application state, and a file library for a structured spreadsheet edit. A small-model experiment should begin with one resettable office task, not personal email, banking or an unrestricted logged-in workstation.

Official project page captured by AIDreamHub; not a locally run product.

Read the 66.0% claim with its conditions

The model card reports a 66.0% two-run average on 361 OSWorld-Verified tasks excluding Google Drive, with a 50-step budget. The configuration includes a 1920×1080 desktop, a bounded screenshot history and a specific hybrid agent. It is publisher evidence, not a general office-automation success rate or our reproduction.

A consulting analysis also highlights that the training recipe and execution scaffold matter. It does not turn the checkpoint into an independently certified worker. Compare tasks under the same application versions, tool schemas, step budget and reset procedure before using model scores for a deployment decision.

How to use: preserve the custom architecture contract

The documented architecture is llava_onevision2 with custom model code, not a drop-in standard Qwen3-VL weight set. Review and pin remote code before using trust_remote_code. The official package provides a dependency-pinned SGLang container and reference agent; the example below is a documented setup route, not an installation we ran.

Download the model, review the container and mount the model directory read-only. Bind the inference port to loopback and use the bundled processor and native /generate protocol for the reference screenshot path. A generic chat request without the image template and action executor does not recreate the published pipeline.

export MODEL_DIR=/absolute/path/to/Mage-VL-Cowork
docker pull xiaoyizhang/mage-vl-cowork:0.1.0
docker run --rm --name mage-vl-cowork --gpus 'device=0' --shm-size=8g -p 127.0.0.1:30001:30001 --mount "type=bind,src=$MODEL_DIR,dst=/models/Mage-VL-Cowork,readonly" xiaoyizhang/mage-vl-cowork:0.1.0

The executor must implement what the model proposes

computer_use returns a post-action screenshot and uses normalised coordinates; bash returns execution text. The environment must parse, authorise and run those actions, then give the actual result back. Keep the current screenshot and file state aligned so a successful direct edit is not followed by a click on a stale dialog.

For an illustrative spreadsheet task, give a disposable file, expected cell values and a save destination. Check values with a parser and reopen the document visually. A tool-call JSON object or a “saved” message is not proof that the correct file was modified.

A bounded office pilot

Select a small set of tasks with deterministic success checks: change a known cell, export a document, move approved sample files. Include a missing-file case, an ambiguous target and a denied action. Stop when the step budget is exhausted; do not quietly grant more permissions to make a demonstration pass.

Measure complete tasks, retries and human interventions, not clicks alone. Distinguish “understood screenshot,” “issued valid action” and “produced accepted document.” Those stages can fail independently.

When scripts or another agent are a better fit

Use deterministic file processing when the task is fully specified and the source format is accessible. Evaluate a hybrid model when visual state or application-specific interaction matters. A managed computer-use service may reduce runtime engineering, but still needs approvals and result checks.

The 4B label does not specify total VRAM: the card lists about 9.48 GB of BF16 shards, before runtime overhead and image/context memory. Do not advertise every laptop as supported. Keep source code, processor and environment revisions with the test report.

Safety and license boundaries

The checkpoint is Apache-2.0-labelled. Its own deployment guidance calls for disposable isolation, scoped authorisation, network restrictions and no personal credentials in the guest. An application screenshot can contain untrusted instructions; text found on screen is not permission to run a shell command.

Require human approval before external submissions or irreversible operations. We reviewed the documentation and commentary, not an independently executed office benchmark. The hand drawing illustrates channel selection, not a measured result.

Questions before adoption

Can I just load the model and let it use my desktop?

Loading weights produces model responses, not a complete safe executor. Use the documented custom architecture, processor, action contract and isolated environment. Do not expose a personal workstation or credentials while testing a research policy.

Does 66.0% mean it completes two thirds of my office work?

No. It is a publisher result on a specific 361-task OSWorld configuration, excluding Google Drive and using a 50-step limit. Your applications, inputs and permissions differ. Run your own resettable task set with objective artifact checks.

When should it use bash rather than GUI clicks?

When a permitted structured file operation can be performed and verified programmatically, direct editing can be the clearer channel. GUI interaction remains useful for visual layouts and live application state. The executor must authorise both channels and report actual outcomes.

Sources reviewed

• Microsoft — model card and execution contract
• Microsoft — reference agent and serving
• Codesprint Consulting — architecture analysis

Explore related tools

• AI agents
• Productivity tools