Key Takeaways

  • Claire Vo tested four personal agents (Grok Bot, Dot, Muse, and Claude Duet) across nine real-world workflows to see which could run a founder's daily load.
  • Grok Bot won the overall test because of its broad range, Figma capabilities, and structured user interface outputs, while Muse took the lead on creative and personal tasks.
  • Claude Duet finished last due to slow speeds and excessive permission popups, though it still solved bug fixing and launch video scripts better than rivals.
  • Vo used an operator agent running computer use to control the competing agents autonomously, eliminating manual human typing during execution.
  • Equalizing agent capability required an automated audit: Vo ran Codex with browser use to verify every agent had the exact same connectors and third-party skills enabled.

The Method

Most software benchmarks test synthetic prompts that do not match daily life. Vo wanted to test whether an AI agent can actually take over real work. She approached the setup in four steps.

First, she had the agents write their own test criteria. “The first thing I did is I went into all my agents,” Vo explained. “I went into Dot, Grockbot, Muse, and Duet, Claude's Duet, and I said, 'I'm going to run a benchmark. Give me like the top tasks that you do for me.'” This produced nine repeatable tasks spanning design work, debugging, creative drafts, and personal scheduling.

Second, Vo stripped out prompt engineering. Instead of multi-paragraph system prompts with detailed constraints, she kept the inputs minimal. “The prompts were just one line. They were very human driven. They weren't those like big AI prompts.” If an assistant requires fifty lines of conditioning just to handle a calendar conflict, it fails as a consumer product.

Third, she standardized connector access. An agent cannot book a flight or audit code if you forgot to link its plugins. Vo used a script where Codex checked the settings pages of each agent: “One tip I have for you all is I had Codex use browser use to check that all my agents were connected to roughly the same stuff and had access to the same skills.”

Fourth, she removed herself from the execution loop. “The way I was going to run this is I was going to have one operator agent which was going to use computer use to run all the other agents.” The operator agent sat in the driver seat, opened each interface, pasted the one-line instruction, and waited for the output.

Where This Breaks Down

Autonomous evaluation sounds clean in theory, but operating an AI through graphical interfaces is still fragile. Vo hit immediate walls with desktop automation tools. “Claude computer use is just hot garbage,” Vo admitted. “I was super frustrated having Claude computer use, but I needed something other than Codex to run Codex.”

Desktop vision models click the wrong coordinates, get trapped in modal dialogs, and drift when web interfaces update. Beyond operator failures, the agents themselves broke under security barriers. Claude Duet stalled because it demanded human approval on simple system actions. When an agent requires five manual confirmations to finish a task, autonomy disappears. The best model on paper loses immediately if its security model prevents it from acting independently.

What to Do With This

Pick the three assistants you use most, give each the exact same one-sentence prompt to plan a company offsite or file an issue, and time how long each takes to finish without your intervention. Count every confirmation dialog you must click. If an assistant forces more than two approvals on an ordinary read-and-draft task, drop it from your daily workflow.