AunySillyMelean-browser-use
Documentation

From docs/benchmark.md · e31dc91 · Built 2026-11-17

Proof

MeasurementValueMeasuredExpires
Runtime dependencies0 dependencies2026-10-252027-01-25
Passing Node tests, including Python suite checks280 tests2026-10-252027-01-25
Failing Node tests at measurement time0 tests2026-10-252027-01-25
Blocking guard rules verified by the suite7 rules2026-10-252027-01-25
Packed tarball size reported before this proof refresh94492 bytes2026-10-252027-01-25
Files in the packed tarball82 files2026-10-252027-01-25
Plain session median input-context tokens per browser step188487 tokens2026-10-25
Runner median input-context tokens per browser step59725 tokens2026-10-25
Plain-to-runner median browser-step token ratio3.16 ratio2026-10-25
Plain arm total browser-step tokens1506374 tokens2026-10-25
Runner arm total browser-step tokens725151 tokens2026-10-25
Plain-to-runner arm total browser-step tokens ratio2.08 ratio2026-10-25
Plain arm whole-run input-context tokens2657290 tokens2026-10-25
Runner arm whole-run input-context tokens2846792 tokens2026-10-25
Plain-to-runner whole-run input-context token ratio0.93 ratio2026-10-25
Estimated break-even browser steps (an estimate from two data points; no run longer than 9 browser steps was measured)15.54 steps2026-10-25
Small task (one page, 4 browser steps), native Claude in Chrome: whole-run input-context tokens1133195 tokens2026-10-25
Small task (one page), with the package, main session plus runner: whole-run input-context tokens1241361 tokens2026-10-25
Small task: plain-to-package whole-run input-context token ratio0.91 ratio2026-10-25
Small task: plain median browser step to runner median browser step token ratio3.26 ratio2026-10-25
Run 1 plain arm dollar cost at list API prices0.5975 usd2026-10-25
Run 1 runner arm dollar cost at list API prices0.5787 usd2026-10-25
Run 2 plain arm dollar cost at list API prices0.5779 usd2026-10-25
Run 2 runner arm dollar cost at list API prices0.4319 usd2026-10-25
Best dollar saving in two measured runs on one setup25.26 percent2026-10-25
Pooled dollar saving across two measured runs on one setup14.02 percent2026-10-25

Reproduce a paired comparison

  1. Choose a public-page task and save its exact prompt. Record the model, browser tools and starting session size for both arms.
  2. Prepare separate scratch folders for the plain arm and the skill-and-runner arm. In the runner scratch folder, install the package into that project:
npx lean-browser-use --project
  1. Run the task prompt once in a plain Claude Code session. Run the same prompt once in a session using the lean-browser-use skill and its runner. Keep each arm's transcripts in its own folder, including subagent transcripts.
  2. Measure each transcript folder with the same package version:
npx lean-browser-use measure --dir ./plain-transcripts
npx lean-browser-use measure --dir ./runner-transcripts

Quote folder paths when they contain spaces. Compare the plain main median with the runner median. For the task totals, sum the main and runner group totals within each arm. Save the meter output and conditions beside the transcripts. How the meter counts explains grouping and token fields.

How far to trust it

These are two paired runs of public-page tasks on one setup. Results move with the model, session size and task. The arms exercise different execution routes, so the comparison describes that pair rather than isolating a single cause.

Tokens re-read are not a price list: cached tokens and fresh input tokens can have different prices. Cost varies by model, and this ballpark guarantees no particular saving. Measure your own similar work to see what changes for you.