Browser & screenshot loop

The client sees the real confBuild result

After a commit, the MCP server opens the persisted project URL in the confBuild editor, waits for the scene, hides common transient UI, and captures the 3D canvas from multiple directions. PNGs are returned to Codex or Claude as native MCP image content. The client also receives machine-readable browser and Three.js diagnostics.

The client—not the server—interprets the images and decides on the next patch.

Browser modes

Mode Behavior Best for
auto Uses a configured CDP browser, otherwise a visible persistent browser, with headless fallback on launch failure Recommended default
headed Starts or reuses a visible Chromium profile and leaves it open for the MCP process lifetime User watches agent work in parallel
headless Starts an isolated invisible browser and closes it after rendering CI and reproducible unattended checks
attached Connects only to CONFBUILD_MCP_CDP_URL Deliberately exposed existing debug-browser session

With browserMode: auto or headed, you can watch the confBuild editor alongside the agent. The profile is stored by default at:

output/playwright/confbuild-mcp/browser-profile

It persists across render calls, so authentication and browser state can be reused. This browser remains separate from your normal Chrome profile.

Attach the exact existing browser

A normally started browser does not expose a safe attach endpoint. To use a specific visible browser session, that browser must be deliberately started with a local Chrome DevTools Protocol endpoint and exposed through CONFBUILD_MCP_CDP_URL.

CONFBUILD_MCP_CDP_URL='http://127.0.0.1:9222' npm run mcp:confbuild

The server never enables remote debugging itself or attempts to take over arbitrary user windows. Protect the debugging endpoint, bind it locally, and preferably use a separate browser profile. Stopping the MCP process does not close an explicitly attached browser.

Asynchronous render sequence

Browser rendering can take longer than a typical tool timeout. The flow therefore uses two tools:

  1. confbuild_render_project starts the job and immediately returns a renderJobId.
  2. confbuild_get_render_result is polled until the job is completed.

Example arguments:

{
  "editSessionId": "edit-…",
  "browserMode": "auto",
  "reuseOpenTab": true,
  "views": ["default", "right", "front", "left"],
  "timeoutMs": 120000
}

The server renders only persisted data. If an edit session is dirty, the client must validate and commit first.

Returned images

The default result contains up to four PNGs:

  • default: current view after zoom-to-fit
  • right: orbit perspective moved to the right
  • front: opposite/front-oriented orbit perspective
  • left: orbit perspective moved to the left

These views are browser-driven orbit presets, not guaranteed orthographic CAD projections. Use confBuild’s dedicated drawing and export features for technical drawings or dimensional verification.

Diagnostics

In addition to images, the job returns values such as:

Field Meaning
meshCount / visibleMeshCount Total and visible Three.js mesh counts
uniqueOutputIds Number of detected unique confBuild outputs
subprojectCount / subsheetCount Detected subprojects and sub-Sheets
attachedCount Objects attached to reference points
unresolvedRefposCount Unresolved reference positions
bounds World-coordinate minimum, maximum, and model size
topLevelOutputIds Sample of top-level output IDs
errorText Visible editor or snackbar errors
browserErrors Recent console and page errors

Diagnostics complement the image; they do not replace it. A high mesh count, for example, says nothing about whether the object is recognizable or assembled correctly.

Client visual checklist

The agent should inspect every image and evaluate at least:

  1. Recognition: Does the silhouette match the request?
  2. Completeness: Are all required major assemblies present?
  3. Scale: Are overall dimensions and proportions plausible?
  4. Connections: Do components meet at their intended interfaces?
  5. Collisions: Are there unintended intersections or duplicate geometry?
  6. Floating parts: Are any objects disconnected or outside the model bounds?
  7. Parametrics: Do visible patterns and spacings remain consistent?
  8. Diagnostics: Are there errors, unresolved references, or extreme bounds?

An agent must not report “visually verified” until the final render is complete and every returned image has actually been inspected.

Watch and intervene in parallel

In headed or attached mode, you can watch the model during the loop. If you save manually while the agent still works from an older revision, revision protection blocks the next MCP commit. The client should re-read the latest version and rebase its changes.

Avoid major manual camera or UI changes during a screenshot series. The renderer closes common dialogs and selection states, but editor UI changes can still affect captures.

Limitations and improvements

  • Canvas captures show the 3D scene, not automatically the spreadsheet, dialogs, or full browser chrome.
  • UI selectors and mouse-driven orbits can shift when the editor changes.
  • Views are visual checks, not dimensional or simulation evidence.
  • A small signed browser extension could later nominate an exact user tab safely.
  • Native camera presets and an editor “agent capture mode” would produce more stable, UI-free images.

Next step

Render and polling arguments are listed in the tool reference. For authentication, timeout, or CDP failures, see security & troubleshooting.

Back to Documentation