How to Build an AI Browser Recovery Harness with CapSolver
Ethan Collins
Pattern Recognition Specialist
TL;DR
- Put browser policy, page-state classification, challenge recovery, checkpoints, telemetry, and cleanup in a harness outside the model.
- Keep one Playwright context for the approved task and use CapSolver Core's
detect,get_captcha_info, orsolve_on_pageat deterministic recovery boundaries. - Allow one bounded recovery attempt, verify that the expected page returns, and route loops or unknown states to operator review.
- Record metadata and artifact references, but redact tokens, cookies, credentials, form contents, and proxy values.
- Test the harness with isolated fixtures and controlled staging pages before attaching it to any AI agent framework.
Introduction
An AI browser recovery harness is the runtime layer that keeps a browser task controlled when page state changes unexpectedly. The model can decide what business step comes next, but the harness should own the Playwright context, approved-host policy, navigation checkpoints, page classification, supported challenge recovery, retry limits, traces, screenshots, and teardown. CapSolver fits inside this layer as a deterministic recovery capability: capsolver-core can detect supported challenges, read parameters, solve them, and fill the result back into the same page. The harness then verifies that the expected application state returned before allowing the agent to continue. This guide builds the policy model, state machine, async context manager, recovery function, artifact recorder, OpenTelemetry spans, tests, and production controls for reliable authorized browser automation.
What Belongs in the Harness
The harness is not the model and not the browser driver alone. It is the control plane between them.
text
Business goal from agent
↓
Browser recovery harness
├─ target policy
├─ Playwright context
├─ state classifier
├─ checkpoint store
├─ CapSolver recovery
├─ retry budget
├─ trace + artifacts
└─ cleanup
↓
Approved page action or operator reviewPlaywright's fixture documentation emphasizes isolated page and browser-context fixtures, reusable setup and teardown, composability, and automatic debug attachments. Those properties translate directly into a production harness.
The CapSolver Core SDK documentation defines four useful browser stages: detect, get_captcha_info, solve, and solve_on_page.
Separate Agent Decisions from Runtime Decisions
The model may decide to open a known product page or read a public status. The harness decides whether the requested host is allowed, whether the current page is expected, whether recovery is supported, and whether the retry budget remains.
| Decision | Owner | Reason |
|---|---|---|
| Next business step | Agent or workflow | Requires task context |
| Host and path permission | Harness policy | Must be deterministic |
| Page-state classification | Harness classifier | Must use trusted DOM/network evidence |
| Challenge recovery call | Harness | Requires secrets and browser object |
| Token/cookie handling | Harness | Sensitive runtime data |
| Continue vs review | Harness state machine | Enforces bounded recovery |
| Final submission | Human or dedicated service | High-impact action |
The CapSolver AI Agents guide explains the same division of labor: the model handles reasoning, while CapSolver's layers perform the supported challenge work.
Define a Target Policy
Start with a narrow policy for approved hosts, paths, actions, and budgets.
python
from dataclasses import dataclass, field
from urllib.parse import urlparse
@dataclass(frozen=True)
class TargetPolicy:
allowed_hosts: set[str]
allowed_path_prefixes: tuple[str, ...]
max_navigations: int = 20
max_recovery_attempts: int = 1
capture_screenshots: bool = True
capture_html: bool = False
allow_form_submission: bool = False
def validate_url(self, url: str) -> None:
parsed = urlparse(url)
if parsed.scheme != "https":
raise PermissionError("Only HTTPS targets are permitted")
if parsed.hostname not in self.allowed_hosts:
raise PermissionError("Host is outside the approved policy")
if not parsed.path.startswith(self.allowed_path_prefixes):
raise PermissionError("Path is outside the approved policy")Use tenant-specific policies. Do not maintain one global allowlist for unrelated customers or projects.
The CapSolver AI and automation FAQ provides integration context, while the CapSolver web-scraping FAQ covers responsible public-data workflows.
Model the Browser State Machine
A recovery harness should use explicit states rather than an unbounded “try again” loop.
python
from enum import Enum
class BrowserState(str, Enum):
EXPECTED_PAGE = "expected_page"
SUPPORTED_CHALLENGE = "supported_challenge"
UNKNOWN_PAGE = "unknown_page"
RECOVERING = "recovering"
RECOVERED = "recovered"
REVIEW_REQUIRED = "review_required"
FAILED = "failed"Permitted transitions can be represented as data:
python
ALLOWED_TRANSITIONS = {
BrowserState.EXPECTED_PAGE: {
BrowserState.EXPECTED_PAGE,
BrowserState.SUPPORTED_CHALLENGE,
BrowserState.UNKNOWN_PAGE,
},
BrowserState.SUPPORTED_CHALLENGE: {
BrowserState.RECOVERING,
BrowserState.REVIEW_REQUIRED,
},
BrowserState.RECOVERING: {
BrowserState.RECOVERED,
BrowserState.REVIEW_REQUIRED,
BrowserState.FAILED,
},
BrowserState.RECOVERED: {
BrowserState.EXPECTED_PAGE,
BrowserState.REVIEW_REQUIRED,
},
}Validate every transition. This makes loops visible and testable.
Create a Browser Checkpoint
A checkpoint records safe metadata needed to determine whether the workflow resumed correctly.
python
from dataclasses import dataclass
from datetime import datetime, timezone
@dataclass
class BrowserCheckpoint:
url: str
title: str
expected_selector: str | None
navigation_index: int
recovery_attempts: int
observed_at: str
async def checkpoint(page, expected_selector, nav_index, attempts):
return BrowserCheckpoint(
url=page.url,
title=await page.title(),
expected_selector=expected_selector,
navigation_index=nav_index,
recovery_attempts=attempts,
observed_at=datetime.now(timezone.utc).isoformat(),
)Do not store storage state, cookies, passwords, tokens, or full form values in the checkpoint.
Classify the Current Page
Use trusted DOM evidence, title, URL, and expected selectors. Never ask the model to infer page state from a screenshot alone.
python
async def classify_page(page, expected_selector: str) -> BrowserState:
if await page.locator(expected_selector).count():
return BrowserState.EXPECTED_PAGE
title = (await page.title()).strip().lower()
html = (await page.content()).lower()
challenge_markers = (
"just a moment...",
"challenge-platform",
"cf-chl-",
)
if any(marker in title or marker in html for marker in challenge_markers):
return BrowserState.SUPPORTED_CHALLENGE
return BrowserState.UNKNOWN_PAGEUse target-specific markers and controlled fixtures. A marker set is a routing heuristic, not an access entitlement.
The CapSolver CAPTCHA-solving FAQ explains supported challenge workflows, and the CapSolver errors FAQ helps classify failures.
Initialize CapSolver Core Once per Harness
The official Core SDK recommends using its async context manager so HTTP connections are reused and released correctly.
python
import os
from capsolver_core import create_capsolver
def create_recovery_client():
return create_capsolver(
api_key=os.environ["CAPSOLVER_API_KEY"],
default_timeout=120,
polling_interval=5,
request_timeout_ms=30000,
source="ai-browser-recovery-harness",
version="1.0.0",
)Do not create a new client for every DOM check. Keep one client for the harness lifecycle and close it during teardown.
Implement a Bounded Recovery Function
Use detect and get_captcha_info for diagnostics, then solve_on_page for the all-in-one browser flow.
python
from capsolver_core import SolveOnPageOptions
async def recover_supported_challenge(
cap,
page,
policy: TargetPolicy,
recovery_attempts: int,
) -> dict:
policy.validate_url(page.url)
if recovery_attempts >= policy.max_recovery_attempts:
return {
"success": False,
"state": BrowserState.REVIEW_REQUIRED,
"reason": "recovery budget exhausted",
}
detected = await cap.detect(page)
if not detected:
return {
"success": False,
"state": BrowserState.REVIEW_REQUIRED,
"reason": "no supported challenge detected",
}
infos = await cap.get_captcha_info(page)
results = await cap.solve_on_page(
page,
options=SolveOnPageOptions(
autofill=True,
throw_on_error=False,
timeout=120,
polling_interval=5,
),
)
errors = [item.error for item in results if item.error]
filled = bool(results) and all(item.filled for item in results)
return {
"success": filled and not errors,
"state": (
BrowserState.RECOVERED
if filled and not errors
else BrowserState.REVIEW_REQUIRED
),
"detected_count": len(detected),
"info_count": len(infos),
"result_count": len(results),
"errors": errors,
}Keep the original page object. The point of solve_on_page is to detect, solve, and fill back within the existing browser session.
Verify Recovery Before Continuing
A successful tool response does not prove that the expected application page returned.
python
async def verify_recovery(
page,
expected_selector: str,
timeout_ms: int = 15000,
) -> bool:
try:
await page.locator(expected_selector).wait_for(
state="visible",
timeout=timeout_ms,
)
return True
except Exception:
return FalseAfter recovery, classify the page again. If the challenge remains or the expected selector is absent, stop and request review.
python
async def recover_and_verify(cap, page, policy, expected_selector, attempts):
result = await recover_supported_challenge(
cap=cap,
page=page,
policy=policy,
recovery_attempts=attempts,
)
if not result["success"]:
return result
if not await verify_recovery(page, expected_selector):
return {
"success": False,
"state": BrowserState.REVIEW_REQUIRED,
"reason": "expected page did not return after recovery",
}
return {
"success": True,
"state": BrowserState.EXPECTED_PAGE,
"reason": "page recovered and verified",
}Build the Async Harness Context
Use an async context manager to guarantee cleanup.
python
from contextlib import asynccontextmanager
from playwright.async_api import async_playwright
@dataclass
class BrowserHarness:
policy: TargetPolicy
playwright: object
browser: object
context: object
page: object
capsolver: object
navigation_count: int = 0
recovery_attempts: int = 0
@asynccontextmanager
async def browser_recovery_harness(policy: TargetPolicy):
async with async_playwright() as playwright:
browser = await playwright.chromium.launch(headless=True)
context = await browser.new_context()
page = await context.new_page()
async with create_recovery_client() as cap:
harness = BrowserHarness(
policy=policy,
playwright=playwright,
browser=browser,
context=context,
page=page,
capsolver=cap,
)
try:
yield harness
finally:
await context.close()
await browser.close()The model or workflow receives controlled methods, not raw unrestricted browser access.
Expose Narrow Harness Actions
python
async def safe_navigate(
harness: BrowserHarness,
url: str,
expected_selector: str,
) -> dict:
harness.policy.validate_url(url)
if harness.navigation_count >= harness.policy.max_navigations:
return {
"success": False,
"state": BrowserState.REVIEW_REQUIRED,
"reason": "navigation budget exhausted",
}
harness.navigation_count += 1
await harness.page.goto(url, wait_until="domcontentloaded")
state = await classify_page(harness.page, expected_selector)
if state == BrowserState.EXPECTED_PAGE:
return {"success": True, "state": state}
if state == BrowserState.SUPPORTED_CHALLENGE:
result = await recover_and_verify(
cap=harness.capsolver,
page=harness.page,
policy=harness.policy,
expected_selector=expected_selector,
attempts=harness.recovery_attempts,
)
harness.recovery_attempts += 1
return result
return {
"success": False,
"state": BrowserState.REVIEW_REQUIRED,
"reason": "unknown page state",
}The agent can request safe_navigate, but the harness owns the policy and recovery path.
Record Redacted Telemetry
OpenTelemetry's GenAI observability guidance describes traces for model and tool operations. It also notes that full prompt and tool content can contain sensitive data. Default to metadata-only spans.
python
from opentelemetry import trace
tracer = trace.get_tracer("capsolver.browser_harness")
async def traced_safe_navigate(harness, url, expected_selector):
with tracer.start_as_current_span("browser.safe_navigate") as span:
span.set_attribute("browser.target_host", url.split("/")[2])
span.set_attribute("browser.navigation_index", harness.navigation_count + 1)
span.set_attribute("browser.recovery_attempts", harness.recovery_attempts)
result = await safe_navigate(harness, url, expected_selector)
span.set_attribute("browser.outcome", str(result.get("state")))
span.set_attribute("browser.success", bool(result.get("success")))
return resultDo not attach tokens, cookies, API keys, proxy credentials, storage state, prompt content, or full page HTML to spans.
Capture Artifacts Only on Failure
Screenshots and HTML can contain personal or confidential data. Capture them only when policy allows, redact where possible, and store short-lived references.
python
from pathlib import Path
import secrets
async def capture_failure_artifacts(harness, directory: Path) -> dict:
artifact_id = secrets.token_hex(12)
screenshot = directory / f"{artifact_id}.png"
await harness.page.screenshot(
path=str(screenshot),
full_page=False,
)
return {
"artifact_id": artifact_id,
"screenshot_path": str(screenshot),
"url": harness.page.url,
"title": await harness.page.title(),
}Use retention limits and access controls. Avoid capturing full-page screenshots when only the top-level state is needed.
The CapSolver browser automation blog contains related implementation patterns, and the CapSolver Chrome extension guide can help teams inspect supported widget parameters during development.
Test the Harness with Fixtures
Use isolated browser contexts and controlled pages. Playwright fixtures provide reusable setup and teardown.
python
import pytest
@pytest.mark.asyncio
async def test_unknown_host_is_rejected():
policy = TargetPolicy(
allowed_hosts={"staging.example.com"},
allowed_path_prefixes=("/qa/",),
)
with pytest.raises(PermissionError):
policy.validate_url("https://other.example.net/qa/test")
@pytest.mark.asyncio
async def test_recovery_budget_is_bounded(fake_cap, fake_page):
policy = TargetPolicy(
allowed_hosts={"staging.example.com"},
allowed_path_prefixes=("/qa/",),
max_recovery_attempts=1,
)
result = await recover_supported_challenge(
cap=fake_cap,
page=fake_page,
policy=policy,
recovery_attempts=1,
)
assert result["state"] == BrowserState.REVIEW_REQUIRED
assert result["reason"] == "recovery budget exhausted"Create fixtures for no challenge, supported challenge, unknown interstitial, successful fill-back, solve failure, post-recovery challenge loop, and missing expected selector.
Define Reliability Metrics
| Metric | Purpose |
|---|---|
| Expected-page rate | Measures successful normal navigation |
| Challenge encounter rate | Shows source friction by approved host |
| Recovery success rate | Measures supported recovery outcomes |
| Challenge-loop rate | Detects repeated interstitial state |
| Unknown-page rate | Finds layout, auth, or policy changes |
| P95 recovery latency | Tracks user-visible delay |
| Operator-review rate | Measures unresolved workflow volume |
| Artifact capture rate | Detects excessive failure logging |
Break metrics down by target policy, route, browser version, challenge type, and harness version. Never label a challenge recovery failure as a business-task failure without preserving both dimensions.
Bonus Code: Use code WEBS at CapSolver Dashboard to get an extra 5% bonus on every recharge.
Comparison Summary
| Approach | Browser ownership | Recovery control | Best use |
|---|---|---|---|
| Direct agent browser access | Agent runtime | Prompt-dependent | Low-risk prototypes only |
| Framework-specific action | Agent framework | Tool wrapper | Fast integration |
| Dedicated recovery harness | Independent control layer | Deterministic state machine | Production reliability and governance |
| Human-only recovery | Operator | Manual | Unsupported or high-risk workflows |
A dedicated harness requires more engineering, but it creates one policy and observability layer that can serve multiple agent frameworks.
Production Checklist
- Use an approved-host and path policy per tenant.
- Keep CapSolver and browser credentials outside prompts and traces.
- Use one browser context per controlled task.
- Classify the page before and after recovery.
- Permit only one recovery attempt unless a reviewed scenario justifies more.
- Stop on unknown pages, repeated challenges, or missing expected selectors.
- Redact tool and browser secrets from telemetry.
- Capture failure artifacts only under an explicit retention policy.
- Require confirmation before submissions, purchases, account changes, or other high-impact actions.
The CapSolver products page lists supported solution categories, while the CapSolver AI blog covers agent-framework examples that can call a harness action.
Responsible Use
Use the browser recovery harness only on systems you own, test, or have explicit authorization to automate. A successful challenge solution does not grant permission to access private content, ignore authentication boundaries, exceed rate limits, or perform transactions. Keep the harness scoped, read-only by default, and auditable. Route uncertainty to a person instead of expanding permissions dynamically.
Conclusion
An AI browser recovery harness turns challenge handling into a controlled runtime capability. It owns the browser context, validates targets, classifies page state, invokes CapSolver Core at a deterministic boundary, verifies the expected page, records redacted telemetry, and stops after a bounded attempt. Agent frameworks can use the harness without gaining direct access to secrets or unrestricted browser control.
Start with CapSolver, implement the state machine against an approved staging application, and add isolated fixtures and reliability gates before production.
FAQ
Is a browser recovery harness an agent framework?
No. It is an independent runtime and policy layer that an agent framework can call. The harness owns browser state, recovery, checkpoints, telemetry, and cleanup.
Why use solve_on_page?
solve_on_page combines detection, parameter extraction, solving, and DOM fill-back on the same Playwright page, which makes it suitable for a controlled browser recovery boundary.
Should the model receive the Playwright page object?
Prefer narrow harness actions such as safe_navigate and read_public_page. Raw page access makes it harder to enforce target, navigation, and recovery policies.
How many recovery attempts should be allowed?
Use one attempt by default. Repeated challenges or unknown page state should route to operator review instead of creating an uncontrolled loop.
What telemetry should be stored?
Store metadata such as target host, harness version, state transitions, latency, normalized errors, and artifact references. Do not store solution tokens, cookies, API keys, proxy credentials, storage state, or private page content.
More
AIAug 27, 2026
How to Build a CAPTCHA Evaluation Harness for AI Agent Tool Calls
Build a CAPTCHA evaluation harness for AI agent tool calls with CapSolver schemas, fixtures, trace graders, assertions, regression datasets, and CI gates.
AIAug 27, 2026
How to Solve Cloudflare Turnstile in LlamaIndex Agents
Learn how to solve Cloudflare Turnstile in LlamaIndex agents with CapSolver, FunctionTool schemas, secure token handling, retries, and browser recovery.
