Store Inventory Data Collection: CAPTCHA Recovery Guide

TL;DR

• Store inventory data collection is reliable only when product identity, location, stock state, and collection time are captured together.
• A CAPTCHA checkpoint must pause the inventory workflow; the job should not continue until the same browser session receives and validates a solution.
• The current CapSolver reCAPTCHA v2 API flow uses ReCaptchaV2TaskProxyLess, websiteURL, and websiteKey, then returns gRecaptchaResponse after a ready result.
• Inventory availability must be verified from an application state such as an in-stock label, pickup location result, cart eligibility response, or structured product payload—not from the solver response alone.
• Bounded polling, one controlled replay, per-store rate limits, and a clear needs_review state prevent loops and misleading stock alerts.
• This workflow is appropriate only for lawful, reasonable, responsible, and user-authorized collection from pages you are permitted to access.

Introduction

Store inventory data collection looks simple until a retail page changes behavior by location, requires a store selector, or pauses an authorized browser session at a CAPTCHA. A dependable collector must preserve the product and store context, solve the checkpoint through the same session, and prove that the page returned a real stock state. CapSolver can serve as the CAPTCHA infrastructure inside that controlled recovery step.

This guide focuses on a concrete retail operations scenario: monitoring public product availability for approved replenishment planning, merchandising QA, or customer-facing stock notifications. It does not assume that a successful CAPTCHA task means the inventory request succeeded. The workflow ends only when the target application supplies a recognized inventory outcome and the collector records enough evidence to distinguish “out of stock” from “unknown.”

What Store Inventory Data Collection Must Return

Store inventory data collection should return a small, stable record that downstream systems can trust. A useful record contains the retailer, canonical product identifier, requested store or postal area, observed availability, collection time, and evidence source. Price may be included, but it should not replace an explicit stock state.

{
  "retailer": "authorized-demo-store",
  "product_id": "SKU-4821",
  "store_id": "STORE-017",
  "postal_area": "10001",
  "availability": "in_stock",
  "quantity_hint": "limited",
  "observed_at": "2026-08-13T09:15:00Z",
  "source": "product-page",
  "verification": "inventory-label-and-store-id",
  "captcha_recovery": "completed"
}

The input is a product URL plus a permitted store context. The operation selects or confirms the location, detects a verification checkpoint, resolves it when authorized, reads the inventory state, and normalizes the result. The output is the record above or a terminal state such as not_found, out_of_stock, needs_review, or policy_denied. Never convert a timeout, challenge loop, login wall, or malformed response into out_of_stock; that mistake creates false business signals.

The CapSolver CAPTCHA-solving FAQ explains the service boundary, while the browser automation guide gives broader context for maintaining page state. In this user case, the browser owns navigation and evidence, CapSolver owns the documented CAPTCHA task, and your application owns authorization, retry policy, and final verification.

Why CAPTCHA Is a Core Inventory Failure Boundary

A retail collector commonly performs several stateful actions before inventory appears: load a product page, accept a region setting, select a store, open a pickup panel, or call a public inventory endpoint initiated by the page. A CAPTCHA can appear before the first page renders or after one of those actions. If the collector treats it as generic HTML, selectors fail and the job may write an empty or incorrect record.

The right response is a state transition, not a blind retry. The collector moves from collecting to captcha_required, freezes the current product and location context, gathers only the documented challenge fields, and calls the provider adapter. A ready provider response moves the workflow to apply_solution; application acceptance moves it to verify_inventory. A rejected result, repeated checkpoint, unexpected host, or exhausted deadline moves it to needs_review.

This design keeps three different truths separate:

• Provider truth: the CAPTCHA task is ready and contains the documented solution field.
• Browser truth: the token or callback was applied to the original page and the checkpoint changed state.
• Business truth: the requested store and product returned a recognized inventory outcome.

Only the third truth allows the job to publish an inventory record. This separation is especially important when a retailer uses cached content, redirects between regional domains, or updates stock asynchronously after the initial HTML is loaded.

Architecture for an Authorized Inventory Collector

Use six components with narrow responsibilities.

1. Scope registry

The scope registry lists approved hostnames, collection purposes, product paths, store regions, schedules, and contact owners. It should also record exclusions: authenticated account areas, employee portals, checkout, payment, personal profiles, and any path that the site owner or your agreement places outside scope. A request that does not match the registry stops before the browser opens.

2. Browser session

The browser establishes locale, store selection, cookies, and navigation state. Keep one product check inside one browser context. Reusing unrelated cookies across stores can create confusing location drift; discarding the context during a CAPTCHA recovery can invalidate the solution. Persist only the minimum data needed for the job and clear it according to your retention policy.

3. Checkpoint detector

The detector looks for explicit widget elements, known response fields, challenge scripts, or a verification route. It also distinguishes a CAPTCHA from ordinary problems such as a 404, a consent dialog, an unavailable store, or an application error. The CapSolver glossary is useful for keeping challenge terminology consistent across logs and runbooks.

4. CapSolver adapter

The adapter receives a narrow request containing challenge type, page URL, site key, and a correlation ID. It reads the API key from secret storage, creates one task, polls that same task with a deadline, validates the result schema, and returns a normalized success or failure. It does not decide which sites are permitted and it does not write inventory data.

5. Inventory extractor

The extractor maps the page or public response into a stable inventory schema. It should prefer durable product identifiers, store IDs, structured data, and explicit availability text over brittle visual position. If the page contains several fulfillment modes, record pickup, shipping, and local delivery separately rather than merging them into one boolean.

6. Evidence and alert layer

The evidence layer stores non-sensitive diagnostic facts: correlation ID, allowed hostname, product ID, store ID, challenge type, provider task ID, elapsed time, final state, and the selector or response field used for verification. It should redact tokens, cookies, API keys, addresses, and any customer data. Alert only on meaningful state changes and require two observations when a single transient result could cause operational noise.

Prerequisites and Responsible-Use Controls

Before implementing CAPTCHA recovery, confirm that you have permission to automate the selected retail pages and that the collection purpose is documented. Respect contractual limits, applicable law, robots instructions where they apply to your use, and reasonable request rates. The Robots Exclusion Protocol describes standardized crawler directives; it is one signal within a broader authorization review, not a grant of access.

Use these prerequisites:

• An allowlisted retail hostname and a named business owner.
• A bounded list of product and store identifiers.
• A dedicated CapSolver API key stored outside source control.
• Python 3.11 or later with requests and Playwright installed.
• A permitted page in staging or production where the widget parameters can be observed lawfully.
• A verification contract for each availability state.
• A rate budget, task budget, and maximum run duration.

Secrets should come from a managed environment or secret store. The OWASP secrets management guidance supports separating credentials from application code, logs, and build artifacts. Do not place a real client key, cookie, solved token, or customer address in an article, prompt, screenshot, or troubleshooting ticket.

Detect reCAPTCHA Without Losing Store Context

The detector should run after every action that can trigger a verification page: initial navigation, store change, pickup panel open, pagination, and inventory refresh. Detection should return structured evidence instead of a simple boolean.

from dataclasses import dataclass

@dataclass(frozen=True)
class CaptchaEvidence:
    kind: str
    website_url: str
    website_key: str

async def detect_recaptcha_v2(page) -> CaptchaEvidence | None:
    frame = page.locator('iframe[src*="recaptcha"]')
    textarea = page.locator('textarea[name="g-recaptcha-response"]')
    if await frame.count() == 0 and await textarea.count() == 0:
        return None

    key = await page.locator('[data-sitekey]').first.get_attribute('data-sitekey')
    if not key:
        raise RuntimeError("reCAPTCHA detected without a readable site key")

    return CaptchaEvidence(
        kind="recaptcha_v2",
        website_url=page.url,
        website_key=key,
    )

The input is the already-open Playwright page. The operation checks two explicit widget signals and reads the page-provided site key. The output is None or a typed evidence object. The function stops with an error when a challenge is visible but the key cannot be confirmed; guessing a key or reusing one from another page would make the recovery unreliable.

Before calling the solver, validate that page.url still belongs to the allowlisted retail host and that the current run still holds the expected product and store identifiers. If a redirect leads to an account page, checkout, payment flow, or unexpected domain, stop and mark policy_denied. Technical capability does not provide permission to access private, restricted, sensitive, or unauthorized data.

Create and Poll One CapSolver Task

The official CapSolver reCAPTCHA v2 task guide documents ReCaptchaV2TaskProxyLess, websiteURL, and websiteKey. The createTask API returns a taskId; the getTaskResult API returns a terminal result. A successful reCAPTCHA v2 result includes solution.gRecaptchaResponse.

import os
import time
import requests

CAPSOLVER_API = "https://api.capsolver.com"

def solve_recaptcha_v2(website_url: str, website_key: str) -> str:
    client_key = os.environ["CAPSOLVER_API_KEY"]
    created = requests.post(
        f"{CAPSOLVER_API}/createTask",
        json={
            "clientKey": client_key,
            "task": {
                "type": "ReCaptchaV2TaskProxyLess",
                "websiteURL": website_url,
                "websiteKey": website_key,
            },
        },
        timeout=30,
    ).json()

    if created.get("errorId") or not created.get("taskId"):
        raise RuntimeError(created.get("errorDescription", "createTask failed"))

    task_id = created["taskId"]
    deadline = time.monotonic() + 120
    while time.monotonic() < deadline:
        result = requests.post(
            f"{CAPSOLVER_API}/getTaskResult",
            json={"clientKey": client_key, "taskId": task_id},
            timeout=30,
        ).json()

        if result.get("status") == "ready":
            token = result.get("solution", {}).get("gRecaptchaResponse")
            if not token:
                raise RuntimeError("ready result did not contain gRecaptchaResponse")
            return token
        if result.get("status") == "failed" or result.get("errorId"):
            raise RuntimeError(result.get("errorDescription", "CAPTCHA task failed"))
        time.sleep(3)

    raise TimeoutError("CAPTCHA task exceeded the 120-second deadline")

The function input is the verified page URL and site key. The operation creates exactly one task and polls only its taskId. The output is the documented token string. Stop conditions are explicit: task creation error, failed status, malformed ready response, network timeout, or the 120-second deadline. A transport retry may repeat the HTTP request for the same task result, but it must not silently create a series of new billable tasks.

Redeem Your CapSolver Bonus Code

Boost your automation budget instantly!
Use bonus code CAP26 when topping up your CapSolver account to get an extra 5% bonus on every recharge — with no limits.
Redeem it now in your CapSolver Dashboard

Apply the Token in the Same Browser Session

Token application is page-specific. On an authorized integration or controlled test page, write the token to the documented response field and invoke the page’s expected callback or submission path. Do not copy a token into a new browser, another product page, or a different store context.

async def apply_recaptcha_token(page, token: str) -> None:
    applied = await page.evaluate(
        """
        (token) => {
          const fields = [...document.querySelectorAll(
            'textarea[name="g-recaptcha-response"]'
          )];
          if (fields.length === 0) return false;
          for (const field of fields) {
            field.value = token;
            field.innerHTML = token;
            field.dispatchEvent(new Event('change', { bubbles: true }));
          }
          return true;
        }
        """,
        token,
    )
    if not applied:
        raise RuntimeError("reCAPTCHA response field disappeared before application")

This example is intentionally limited to the response field visible in the current page. Some implementations also require a documented callback. Inspect the page you own or are authorized to test and connect to its real integration contract. Never invent a callback name. After application, wait for the widget or verification route to change state, then resume the inventory action once.

Extract and Verify the Inventory Result

Inventory verification should bind the observed state to the requested product and store. A selector that says “available” is insufficient if the store label silently changed or the product page redirected to a variant.

from datetime import datetime, timezone

async def read_inventory(page, expected_sku: str, expected_store: str) -> dict:
    sku = (await page.locator('[data-product-sku]').first.get_attribute('data-product-sku'))
    store = (await page.locator('[data-store-id]').first.get_attribute('data-store-id'))
    label = (await page.locator('[data-inventory-status]').first.inner_text()).strip()

    if sku != expected_sku or store != expected_store:
        raise RuntimeError("product or store context changed during collection")

    normalized = {
        "In stock": "in_stock",
        "Limited stock": "limited",
        "Out of stock": "out_of_stock",
    }.get(label)
    if not normalized:
        raise RuntimeError(f"unrecognized inventory label: {label!r}")

    return {
        "product_id": sku,
        "store_id": store,
        "availability": normalized,
        "observed_at": datetime.now(timezone.utc).isoformat(),
        "verification": "product-store-status",
    }

The selectors are placeholders for a page you control or have permission to automate. The input is the current page plus expected identifiers. The operation checks identity before normalizing the label. The output is a verified inventory record. The function stops on missing elements, changed identifiers, or an unfamiliar status. Those stops protect the downstream system from a common failure: turning a changed page layout into a false out-of-stock event.

For production, add a second evidence channel when available. Examples include a structured product object, an XHR response initiated by the page, a pickup panel store label, or a cart-eligibility check permitted by your agreement. The collector should require agreement between identity and availability evidence, not duplicate requests simply to increase confidence.

Assemble the End-to-End State Machine

A predictable state machine makes the workflow observable and prevents recursive recovery.

SCOPED
  -> OPEN_PRODUCT
  -> CONFIRM_STORE
  -> DETECT_CHECKPOINT
       -> no challenge: READ_INVENTORY
       -> reCAPTCHA v2: CREATE_ONE_TASK
            -> ready: APPLY_IN_SAME_SESSION
            -> failed/deadline: NEEDS_REVIEW
       -> unsupported/ambiguous: NEEDS_REVIEW
  -> REPLAY_INVENTORY_ACTION_ONCE
  -> VERIFY_PRODUCT + STORE + AVAILABILITY
       -> valid: WRITE_RECORD
       -> repeated challenge: NEEDS_REVIEW
       -> changed context: POLICY_DENIED

The job should carry one correlation ID through every state. Record the transition time and outcome, but never log the solved token or client key. The CapSolver errors and troubleshooting FAQ can help operators distinguish provider failures from browser and application failures.

Error Recovery by Failure Class

Detection failure

Symptoms include missing site key, an unexpected verification page, or a widget family the adapter does not recognize. Capture a redacted screenshot and the top-level host, then stop. Do not send guessed parameters or treat every iframe as reCAPTCHA.

Provider task failure

A nonzero errorId, failed status, malformed ready result, or polling deadline is a provider-boundary failure. Preserve the taskId, documented error description, elapsed time, and correlation ID. Retry only if the error class is explicitly transient and the remaining task budget allows it.

Token application failure

The response field may disappear because the page navigated, rerendered, or changed store context. Do not apply the token to a replacement page automatically. Re-run scope and context checks, then either restart the single product check from a clean state or route it to review.

Application rejection

The provider can return a ready task while the page rejects the solution because the session, page timing, widget metadata, or callback path does not match. Classify this as an application failure. One controlled replay is enough. If the CAPTCHA repeats, stop instead of starting a loop.

Inventory ambiguity

If the page loads but stock evidence is absent or conflicting, record unknown, not out_of_stock. Alert when ambiguity crosses a threshold for a retailer or template, because it often indicates a markup change rather than a real inventory event.

The HTTP semantics specification helps distinguish transport status from application meaning. A 200 OK only describes the HTTP response; it does not prove that a location was selected, a CAPTCHA was accepted, or inventory was returned.

Data Quality and Operational Controls

Good inventory automation minimizes collection while maximizing confidence. Poll at a frequency justified by the business need and permitted by the source. Cache stable product metadata. Schedule store checks with jitter within the approved window, not bursty parallel requests. Use conditional fetches where the source supports them, and stop a run when the challenge rate or application error rate rises unexpectedly.

Track these metrics separately:

• Page load success rate.
• CAPTCHA appearance rate by retailer and route.
• CAPTCHA task ready, failed, and deadline rates.
• Application acceptance rate after a ready task.
• Inventory extraction success rate.
• Unknown-state rate and repeated-challenge rate.
• Median and percentile latency by workflow state.
• Records suppressed because product or store identity changed.

Do not optimize only for solver completion. A high ready rate with a low application acceptance rate indicates an integration problem. A high acceptance rate with a rising unknown inventory rate indicates an extractor or page-template problem. The metrics should point operators to the layer that owns the failure.

When data feeds alerts, deduplicate repeated observations and define a stability rule. For example, notify on out_of_stock only after two valid checks separated by the normal collection interval, while an in_stock transition may require a fresh successful observation. Keep the rule visible in configuration so business teams can audit why an alert was sent.

Pre-Production Test Plan

Test the workflow against pages you own or are explicitly authorized to automate. Use fixtures for ordinary inventory states and a controlled CAPTCHA integration for recovery tests.

1. Verify that an allowlisted product with no challenge returns the correct SKU, store, and availability.
2. Verify that reCAPTCHA detection returns the exact URL and site key shown by the page.
3. Mock createTask failure and confirm that no inventory record is written.
4. Mock processing results until the deadline and confirm that only one task was created.
5. Return a ready result without gRecaptchaResponse and confirm schema validation fails.
6. Change the store ID during recovery and confirm the workflow stops.
7. Repeat the CAPTCHA after token application and confirm the job enters needs_review.
8. Remove the inventory selector and confirm the result is unknown, not out_of_stock.
9. Inspect logs to ensure keys, cookies, addresses, and tokens are redacted.
10. Confirm that an unapproved hostname is denied before a network call to CapSolver.

The guide to choosing a CAPTCHA-solving API provides additional evaluation criteria, but the acceptance test for this user case remains business-specific: the system must return the right product, right store, and right availability after a bounded recovery.

Conclusion

Store inventory data collection becomes dependable when CAPTCHA handling is treated as a controlled state within a verified retail workflow. Preserve the product and store context, create one documented task, apply the solution in the same authorized session, replay the inventory action once, and publish data only after identity and availability checks pass. CapSolver provides the CAPTCHA task infrastructure; your collector remains responsible for scope, rate limits, data quality, and stop conditions.

FAQ

Q: What is the minimum output for store inventory data collection?

The minimum trustworthy output contains a canonical product ID, store or region ID, normalized availability, observation time, and the evidence used to verify the state. A blank page or failed checkpoint must never be converted into out-of-stock.

Q: Why must the same browser session be preserved during CAPTCHA recovery?

The same session preserves the page, cookies, product selection, store selection, and widget lifecycle associated with the checkpoint. Moving the result to another context can cause rejection or attach the observation to the wrong store.

Q: How many CAPTCHA retries should an inventory job make?

Use a small explicit budget: one task and one controlled application replay is a practical default. A repeated challenge should enter needs_review so the integration can be inspected without an expensive or disruptive loop.

Q: Does a ready CapSolver task prove that inventory was collected?

No. A ready task proves only that the provider returned a solution. The browser must accept it, and the application must still return a recognized product, store, and inventory state.

Q: Can this workflow collect inventory from private retailer portals?

Only when the portal owner has explicitly authorized that automation and the workflow complies with the applicable agreement and law. Technical capability does not grant permission to access private, restricted, sensitive, or unauthorized data.