Custom Task

A Custom Task is a user-defined scraping configuration that lets you manually build a crawler tailored to any website’s structure and data requirements.

Definition

A Custom Task is a flexible scraping setup that you construct from scratch rather than using a pre-built template. It allows you to navigate to any target site, interact with elements, and specify exactly what data to extract and how. This mode is especially useful when no ready-made template exists for a particular site or workflow. Through an interactive interface, users can select page elements, define actions, and shape the extraction logic step by step. Custom Tasks support complex scenarios like logging in, clicking through menus, and handling dynamic content.

Pros

• Enables scraping from virtually any website regardless of structure.
• Offers fine-grained control over navigation and data selection.
• Supports advanced interactions like clicks, pagination, and login flows.
• Doesn’t require coding experience thanks to a visual interface.
• Adaptable to changes in site layout or extraction needs.

Cons

• More time-consuming to set up compared to using templates.
• Requires a deeper understanding of site structure and elements.
• Complex tasks may be harder to maintain over time.
• Manual setup increases the risk of configuration errors.
• Not ideal for users who prefer fully automated templates.

Use Cases

• Extracting data from niche or custom websites without templates.
• Scraping content behind login forms or interactive elements.
• Building crawlers that navigate complex menus or filters.
• Handling dynamic sites with JavaScript-rendered content.
• Creating bespoke workflows for specific business data needs.