CAPSOLVER
Blog
How to Solve Cloudflare in 2024: Solve Cloudflare Turnstile and Challenge By Using CapSolver

How to Solve Cloudflare in 2024: Solve Cloudflare Turnstile and Challenge By Using CapSolver

Logo of CapSolver

Ethan Collins

Pattern Recognition Specialist

07-May-2024

How to Solve Cloudflare in 2024: Solve Cloudflare Turnstile and Challenge By Using CapSolver

Cloudflare offers robust protection measures, including the Cloudflare Challenge and Turnstile CAPTCHA, to secure websites from malicious bots and automated scrapers. Understanding and overcoming these challenges can be crucial for those needing access beyond these protective barriers for legitimate purposes.

Understanding Cloudflare's Security Measures

Cloudflare employs various techniques to detect and mitigate unwanted traffic. Among these are the Cloudflare Challenge, which often involves JavaScript challenges or CAPTCHA tests, and Turnstile, a more user-friendly CAPTCHA that minimizes user interaction. These systems are designed to distinguish between human users and automated bots effectively.

Strategies for Solving Cloudflare Challenges

  1. Automated Browsers:

    • Tools like Selenium or Puppeteer can simulate real user interactions. To mask their automated nature, plugins such as puppeteer-extra-plugin-stealth can be used to pass the Cloudflare Challenge by emulating natural browser behavior.
  2. High-Quality Proxies:

    • Utilizing residential proxies that rotate IP addresses can help mimic genuine user access patterns, thereby reducing the likelihood of being blocked by Cloudflare's security measures.
  3. CAPTCHA Solving Services:

    • For challenges / captchas that require solving

Detailed Guide to solving Cloudflare Challenge

First, let's take a look of how cloudflare challenge looks like:

Sometimes, this page could have turnstile

If have a similar page where you must wait until the verification is completed and usually have additional checks with turnstile, then it's cloudflare challenge.
There are some requeriments for solve this type of challenge:

  • Token returned in the response of the method getTaskResult is the value of the cookie cf_clearance that you will need to create.
  • Must use the same user-agent that the method getTaskResult return
  • Must use the same proxy IP used for solve the challenge
  • Must use the cookies of the response
  • Must use the headers of the response
  • Use TLS chrome 120 version

Example of how to solve cloudflare 5s interface with Python

python Copy
# -*- coding: utf-8 -*-
import time
import requests
import tls_client

API_KEY = ""  # TODO: your api key
page_url = ''  # TODO: your page url 'https://example.com
proxy = ""  # TODO: your proxy


def call_capsolver():
    data = {
        "clientKey": API_KEY,
        "task": {
            "type": 'AntiCloudflareTask',
            "websiteURL": page_url,
            "proxy": proxy,
        }
    }
    uri = 'https://api.capsolver.com/createTask'
    res = requests.post(uri, json=data)
    resp = res.json()
    task_id = resp.get('taskId')
    if not task_id:
        print("no get taskId:", res.text)
        return
    print('created taskId:', task_id)

    while True:
        time.sleep(1)
        data = {
            "clientKey": API_KEY,
            "taskId": task_id
        }
        response = requests.post('https://api.capsolver.com/getTaskResult', json=data)
        resp = response.json()
        status = resp.get('status', '')
        if status == "ready":
            print("successfully => ", response.text)
            return resp.get('solution')
        if status == "failed" or resp.get("errorId"):
            print("failed! => ", response.text)
            return


def request_site(solution):
    # TODO if you want to use tls_client
    session = tls_client.Session(
         client_identifier="chrome_120",
        random_tls_extension_order=True
    )
    return session.get(
        page_url,
        headers=solution.get('headers'),
        cookies=solution.get('cookies'),
        proxy=proxy,
        allow_redirects=True,
    )


def main():
    # first request:
    res = request_site({})
    print('1. response status code:', res.status_code)
    if res.status_code != 403:
        print("your proxy is good and didn't get the cloudflare challenge")
        return
    elif 'window._cf_chl_opt' not in res.text:
        print('==== proxy blocked ==== ')
        return

    # call capSolver:
    solution = call_capsolver()
    if not solution:
        return

    # second request (verify solution):
    res = request_site(solution)
    print('2. response status code:', res.status_code)
    if res.status_code == 200:
        ##print("2. response text:\n", res.text)
        print("successfully passed the cloudflare challenge")


if __name__ == '__main__':
    main()

Detailed Guide to solving Cloudflare Turnstile Captcha

Cloudflare offer 3 types of cloudflare turnstile captcha

  • Managed challenge

  • Non-interactive challenge

  • Invisible challenge
    not visible, you can check on the network / scripts loaded and see if turnstile is used

Turnstile CAPTCHA is Cloudflare’s replacement for traditional CAPTCHA challenges, focusing on user experience and security. Here’s how to approach solving it:

  1. Understand the Challenge:

    • Turnstile CAPTCHA might not always appear visibly; sometimes, it operates in the background, analyzing user interactions to verify authenticity.
  2. Integration of CAPTCHA Solving APIs:

    • Some CAPTCHA solving services have begun offering solutions for Turnstile by analyzing network traffic and interaction patterns. Utilize these APIs to submit Turnstile challenges for resolution.
  3. Maintain Browser Consistency:

    • Ensure that your automated solutions maintain consistent browser signatures, as Turnstile can analyze these to detect automation.
  4. Adapt to JavaScript Challenges:

    • Cloudflare may deploy JavaScript computations to test the browser's environment. Tools like Puppeteer can be scripted to solve these challenges dynamically by executing JavaScript code as a regular browser would.

Example of how to solve cloudflare turnstile captcha with Python

python Copy
import time
from curl_cffi import requests

CAPSOLVER_API_KEY = "Your CAPSOLVER.COM API KEY"
PAGE_URL = ""
WEBSITE_KEY = ""

def solvecf(metadata_action=None, metadata_cdata=None):
    url = "https://api.capsolver.com/createTask"
    task = {
        "type": "AntiTurnstileTaskProxyLess",
        "websiteURL": PAGE_URL,
        "websiteKey": WEBSITE_KEY,
    }
    if metadata_action or metadata_cdata:
        task["metadata"] = {}
        if metadata_action:
            task["metadata"]["action"] = metadata_action
        if metadata_cdata:
            task["metadata"]["cdata"] = metadata_cdata
    data = {
        "clientKey": CAPSOLVER_API_KEY,
        "task": task
    }
    response_data = requests.post(url, json=data).json()
    print(response_data)
    return response_data['taskId']


def solutionGet(taskId):
    url = "https://api.capsolver.com/getTaskResult"
    status = ""
    while status != "ready":
        data = {"clientKey": CAPSOLVER_API_KEY, "taskId": taskId}
        response_data = requests.post(url, json=data).json()
        print(response_data)
        status = response_data.get('status', '')
        print(status)
        if status == "ready":
            return response_data['solution']

        time.sleep(2)


def main():
    start_time = time.time()
    
    taskId = solvecf()
    solution = solutionGet(taskId)
    if solution:
        user_agent = solution['userAgent']
        token = solution['token']

    print("User_Agent:", user_agent)
    print("Solved Turnstile Captcha, token:", token)

    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Time to solve the captcha: {elapsed_time} seconds")

if __name__ == "__main__":
    main()

Example of how to solve cloudflare turnstile captcha with Nodejs

js Copy
const axios = require('axios');

const CAPSOLVER_API_KEY = "";
const PAGE_URL = "";
const WEBSITE_KEY = "";

async function solvecf(metadata_action = null, metadata_cdata = null) {
    const url = "https://api.capsolver.com/createTask";
    const task = {
        type: "AntiTurnstileTaskProxyLess",
        websiteURL: PAGE_URL,
        websiteKey: WEBSITE_KEY,
    };
    if (metadata_action || metadata_cdata) {
        task.metadata = {};
        if (metadata_action) {
            task.metadata.action = metadata_action;
        }
        if (metadata_cdata) {
            task.metadata.cdata = metadata_cdata;
        }
    }
    const data = {
        clientKey: CAPSOLVER_API_KEY,
        task: task
    };
    const response = await axios.post(url, data);
    console.log(response.data);
    return response.data.taskId;
}

async function solutionGet(taskId) {
    const url = "https://api.capsolver.com/getTaskResult";
    let status = "";
    while (status !== "ready") {
        const data = { clientKey: CAPSOLVER_API_KEY, taskId: taskId };
        const response = await axios.post(url, data);
        console.log(response.data);
        status = response.data.status;
        console.log(status);
        if (status === "ready") {
            return response.data.solution;
        }
        await new Promise(resolve => setTimeout(resolve, 2000));
    }
}

async function main() {
    const start_time = Date.now();
    
    const taskId = await solvecf();
    const solution = await solutionGet(taskId);
    if (solution) {
        const user_agent = solution.userAgent;
        const token = solution.token;

        console.log("User_Agent:", user_agent);
        console.log("Solved Turnstile Captcha, token:", token);
    }

    const end_time = Date.now();
    const elapsed_time = (end_time - start_time) / 1000;
    console.log(`Time to solve the captcha: ${elapsed_time} seconds`);
}

main().catch(console.error);

Compliance Disclaimer: The information provided on this blog is for informational purposes only. CapSolver is committed to compliance with all applicable laws and regulations. The use of the CapSolver network for illegal, fraudulent, or abusive activities is strictly prohibited and will be investigated. Our captcha-solving solutions enhance user experience while ensuring 100% compliance in helping solve captcha difficulties during public data crawling. We encourage responsible use of our services. For more information, please visit our Terms of Service and Privacy Policy.

More

Cloudflare TLS Fingerprinting: What It Is and How to Solve It
Cloudflare TLS Fingerprinting: What It Is and How to Solve It

Learn about Cloudflare's use of TLS fingerprinting for security, how it detects and blocks bots, and explore effective methods to solve it for web scraping and automated browsing tasks.

Cloudflare
Logo of CapSolver

Lucas Mitchell

28-Feb-2025

How to Extract Data from a Cloudflare-Protected Website
How to Extract Data from a Cloudflare-Protected Website

In this guide, we'll explore ethical and effective techniques to extract data from Cloudflare-protected websites.

Cloudflare
Logo of CapSolver

Lucas Mitchell

20-Feb-2025

How to Fix Cloudflare Errors 1006, 1007, and 1008 Quickly
How to Fix Cloudflare Errors 1006, 1007, and 1008 Quickly

Cloudflare errors 1006, 1007, and 1008 can block your access due to suspicious or automated traffic. Learn quick fixes using premium proxies, user agent rotation, human behavior simulation, and IP address changes to overcome these roadblocks for smooth web scraping.

Cloudflare
Logo of CapSolver

Ethan Collins

05-Feb-2025

How to Bypass Cloudflare Challenge While Web Scraping in 2025
How to Bypass Cloudflare Challenge While Web Scraping in 2025

Learn how to bypass Cloudflare Challenge and Turnstile in 2025 for seamless web scraping. Discover Capsolver integration, TLS fingerprinting tips, and fixes for common errors to avoid CAPTCHA hell. Save time and scale your data extraction.

Cloudflare
Logo of CapSolver

Aloísio Vítor

23-Jan-2025

How to Solve Cloudflare Turnstile CAPTCHA by Extension
How to Solve Cloudflare Turnstile CAPTCHA by Extension

Learn how to bypass Cloudflare Turnstile CAPTCHA with Capsolver’s extension. Install guides for Chrome, Firefox, and automation tools like Puppeteer.

Cloudflare
Logo of CapSolver

Adélia Cruz

23-Jan-2025

How to Solve Cloudflare by Using Python and Go in 2025
How to Solve Cloudflare by Using Python and Go in 2025

Will share insights on what Cloudflare Turnstile is, using Python and Go for these tasks, whether Turnstile can detect Python scrapers, and how to effectively it using solutions like CapSolver.

Cloudflare
Logo of CapSolver

Lucas Mitchell

05-Nov-2024