Hitting a CAPTCHA while performing web automation is something that leaves many people feeling overwhelmed. These CAPTCHAs, specifically designed to differentiate between human users and automated scripts, are undoubtedly a huge challenge to using Selenium for web crawling or automation tasks. 2024's complete guide will provide you with a comprehensive solution to help you efficiently tackle these obstacles, most notably the two more common types of CAPTCHA, funcaptcha and recaptcha, but of course we also need to incorporate a third-party CAPTCHAs solution tool, CapSolver.
Table of Contents
- What is CAPTCHA
- CAPTCHA Examples
- Prerequisites
- Method 1: Solving CAPTCHA via Capsolver API
- Understanding HTML Forms
- Handling Hidden Textarea Elements
- Getting the Token
- Using Capsolver Python SDK to Get the Token
- Using the Token in Selenium
- Method 2: Solving CAPTCHA with Capsolver Extension
- Downloading and Repackaging Capsolver Extension
- Using Capsolver Extension in Selenium
Bonus Code
Claim your Bonus Code for top captcha solutions; CapSolver: WEBS. After redeeming it, you will get an extra 5% bonus after each recharge, Unlimited
What is CAPTCHA
CAPTCHA requires users to perform specific tasks, such as entering the text displayed in an image or clicking on images from a set that match certain criteria.These tasks are designed to verify whether the user is a human or a robot. Due to the dynamic nature of these tasks, they can only be successfully completed through human cognition and correct interpretation of information—areas where artificial intelligence often struggles.
Some CAPTCHA variants also support visually impaired individuals by generating audio instead of images.
CAPTCHA Examples
Google's open-source CAPTCHA widget, reCAPTCHA, is widely used because it supports various major screen readers such as JAWS and NVDA for IE, Edge, or Chrome on Windows OS, ChromeVox for Chrome OS, and VoiceOver for Safari and Chrome on Mac OS.
Prerequisites
- Google Chrome: Install the latest version of Chrome, as we will be interacting with Chrome through code.
- Python: Ensure you have Python installed, and that the version is 3 or above. Versions below 3 are no longer recommended.
- Selenium: The Python library for the automation tool Selenium.
- Capsolver Python SDK: The official Capsolver Python SDK, which allows easy integration with Capsolver.
- Capsolver Extension: The official Capsolver Chrome extension, which can automatically solve various challenges for you.
Method 1: Solving CAPTCHA via Capsolver API
We'll use this demo page as an example to demonstrate how to solve reCAPTCHA in Python Selenium using the Capsolver API.
Understanding HTML Forms
Before we start, we need to understand the basics of HTML forms. Observe this page and open the developer tools. Manually solve the reCAPTCHA and then click the Submit button. You will see a POST request sent, submitting three fields: ex-a, ex-b, and g-recaptcha-response, as shown in the image below:
These three fields correspond to two input elements and one textarea element under the form in the initial HTML source code, as shown in the image below:
Our approach is to solve the reCAPTCHA using the Capsolver API, obtain the token, input it into the textarea element, and then click the submit button to submit the form.
Handling Hidden Textarea Elements
When inputting the token into the textarea element, note that the textarea element on the webpage has a CSS style display: none, which means the textarea is not visible. In this case, if you try to input content into the textarea element directly in Selenium, it will throw an error:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable, because the textarea element is not interactable at this moment. To solve this, we need to set the CSS style of the textarea element to display: block. The specific operation method will be reflected in the code later.
Getting the Token
Using the Capsolver API requires us to provide the websiteKey, which can be found by searching for the keyword data-sitekey in the page source code:
Using Capsolver Python SDK to Get the Token
Here's how to get the token using the Capsolver Python SDK:
import capsolver
capsolver.api_key = "your api key"
solution = capsolver.solve({
"type": "ReCaptchaV2TaskProxyLess",
"websiteKey": "6LfW6wATAAAAAHLqO2pb8bDBahxlMxNdo9g947u9",
"websiteURL": "https://recaptcha-demo.appspot.com/recaptcha-v2-checkbox.php",
})
token = solution["gRecaptchaResponse"]
print(token)Using the Token in Selenium
Next, we need to use the token with Selenium. There are two key operations in Selenium:
- Make the textarea element visible so that it can be interacted with and the token can be input into it.
- Locate the submit button and click it to submit the form.
These operations involve locating elements and interacting with elements. If you're not familiar with Selenium, you can refer to Web Scraping with Selenium and Python | Solving Captcha When Web Scraping
Combining with the Capsolver API, the complete code is as follows:
import capsolver
from selenium import webdriver
from selenium.webdriver.common.by import By
# Initialize Chrome Options object and access target website
chrome_options = webdriver.ChromeOptions()
driver = webdriver.Chrome(options=chrome_options)
url = "https://recaptcha-demo.appspot.com/recaptcha-v2-checkbox.php"
driver.get(url)
# Calling the Capsolver API to solve ReCaptcha
capsolver.api_key = "your api key"
solution = capsolver.solve({
"type": "ReCaptchaV2TaskProxyLess",
"websiteKey": "6LfW6wATAAAAAHLqO2pb8bDBahxlMxNdo9g947u9",
"websiteURL": url,
})
token = solution["gRecaptchaResponse"]
print(f"Token returned by capsolver: {token}")
# Change the display style property of textarea to block to make it visible
driver.execute_script("document.getElementById('g-recaptcha-response').style.display = 'block';")
# Simulate inputting tokens into textarea
textarea = driver.find_element(By.ID, "g-recaptcha-response")
textarea.send_keys(token)
# Simulate clicking and submit the form
submit_btn = driver.find_element(By.CSS_SELECTOR, "button[type='submit']")
submit_btn.click()
input("Press any key to exit.")
driver.close()The above code demonstrates how to solve reCAPTCHA using the Capsolver API in Python Selenium. After successfully solving it, you will see the following page:
Method 2: Solving CAPTCHA with Capsolver Extension
Method 1 involved solving CAPTCHA through the Capsolver API, which includes many complex operations. If you are looking for a simpler and more convenient way to solve CAPTCHA, then Capsolver Extension is your best choice. It can easily integrate into browsers like Chrome and Firefox. Capsolver Extension can automatically recognize and solve various CAPTCHA challenges in just a few seconds, without any human intervention, allowing you to enjoy Capsolver's CAPTCHA solving service without writing any code.
We take FunCaptcha as an example, with the target webpage
Downloading and Repackaging Capsolver Extension
Since we are using Capsolver Extension in Selenium, we need to download the zip file from Capsolver's official GitHub. Using Capsolver Extension requires you to enter your API key, as shown below:
Interacting with browser extensions in Selenium can be quite cumbersome, so we can pre-fill the API key in the extension's configuration file and then load it directly in Selenium. Extract the Capsolver Extension zip file we downloaded, and enter your API key in the \assets\config.js file, as shown below:
Next, we use Chrome's built-in extension packaging feature to repackage the Capsolver Extension. Note that you cannot simply compress the folder back into a zip file; such an extension would be unusable. In Chrome, visit chrome://extensions/, enable developer mode, select the Pack Extension option, and import the entire Capsolver extension folder. After repackaging, you will get a .crx file.
Using Capsolver Extension in Selenium
Use the add_extension method to load the newly repackaged .crx format Capsolver Extension. The sample code is as follows:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_extension(r"C:\path\to\CapSolver.Browser.Extension-chrome-v1.14.0.crx")
driver = webdriver.Chrome(options=chrome_options)
driver.get("https://iframe.arkoselabs.com/3117BF26-4762-4F5A-8ED9-A85E69209A46/index.html")
input("Press any key to exit.")
driver.close()Run the code, and you will see that Capsolver Extension automatically solves the FunCaptcha challenge:
Conclusion
Whether using the Capsolver API or the Capsolver Extension, you can perfectly solve CAPTCHA in Python Selenium. If you have any questions, refer to the Capsolver documentation for more useful information.
