How to crawl a mobile version of a website?
The opportunity to crawl mobile versions of websites will enrich any website audit tool.
With DataForSEO OnPage API, your tool will be able to scan mobile versions of websites, providing essential on-page metrics.
To crawl a mobile version of a website, add the custom_user_agent field to the On-Page API Task POST body and specify the user agent for a web browser that runs on the iOS or Android operating system.
If you want to acquire Core Web Vitals and browser-specific metrics (such as loading speed, for example), you should also add the enable_browser_rendering field and set it to true.
Note: when enable_browser_rendering is set to true, additional charges apply.
Also, note that using enable_browser_rendering requires setting enable_javascript and load_resources parameters to true, which are also paid parameters. For more information on the cost of tasks with additional parameters, please refer to this help article.
You can structure your POST request as in the example below:
[
{
"target": "dataforseo.com",
"max_crawl_pages": 10,
"custom_user_agent": "Mozilla/5.0 (Linux; Android 12) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.71 Mobile Safari/537.36",
"enable_browser_rendering":"true",
"enable_javascript": "true",
"load_resources": "true"
}
]