Earlier, when you set
1 and specified a
start_url to crawl only one page, our system still had to make numerous additional requests to obtain data on sitewide checks, and thus the crawl was taking longer to complete.
To minimize the crawl time and our system load, we have made the following changes:
1 If you set
max_crawl_pages to 1 and do not specify
start_url or set a homepage in it, the following sitewide checks will be disabled:
2 If you set
1 and specify
start_url other than a homepage, all sitewide checks will be disabled.
Don’t hesitate to contact our support team if you have any quesitons.