Do you remove spammy backlinks and domains from the index?
Our database contains over 2.8Tn live backinks and over 343.6B live indexed pages. We ensure data integrity and quality by leaving out spammy referring domains and networks. For instance, when our crawler sees a significant number of domains or subdomains with similar content and suspicious linking patterns, we employ a custom minimum quality threshold to avoid indexing domains and subdomains that do not pass it. Such filtering mechanisms prevent the proliferation of spammy domains in our index.
We also continuously crawl the web to collect the freshest information and update our index. New backlinks are made available every second via our Backlinks API, outdated backlinks are removed from our index in up to one year.
It’s worth noting, that our index architecture is designed to provide access to historical data separately from fresh stats. You can obtain historical backlink data dating back to the beginning of 2019 – that’s when we started developing our index.
Finally, in both Backlinks API and Domain Backlink Stats Database, we offer a proprietary Spam Score metric designed to help our users reduce the amount of manual work. It sheds light on the spam level of a page or backlink, a group of pages, or even an entire domain, using a scale from 0% to 100%. To calculate this Score, we have an algorithm in place that automatically evaluates each scanned page based on 18 meticulously selected signals that include domain name length, the ratio of external links to internal links, and protocol usage (HTTP or HTTPS). Learn more.