You can check out this session on September 13, 2019

Speaker

François Goube

François Goube

OnCrawl
@francoisgoube

About this Session

Explore all the ways search engines are exploiting your robots.txt. Francois will help you understand how Google is using the robots.txt file to understand your website. Spam detection, optimizing crawl budget, prioritizing crawling queues, we will dive into the most strategic use cases to show how to leverage the use of robots.txt.

Sign up to the mailing list for more info about the event
How we use the data in this form.