SERPSTATBOT FAQ

What is the purpose of serpstatbot?

Our bot constantly crawls the web to add new links to our database and monitor the link that it found earlier. That's why we can offer our users the most comprehensive data. This data collected by serpstatbot helps digital marketers in planning and monitoring their online marketing campaigns.

What happens to the crawled data?

Crawled data is added to the public backlinks index that we maintain.

How do you handle 404 or 301 pages?

We collect historical data to make sure that no temporary change has an impact on your site. If the links to these pages still exist, serpstatbot will continue to find and follow them. You can check Google 404 policy to find more information.

The bot crawls links with rel=nofollow

Google claims that rel=nofollow links don’t influence the Pagerank, so the crawler still visits the target page. If you don’t want serpstatbot to crawl such links, then use the robots.txt file to disallow the target page. You can find more information in Wikipedia Nofollow.

How can I block serpstatbot?

Serpstatbot addresses to the robots.txt standard. If you don’t want the bot to crawl your website then add the following text to your robots.txt:

User-agent: serpstatbot

Disallow: /

Please always ensure the bot can get robots.txt file itself. If not, then it will crawl your site by default.

If you think that serpstatbot doesn’t follow your robots.txt commands, please get in touch with us via email: serpstatbot@serpstatbot.com. Please attach the URL to your website and log entries that show bot trying to retrieve pages that it was not supposed to.

What commands in robots.txt does serpstatbot support?

Serpstatbot supports the following additions to robots.txt:

Why doesn’t my robots.txt block work on serpstatbot?

There are several reasons:

How can I slow down serpstatbot?

You can slow down bot by adding the following settings to your robots.txt file:

User-Agent: serpstatbot

Crawl-Delay: 5

Crawl-Delay must be an integer: it stands for the number of seconds to wait between requests. Serpstatbot makes a delay of up to 20 seconds between requests to your site. Please note that your website can be scanned from several serpstatbots at the same time. High Crawl-Delay should minimize the impact on your site. This parameter will also be valid if it has been used for the * wildcard character.

If our bot finds out that you have used Crawl-Delay for any other bot, then it will automatically slow down the crawling process.

What is the current version of serpstatbot?

Тут впишем версию и дату релиза

How do I verify that requests are from you?

Unfortunately, we can’t restrict our bots to a limited number of IP addresses. However, we can send a pre-made string of identifiers with all requests to your site. This can be a part of the http or https headers in the CRAWLER-IDENT field or a part of the User-Agent line. We won’t forward this string to anyone else and send it to another domain or subdomain, so you will know that requests containing this string come from our network. To take advantage of this opportunity, please contact serpstatbot@serpstatbot.com with information about your website and the identifier you want to send. We can also generate a random identifier string for you.

Send article

Save the link and read the article later.
Choose where to send the link to the article.

email_image Mail
pocket_image Pocket
flipboard Flipboard
facebook_image Messenger
telegram_image Telegram

Поделитесь статьей с вашими друзьями

Войти Регистрация

Вы исчерпали лимит запросов.

Or email
Забыли пароль?
Or email
Back To Login

Don’t worry! Just fill in your email and we’ll send over your password.

Are you sure?

Awesome!

To complete your registration you need to enter your phone number

Back

We sent confirmation code to your phone number

Your phone Resend code queriesLeft

smthWentWrong

contactSupport
telegramVerification telegramBotLink
Choose the project to view data on it

Персональная презентация сервиса

Serpstat is all about saving time, and we want to save yours! One of our specialists will contact you and discuss options going forward.

These may include a personal demonstration, a trial period, comprehensive training articles & webinar recordings, and custom advice from a Serpstat specialist. It is our goal to make you feel comfortable while using Serpstat.

Name
Email
Phone
We are glad of your comment

Upgrade your plan

Export is not available for your account. Please upgrade to Plan A or higher to get access to the tool. Learn more.

Sign Up Free

Спасибо, мы с вами свяжемся в ближайшее время

Invite
Просмотр Редактирование

E-mail
Message
Optional
E-mail
Message
Optional

У вас закончились лимиты

Вы достигли лимита на количество созданных проектов и больше не можете создавать новые проекты. Увеличьте лимиты или удалите существующие проекты.

Я хочу больше лимитов

Christmas is a time for miracles.

You are almost on the finish line of our Christmas quest. The last brick of your lego-promocode is left on the way ap 55% discount.

Did not find previous lego-bricks? Fill the form anyway.

Name
Email
Phone

Missing translations for "English" English | Русский | Українська | German

  • Такого пользователя...add
  • All-in-Oneadd
  • Блогadd
  • Поддержкаadd
  • This site uses cookies and other tracking...add
  • Accept and continueadd
  • Руководствоadd
  • Попробовать бесплатноadd
  • Pricesadd
  • Pressadd
  • Clients Serpstatadd
  • Customer Successadd
  • Communityadd
  • Member Perksadd
  • Affiliate Programadd
  • Webinarsadd
  • PPCadd
  • Toolsadd
  • Site auditadd
  • SERP crawlingadd
  • Languagesadd
  • in 2013. From 2015 - fully independent.add
  • Save the link and read the article later.add
  • Email is Emptyadd
  • The article is sent to you by mailadd
  • Регистрацияadd
  • Wrong emailadd
  • Passwordadd
  • Забыли пароль?add
  • I agree to Serpstat'sadd
  • Don’t worry! Just fill in your email and...add
  • Confirmadd
  • To complete your registration you need to...add
  • Backadd
  • Resend codeadd
  • contactSupportadd
  • Code is invalidadd
  • Персональная...add
  • Nameadd
  • Optionaladd
  • See Serpstat in actionadd
  • Upgrade your planadd
  • Upgrade planadd
  • Просмотрadd
  • Messageadd
  • Addadd
  • Select Alladd
  • У вас закончились лимитыadd
  • Christmas is a time for miracles.add
  • Write your discount codeadd
  • Successadd
  • Erroradd
  • Loading... Wait for a while.add
  • Некоторые функции...add
  • SEO platformadd
  • Тарифыadd
  • Улучшить планadd
  • By using the website, you agree to ouradd
  • Входadd
  • Language:add
  • or see our plans & pricesadd
  • About usadd
  • Bannersadd
  • Help centeradd
  • Knowledge baseadd
  • Blogadd
  • Experts We Loveadd
  • Discountsadd
  • Rankadd
  • Legal infoadd
  • Rank trackeradd
  • Competitor analysisadd
  • Keywords clusteringadd
  • Download pluginadd
  • Mailadd
  • Choose where to send the link to the article.add
  • Not valid emailadd
  • Поделитесь статьей с...add
  • Вы исчерпали лимит...add
  • Email Addressadd
  • Sign Inadd
  • Wrong phoneadd
  • Sign Up Freeadd
  • Send Emailadd
  • Canceladd
  • Telephone is invalidadd
  • We sent confirmation code to your phone...add
  • queriesLeftadd
  • telegramVerificationadd
  • Confirmation codeadd
  • Serpstat is all about saving time, and we...add
  • Emailadd
  • Textadd
  • Choose planadd
  • Export is not available for your account....add
  • Спасибо, мы с вами...add
  • Редактированиеadd
  • Optionaladd
  • Select somethingadd
  • Clear Alladd
  • Вы достигли лимита на количество...add
  • You are almost on the finish line of our...add
  • Add discount codeadd
  • Warningadd
  • Competitors list {start_name}{end_name}...add
  • Yesadd
  • Зарегистрируйтесь...add
  • Возможностиadd
  • Примерыadd
  • Запросить демоadd
  • Privacy Policyadd
  • FAQadd
  • Growth hacking tool for SEO, PPC and...add
  • Featuresadd
  • Statisticsadd
  • Contact usadd
  • Serpstat Академияadd
  • Serpsat APIadd
  • Send a guest postadd
  • User Testimonialsadd
  • Affiliate Program For Study Centersadd
  • SEOadd
  • Affiliate Agreementadd
  • Backlinksadd
  • Keyword researchadd
  • Text Analyticsadd
  • Was born and developed in SEO and PPC agencyadd
  • Send articleadd
  • Backadd
  • Submitadd
  • Войтиadd
  • Or emailadd
  • Cannot be emptyadd
  • Запомнить меняadd
  • Phoneadd
  • Back To Loginadd
  • Are you sure?add
  • Awesomeadd
  • Enter Phoneadd
  • Your phoneadd
  • smthWentWrongadd
  • telegramBotLinkadd
  • Choose the project to view data on itadd
  • These may include a personal...add
  • We are glad of your commentadd
  • Confirm that you are not a robotadd
  • Contact adminadd
  • Learn more.add
  • Inviteadd
  • E-mailadd
  • Canceladd
  • Searchadd
  • No results matchedadd
  • Я хочу больше лимитовadd
  • Did not find previous lego-bricks? Fill...add
  • No keywordsadd
  • Infoadd
  • Competitors list updated...add
  • Noadd