Bot Filtering
      Back to home
      1. Help Center
      2. Configuration
      3. Bot Filtering

      How Do You Filter Bots?

      THIS ARTICLE WILL HELP YOU:

      Understand Filtering Bots

      Convert currently filters bots in the reporting side if the user agent contains the following words:

      'abachobot','abilogicbot','accoona-ai-agent','anyapexbot','arachmo','b-l-i-t-z-b-o-t','baiduspider','becomebot','bimbot','blitzbot','boitho.com-dc','boitho.com-robot','btbot','cerberian drtrs','charlotte','converacrawler','cosmos','curl','dataparksearch','diamondbot','discobot','emeraldshield.com webbot','envolk[its]spider','esperanzabot','exabot','fast enterprise crawler','fast-webcrawler','fdserobot','findlinks','furlbot','fyberspider','g2crawler','gaisbot','galaxybot','geniebot','gigabot','girafabot','googlebot','googlebot-image','hl_ftien_spider','htdig','ia_archiver','ichiro','irlbot','issuecrawler','jyxobot','lapozzbot','larbin','libwww-perl','link valet','link validity check','linksmanager.com_bot','linkwalker','lmspider','lwp-trivial','mabontland','mediapartners-google','mj12bot','mnogosearch','mogimogi','mojeekbot','mojoo robot','morning paper','msnbot','msrbot','mvaclient','netresearchserver','ng-search','nicebot','notifixious','noxtrumbot','nusearch spider','nutchcvs','obot','oegp','omniexplorer_bot','online link validator','oozbot','orbiter','pagebiteshyperbot','peach','polybot','pompos','psbot','ptst/','pycurl','python-urllib','rampybot','reciprocal link system pro','rel link checker lite','rufusbot','sandcrawler','sbider','scrubby','searchsight','seekbot','semanticdiscovery','sensis web crawler','seochat::bot','shim-crawler','shopwiki','shoula robot','silk','sitebar','snappy','sogou spider','speedy spider','sqworm','stackrambler','surveybot','synoobot','teoma','terrawizbot','thesubot','thumbnail.cz robot','tineye','turnitinbot','updated','vagabondo','vivante link checker','voilabot','vortex','voyager','vyu2','w3c-checklink','webcollage','websquash.com','wf84','wofindeich robot','xaldon_webspider','xenu link sleuth','yacy','yahoo! slurp','yahoo! slurp china','yahooseeker','yahooseeker-testing','yooglifetchagent','zao','zealbot','zspider','zyborg'

      Read this article if you want to exclude bots from viewing variations as well, keeping in mind the Cloaking guidelines.

      Mobile Googlebot

      There are 9 different googlebots:

      • Googlebot (Google Web search)
      • Google Smartphone
      • Google Mobile (Feature phone)
      • Googlebot Images
      • Googlebot Video
      • Googlebot News
      • Google Adsense
      • Google Mobile Adsense
      • Google Adsbot (landing page quality check)

      however the user-agent that the web and mobile are using is this: Googlebot. Here https://support.google.com/webmasters/answer/1061943?hl=en you can find the official list from Google.

      Google retired "Googlebot-Mobile" for smartphones as a user agent back in August 2015. From then on, the user-agent for smartphones is identified itself simply as "Googlebot" but is listed "mobile" elsewhere in the user-agent string.

      Canonical Tags

      You don't need to add canonical tags to the variation of your experiments. The reason is that we filter search bots, which result on non indexing of experience variations.