Century Bot
Centurybot
"Centurybot" is the robot we use to crawl pages for index. The request sent by this robot can be identified by "User-Agent" header field. Something like following can be observed on server side:
Politeness
Centurybot is a very polite robot and crawls much less often than other crawlers. Typically, it will retrieve one page in a few seconds. It could bring observable traffic but it is unlikely to cause any issues due to its conservative politeness setting. If somehow this crawler causes undesirable impact for your website, please let us know.
robots.txt
Centurybot respects robots.txt setting, it follows what other bot does, like googlebot.