Welcome to WebmasterWorld Guest from 126.96.36.199
You can create Bing and Google Webmaster accounts and configure your domains to have a lower crawl-delay
is the problem on my site or is it the hosting?It is the hosting. A Bad Gateway error means that there is a miscommunication between a proxy server and one of their servers. In other words, they don't give the proxy server the response it expects. In other words they're blocking access to your site.
which year we are in
You can create Bing and Google Webmaster accounts and configure your domains to have a lower crawl-delay. We also recommend configuring a robots.txt file. This will reduce the rate that crawlers initiate requests with your site
If you would like to reduce traffic from crawlers such as Yandex or Baidu these typically need to be done utilizing something in the nature of an .htaccess block.
can getting a large number of unknown bots have a negative impact on my SEO in Google?
If it's your own server, why is anyone talking about htaccess? There shouldn't be any. Except maybe temporarily if you're making major changes. Otherwise everything should be happening in the config file.
Or possibly which planet we're on?LOL :)
What can I do to limit them?
The host recommends adding a caching mechanism to the code of my site. I'm not familiar with this