A lot of Bots are accessing my Website, which are still counted in the visitor log. This is screwing up all kinds metrics like visitor numbers, etc.
Here are some of the user Agents
""Mozilla/5.0 (compatible; DotBot/1.1; http://www.opensiteexplorer.org/dotbot, firstname.lastname@example.org)"
"Mozilla/5.0 (compatible; Googlebot/2.1; +https://www.contentkingapp.com/)"
What can I do to prevent this?
Also the exclusion of User Agent in the admin interface when placing “Googlebot” in there seems to do nothing at atll.
I also have problems with crawlers that send no User Agent at all. All that is found in the user agent is two hyphens.