Ignoring Bots and Spiders in Statistics


(Avi) #1

I use the Python log importer to import server access logs.

This imports requests from bots and spiders, which is desired as I use Matomo’s database to detect web scrappers.

However I would like to ignore bots and spiders in the statistics.

Is it possible for them to be ignored somehow? Or differentiated from other traffic?

I would prefer to do this by IP address over user agent, as some scrappers attempt to spoof Googlebot etc.

I have a list of verified search engine and good bot IPs detected by net name.


(Lukas Winkler) #2

Hi,

Matomo does by default ignore all bots when using log import (to turn this off (so all bots are tracked) you need to specify --enable-bots)

The list of Bots Matomo can detect is maintained in the device detector project:


http://devicedetector.net/

I am not sure if blocking by a IP-address is supported yet. Worst case you could preprocess the log files and remove the lines with your blocked IP-addresses.