Ignoring Bots and Spiders in Statistics

General questions
1

I use the Python log importer to import server access logs.

This imports requests from bots and spiders, which is desired as I use Matomo’s database to detect web scrappers.

However I would like to ignore bots and spiders in the statistics.

Is it possible for them to be ignored somehow? Or differentiated from other traffic?

I would prefer to do this by IP address over user agent, as some scrappers attempt to spoof Googlebot etc.

I have a list of verified search engine and good bot IPs detected by net name.

2

Hi,

Matomo does by default ignore all bots when using log import (to turn this off (so all bots are tracked) you need to specify --enable-bots)

The list of Bots Matomo can detect is maintained in the device detector project:

http://devicedetector.net/

I am not sure if blocking by a IP-address is supported yet. Worst case you could preprocess the log files and remove the lines with your blocked IP-addresses.

1 Like