Custom Segment seems to fail and reports 0 visits


#1

Hi Piwik,

We use crontab to hourly import and process visits from Apache based logs.

We have created 3 segments to filter visits as NoBOT, External Visitors IPs and Internal Visitors IPs.

We noted in some days reports filtered with ‘External Visitors’ have 0 visits, unique visitors, actions per visit and so on, but same reports have not 0 for stats like pageviews, unique pageviews, downloads, unique downloads.

Same reports of same days filtered with NoBOT or no filtered (All Visits) have all stats OK.

In addition, the report ‘Visits per server time’ filtered with ‘External Visitors’ is OK, and each of 24 hours has all the filtered visits (i.e., 3033 vs 2052).

This behavior was repeated randomly without an apparent reason on different days, some days on December, some days on January (i.e. 8th and 12th).

We have tried to reprocess these days invalidating data and reprocessing reports, with no success.

We have also tried to purge db log tables, reimport visits logs, invalidate and archive them, with no success.

We did not find any error message during import or archive stage.

In attach 3 pics, allvisits.png (no filters), ‘external visits segment’ (8th and 12th january with 0 visits), and ‘nobot segment’.

Any idea ???

Thanks in advance
kiwipro


#2

May I ask you whether your custom segments have “AND” combinations? I have a similar issue when creating simple segments that e.g. combine two goals with the AND operator or two URL filters.


#3

Two of custom segments have a combination of AND and OR operators, and these are the segments where reports have 0 visits.


#4

as far as I tried, all AND combinations (also mixed with OR) with goals and urls do not work for me. unfortunately I did not get any reply to my post yet so I have no solution either.


(Matthieu Aubry) #5

Hi guys,

are you able to reproduce a bug using very simple use case? if so, it would be nice to create an issue on our tracker,with steps to reproduce, and the “problematic” segment that does not work for you. We would then take a look and investigate. thanks


#6

Hi Matt,

thanks for your reply.

We have some trouble regarding the chance to reproduce the bug, because it happens occasionally and in an irregular way.

Here is our segment “EXTERNAL NO-BOT” that fails:
operatingSystem != BOT AND (visitIp<10.0.0.0 OR visitIp>10.255.255.255) AND (visitIp<172.16.0.0 OR visitIp>172.31.255.255) AND (visitIp<192.168.0.0 OR visitIp>192.168.255.255).

As example, at January 8th and 12th this segment fails and reports 0 visits, but AllVisits and other segments in the same days are OK (first attached picture).

More strange, PageViews and Downloads metrics are OK for all filtered reports (second attached pic).

We have tried with no success to re-archive these days, invalidating the dates (adding [Debug] section to config.ini.php) and running the command line core:archive (…console core:archive --url= --force-periods=day --force-date-range=2015-01-08,2015-01-08 --force-idsites=9).

From then, we haven’t had no more issues with this filter and all reports are OK so far.

Hope this can help you.

Regards
kiwipro


(Matthieu Aubry) #7

Hi there,

if you somehow can reproduce the bug in more “controlled” way then please make sure to create a bug report at: Issues · matomo-org/piwik · GitHub with steps to reproduce and information. we would definitely take a look!


#8

Hi matt,

we have tried several times invalidating and re-archiving other different days, but we haven’t been able to reproduce the bug.

The segment only fails on specific days (8th and 12th of january), and we can reproduce the issue (visits filtered by the segment = 0) every time.

As mentioned before, every hour we import from apache logs and we archive imported visits applying the segments, and in last month there were no issues so far.

Can you suggest us one o more queries to get from db the results have been produced by the segment in a specific day or week, or some command to run while piwik is processing visits applying the segments ??

We could get some data useful to better understand what’s happened, or capture some query if the issue will happen in the future,in order to share with you these data.

Thanks in advance.
kiwipro