Hi,
I have been moved old Piwik server data to another server. Piwik with moved data seems to be work ok and old segments are working fine.
Now I have created simple index.html page to test configurations where tracking code has been set just before end of body.
I have visited yesterday on index.html page one time with IE and two times with Chrome. Visits are tracked which you can see from image.
“All visits” segment on dashboard shows that Visitor browsers are IE and Chrome and number of visits are ok. So this also proves that visits are tracked.
If I create some simple segment like “Device type Is Desktop” just to test configurations then dashboard shows only message: “There is no data for this report.”
Archiving has been set to run hourly:
#!/bin/sh
MAILTO="first.last@email.com"
0 * * * * nginx /usr/bin/php /var/www/piwik/console core:archive --url=http://foo.baz.bar >> /piwik-log/piwik-archive-$(date +\%Y\%m\%d).log 2>> /log/piwik-archive-errors.log
Log files are not containing any errors.
If I run archiving manually like as below then segment start to work but this arching takes about 40 hours with current database (over 20M visits).
./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=http://foo.baz.bar
Should I always run archiving with --force* attributes when I create new segment or do I have some configuration issue here?
Details:
CentOS 6.6, nginx/1.0.15, MariaDB 10.0.17, PHP-FPM 5.3.3, Piwik 2.13.1
config.ini.php:
browser_archiving_disabled_enforce = 1
enable_create_realtime_segments = 0
enable_browser_archiving_triggering = 0
I just figured out what happens. Segment are working ok now but it takes very long time after creating segment.
Log files from clock 10:00 to 20:00 are like below:
INFO [2015-05-08 hh:mm:01] ---------------------------
INFO [2015-05-08 hh:mm:01] INIT
INFO [2015-05-08 hh:mm:01] Piwik is installed at: http://foo.baz.bar/index.php
INFO [2015-05-08 hh:mm:01] Running Piwik 2.13.1 as Super User
INFO [2015-05-08 hh:mm:01] ---------------------------
INFO [2015-05-08 hh:mm:01] NOTES
INFO [2015-05-08 hh:mm:01] - Reports for today will be processed at most every 3600 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO [2015-05-08 hh:mm:01] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO [2015-05-08 hh:mm:01] - Archiving was last executed without error 59 min 53s ago
INFO [2015-05-08 hh:mm:01] - Will pre-process 1 Segments for each website and each period: siteSearchKeyword=@foo
INFO [2015-05-08 hh:mm:07] - Will process 0 websites with new visits since 59 min 53s
INFO [2015-05-08 hh:mm:07] ---------------------------
INFO [2015-05-08 hh:mm:07] START
INFO [2015-05-08 hh:mm:07] Starting Piwik reports archiving...
INFO [2015-05-08 hh:mm:07] Done archiving!
INFO [2015-05-08 hh:mm:07] ---------------------------
INFO [2015-05-08 hh:mm:08] SUMMARY
INFO [2015-05-08 hh:mm:08] Total visits for today across archived websites: 0
INFO [2015-05-08 hh:mm:08] Archived today's reports for 0 websites
INFO [2015-05-08 hh:mm:08] Archived week/month/year for 0 websites
INFO [2015-05-08 hh:mm:08] Skipped 1 websites: no new visit since the last script execution
INFO [2015-05-08 hh:mm:08] Skipped 0 websites day archiving: existing daily reports are less than 3600 seconds old
INFO [2015-05-08 hh:mm:08] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
INFO [2015-05-08 hh:mm:08] Total API requests: 0
INFO [2015-05-08 hh:mm:08] done: 0/0, 0 vtoday, 0 wtoday, 0 wperiods, 0 req, 22 ms, no error
INFO [2015-05-08 hh:mm:08] Time elapsed: 0.023s
INFO [2015-05-08 hh:mm:08] ---------------------------
INFO [2015-05-08 hh:mm:08] SCHEDULED TASKS
INFO [2015-05-08 hh:mm:08] Starting Scheduled tasks...
INFO [2015-05-08 hh:mm:08] Scheduler: executing task Piwik\Plugins\ExamplePlugin\Tasks.myTask...
INFO [2015-05-08 hh:mm:08] Scheduler: finished. Time elapsed: 0.001s
INFO [2015-05-08 hh:mm:08] done
INFO [2015-05-08 20:00:08] ---------------------------
Similar logging all day but then at clock 21:00:
INFO [2015-05-08 21:00:01] ---------------------------
INFO [2015-05-08 21:00:01] INIT
INFO [2015-05-08 21:00:01] Piwik is installed at: http://foo.baz.bar/index.php
INFO [2015-05-08 21:00:01] Running Piwik 2.13.1 as Super User
INFO [2015-05-08 21:00:02] ---------------------------
INFO [2015-05-08 21:00:02] NOTES
INFO [2015-05-08 21:00:02] - Reports for today will be processed at most every 3600 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO [2015-05-08 21:00:02] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO [2015-05-08 21:00:02] - Archiving was last executed without error 59 min 54s ago
INFO [2015-05-08 21:00:02] - Will pre-process 1 Segments for each website and each period: siteSearchKeyword=@foo
INFO [2015-05-08 21:00:08] - Will process 0 websites with new visits since 59 min 53s
INFO [2015-05-08 21:00:08] - Will process 1 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 2
INFO [2015-05-08 21:00:08] ---------------------------
INFO [2015-05-08 21:00:08] START
INFO [2015-05-08 21:00:08] Starting Piwik reports archiving...
INFO [2015-05-08 21:00:08] Will pre-process for website id = 2, day period
INFO [2015-05-08 21:00:08] - pre-processing all visits
INFO [2015-05-08 21:00:09] - pre-processing segment 1/27 ...
INFO [2015-05-08 21:00:09] - pre-processing segment 2/27 ...
INFO [2015-05-08 21:00:09] - pre-processing segment 3/27 ...
INFO [2015-05-08 21:00:10] - pre-processing segment 4/27 ...
INFO [2015-05-08 21:00:10] - pre-processing segment 5/27 ...
INFO [2015-05-08 21:00:10] - pre-processing segment 6/27 ...
INFO [2015-05-08 21:00:11] - pre-processing segment 7/27 ...
INFO [2015-05-08 21:00:11] - pre-processing segment 8/27 ...
INFO [2015-05-08 21:00:11] - pre-processing segment 9/27 ...
INFO [2015-05-08 21:00:12] - pre-processing segment 10/27 ...
...
So archiving doesn’t start to process segments at the first archive job. I looked log files and this behavior occurs every day. Is this normal?
EDIT: and this processing takes very long time. Hourly cron job cannot be set without lockfile or similar way to prevent simultaneous cron jobs.
