Two weeks "0 visits"... recalculation possible?


(Oliver Lippert) #1

Hey there,
currently there are some troubles using PIWIK. One of theese is as following:

I have an page with about 100 users per day or more.
But if I use the Weekly Graph-View, two weeks have “0 visitors”, that isn’t correct.

Look at the Screenshot.

  1. WHAT could be the Problem?
  2. HOW could I may fix this?

Any help is welcome :slight_smile:

Regards, Oli


#2

Same here, but for the months view. As far as I can tell the details are present in the database and the week/day view is showing data for these months.


(Matthieu Aubry) #3

See Troubleshooting - Analytics Platform - Matomo


#4

Excellent, that worked great. I renamed the relevant tables and visited in the browser again, but it kept timing out. I then called
index.php?module=API&method=CoreAdminHome.invalidateArchivedReports&idSites=1,3,5&dates=2012-01-01,2011-10-15&token_auth=xyz
with the relevant dates in the months that didn’t work and the correct token_auth, then I ran the archive.php on the server which fixed it. Probably would have worked without renaming the database.

Thanks for your pointer!


(Oliver Lippert) #5

Hey Matt, thanks for replying… now 4 weeks have 0 visits, AFTER processing…

Heres the Log:


INFO [2014-01-21 18:57:34] [48411] ---------------------------
INFO [2014-01-21 18:57:34] [48411] INIT
INFO [2014-01-21 18:57:34] [48411] Querying Piwik API at: http://piwik.xyz.de/index.php
INFO [2014-01-21 18:57:34] [48411] Running Piwik 2.0.3 as Super User: admin
INFO [2014-01-21 18:57:34] [48411] ---------------------------
INFO [2014-01-21 18:57:34] [48411] NOTES
INFO [2014-01-21 18:57:34] [48411] - Reports for today will be processed at most every 120 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO [2014-01-21 18:57:34] [48411] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO [2014-01-21 18:57:34] [48411] - Archiving was last executed without error 7 min 3s ago
INFO [2014-01-21 18:57:34] [48411] - Will process 2 websites with new visits since 7 min 3s , IDs: 10, 15
INFO [2014-01-21 18:57:34] [48411] - Will process 2 other websites because some old data reports have been invalidated (eg. using the Log Import script) , IDs: 23, 10
INFO [2014-01-21 18:57:34] [48411] ---------------------------
INFO [2014-01-21 18:57:34] [48411] START
INFO [2014-01-21 18:58:01] [48411] Will pre-process the following 1 Segments for this website (id = 10): referrerUrl=@facebook.com
INFO [2014-01-21 18:58:01] [48411] Archived website id = 10, period = day, Time elapsed: 26.749s
INFO [2014-01-21 18:58:01] [48411] Will pre-process the following 1 Segments for this website (id = 10): referrerUrl=@facebook.com
INFO [2014-01-21 18:58:03] [48411] Archived website id = 10, period = week, 885 visits, Time elapsed: 1.947s
INFO [2014-01-21 18:58:03] [48411] Will pre-process the following 1 Segments for this website (id = 10): referrerUrl=@facebook.com
INFO [2014-01-21 18:58:19] [48411] Archived website id = 10, period = month, 2033 visits, Time elapsed: 15.711s
INFO [2014-01-21 18:58:19] [48411] Will pre-process the following 1 Segments for this website (id = 10): referrerUrl=@facebook.com
INFO [2014-01-21 18:58:39] [48411] Archived website id = 10, period = year, 2033 visits, Time elapsed: 20.171s
INFO [2014-01-21 18:58:39] [48411] Archived website id = 10, today = 65 visits, 8 API requests, Time elapsed: 64.609s [1/2 done]
INFO [2014-01-21 18:58:43] [48411] Archived website id = 15, period = day, Time elapsed: 3.902s
INFO [2014-01-21 18:58:49] [48411] Archived website id = 15, period = week, 16847 visits, Time elapsed: 5.631s
INFO [2014-01-21 19:03:06] [48411] Archived website id = 15, period = month, 111525 visits, Time elapsed: 257.914s
INFO [2014-01-21 19:03:26] [48411] Archived website id = 15, period = year, 437526 visits, Time elapsed: 19.493s
INFO [2014-01-21 19:03:26] [48411] Archived website id = 15, today = 1713 visits, 4 API requests, Time elapsed: 286.942s [2/2 done]
INFO [2014-01-21 19:03:26] [48411] Starting Piwik reports archiving...
INFO [2014-01-21 19:03:26] [48411] Done archiving!
INFO [2014-01-21 19:03:26] [48411] ---------------------------
INFO [2014-01-21 19:03:26] [48411] SUMMARY
INFO [2014-01-21 19:03:26] [48411] Total daily visits archived: 1778
INFO [2014-01-21 19:03:26] [48411] Archived today's reports for 2 websites
INFO [2014-01-21 19:03:26] [48411] Archived week/month/year for 2 websites
INFO [2014-01-21 19:03:26] [48411] Skipped 27 websites: no new visit since the last script execution
INFO [2014-01-21 19:03:26] [48411] Skipped 0 websites day archiving: existing daily reports are less than 120 seconds old
INFO [2014-01-21 19:03:26] [48411] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
INFO [2014-01-21 19:03:26] [48411] Total API requests: 12
INFO [2014-01-21 19:03:26] [48411] done: 2/2 100%, 1778 v, 2 wtoday, 2 wperiods, 12 req, 351553 ms, no error
INFO [2014-01-21 19:03:26] [48411] Time elapsed: 351.553s
INFO [2014-01-21 19:03:26] [48411] ---------------------------
INFO [2014-01-21 19:03:26] [48411] SCHEDULED TASKS
INFO [2014-01-21 19:03:26] [48411] Starting Scheduled tasks...
INFO [2014-01-21 19:03:26] [48411]  No task to run
INFO [2014-01-21 19:03:26] [48411] done
INFO [2014-01-21 19:03:26] [48411] ---------------------------

Please help!?

Regards,
Oli


(Matthieu Aubry) #6

Do you have data in the “Visitor log” for these dates?

Did you try delete the piwik_archive tables for these months & reprocess?


(Oliver Lippert) #7

Hey matt, THANKS for your replys :slight_smile:

Do you have data in the “Visitor log” for these dates?
-> Yes I do (see attachment)

Did you try delete the piwik_archive tables for these months & reprocess?
-> I deleted piwik_archive_blob_2013_12 and piwik_archive_numeric_2013_12, than reprocessed (that wasn’t that easy as your HowTo/FAQ described it), now they are existing anymore.

My Piwik-Instance currently tracks 29 Websites, in December there whre 80.000 visits with 140.000 actions. Is there any way to work on that problem without destroying the Reports of the other existing Pages?


(Matthieu Aubry) #8

it’s more simple to delete all reports and have them reprocess. Should be quite fast!

Otherwise you can delete only the rows from the archive tables, for the particular idsite


(Oliver Lippert) #9

Hey matt,
OK I delete everything thats needed to delete, but HOW can I now fix my problem??


(Oliver Lippert) #10

Hey matt,
is there any response or do I belong now on my own to this Problem?


(Oliver Lippert) #11

It seems that I did helped me Out this way:

[ul]
[li] Remove piwik_archive_*-Tables for the “defect” month
[/li][li] Log into Piwik as admin
[/li][li] GoTo Settings
[/li][li] Activate data-processing through visiting the UI
[/li][li] modify INI (“always_archive_data_period = 1” and “always_archive_data_day = 1”)
[/li][li] Open the “defect” website
[/li][li] Open the “defect” report, like Week (in my case weeks of december)
[/li][li] WAIT a long Time (pending on your Server and your amount of Data)
[/li][/ul]

So now my Weeks in December are all successfully processed again. The Month-Report doesn’t work also, the data-calculation in the way I described it is in progress since some minutes. I think this will work also successfully :slight_smile:

Regards,
Oli

UPDATE: It worked, but I have to do it for ALL Websites :confused: (round about 30 at the moment)
@matt: please write me something that I can do ^^

UPDATE 2: With my second Problem I did help me out like this way:

[ul]
[li] Log into Piwik as admin
[/li][li] GoTo Settings
[/li][li] Activate data-processing through visiting the UI
[/li][li] Open the “All Webseites” view
[/li][li] Open the “defect” report, like Week (in my case weeks of december) or Months (in my case december)
[/li][li] WAIT a long Time (pending on your Server and your amount of Data)
[/li][/ul]

Then all Pages will be processed…
Now I don’t see any calculation-problems anymore.


(Oliver Lippert) #12

The Problem still occurs :confused:

On some pages there are often weeks with 0 visits.

Can anyone analyze this Problem?


(Oliver Lippert) #13

I still hat die archive.php-CronJob running. Changed to the “new” way of doing the cron - the Shell-Script.

I do also follow this FAQ How to - Analytics Platform - Matomo to remove ALL my old data (year per year) and reprocess ist.
Tested it on an month where I knew it hat “0 visits-Weeks” in one month, and they where re-calculated correctly.

So it will take some hours to re-calculate all the Data since 2009 (DB-Size is round about 3.5 GB). I will write again if it helped out or if it doesn’t.

Thanks for the “new” FAQs and the “new” CronJob (didn’t mind to get noticed about an new way to do the cronjobs).

Update:
Hey Folks,
something really strange: Archiving an small page, from 2010 till now, with just 2916 Visits with 10085 Actions the archiving took 1045 seconds. Its an netcup VPS with 4 GB and one Core CPU. Is it OK to be so slow? Can I tune up some parameters to get faster?
Or is it even fast?

The archiving of the 37 Websites will took a long time, because this page is the smalest one. The archiving-process will re-process pages for two or three years of data with 30.000 Visits per month…

Update 2:
To be more precise:

INFO CoreConsole[2014-07-08 18:46:07] [6b85d] Archived website id = 1, period = day, 2916 visits in last last10000 days, 0 visits today, Time elapsed: 357.521s
INFO CoreConsole[2014-07-08 18:52:35] [6b85d] Archived website id = 1, period = week, 2916 visits in last last10000 weeks, 0 visits this week, Time elapsed: 388.064s
INFO CoreConsole[2014-07-08 18:57:25] [6b85d] Archived website id = 1, period = month, 2916 visits in last last10000 months, 0 visits this month, Time elapsed: 289.688s
INFO CoreConsole[2014-07-08 18:57:36] [6b85d] Archived website id = 1, period = year, 2916 visits in last last10000 years, 0 visits this year, Time elapsed: 10.336s
INFO CoreConsole[2014-07-08 18:57:36] [6b85d] Archived website id = 1, 4 API requests, Time elapsed: 1045.657s [1/37 done]
INFO CoreConsole[2014-07-08 19:11:22] [6b85d] Archived website id = 2, period = day, 43564 visits in last last10000 days, 22 visits today, Time elapsed: 826.186s
INFO CoreConsole[2014-07-08 19:31:22] [6b85d] Archived website id = 2, period = week, 43568 visits in last last10000 weeks, 53 visits this week, Time elapsed: 1199.866s

(mike) #14

Hi Oliver. Your problem sounds a lot like mine – the archive script is just really slow. If you don’t get a reply here, you might want to see if I have better luck over on my thread: 301 Moved Permanently. In any case, I’ll be watching this thread too.


(Oliver Lippert) #15

Howdy,
after round about 8 Hours (30743 Seconds) the process finished its goal.

All my Data is now up to date. Seems to have no weeks with zero visits. I hope it will stay so.

@Mike: Thanks for the hint, I will watch the Thread.