Piwik data getting lost every feew days! I'm slowly getting pissed (td)


(yamyam) #1

As there has been any help or even response to this support request at all, it has been deleted.


(yamyam) #2

As there has been any help or even response to this support request at all, it has been deleted.


(yamyam) #3

I have to bring this up again as this happened the third time now and i slowly getting pissed! :X

My pageviews and visitors are increasing daily, and my cronjob is doing the archive.php every 1 hour.
i DO NOT have settings to deleted old data, but WHYEVER every 2-3 days a bunch of data is getting lost!!! (pageviews as also visitors als also nations, and so on)

Example:
Pageviews 6 days ago: > 20.000
One day later: BELOW 20.000
Yesterday: > 22.000
Today: ~19.000 !!!

Same happens with amount of visitors and other data!

Whats wrong with piwik? Its useless if data is getting deleted even if i do not have any setting which say it will.
I really do not need this so if noone will help me i gonna kick piwik of my server and use google analytics again.
this is at least working and not killing my data!

So please finally help after one week.
I am asking for serious help.

THANK YOU! (tu)


(Matthieu Aubry) #4

I’ve never heard this problem before. Can you post some screenshots to show that the problem occurs for you?


(yamyam) #5

Sure, just an example:

1st screenshot is some days ago (as we reached 20.000 pageviews), 2nd screenshow below is from just now (few seconds ago).
As you see all data has decreased, even if my settings are set to NOT delete old data and my archive.php is done by a cronjob hourly.

Using the counter plugin, which just gets the data from the API, so i don’t think it is a bug of the plugin.

I am just wondering as it seems to be the same scheme every week:
The data is counted correctly and increasing daily, and then (like every sunday for example), some data egts “lost” and the counter shows the decreased amount of pageviews, nations and visitors for about 24 hours (even if there is traffic on the site every day 24/7), until it starts to increase again.

Like a weekly “clean up” with one day brake … lol.

HELP! :slight_smile:

I have sent you the URL of the page on PM.
If you wanna see this mystery, just keep writing down those three data on daily basis and wait until the error occurs again.

Thanks.


(Matthieu Aubry) #6

I cant see the image. Are you using Counter plugin? Then maybe it’s a bug in the Counter plugin rather than Piwik?


(yamyam) #7

[quote=matt]
I cant see the image. Are you using Counter plugin? Then maybe it’s a bug in the Counter plugin rather than Piwik?[/quote]
Image is viewable to me, check http://www7.pic-upload.de/24.02.14/umeqrc4q5ufb.png
I don’t think it is the Counter Plugin as the Plugin just gets the data from the piwik API. Another reason: Even if i select the same timerange in piwik itself, i get the same “wrong” (decreased) data. The counter plugin just can show what piwik is calculating.

Greets


#8

matt the problem is not in the Counter plugin. I just use a ‘UserCountry.getCountry’ and ‘Live.getCounters’ API methods for fetching statistics about visits and views.

I cannot understand why it’s happen, on my test platform on a production site I get wrong data only for number of countries(because Piwik count countries like a ‘Anonymous Proxy’ or simply ‘Unknown’). But unrecognized countries was for reports when the GeoIP was outdated. At least for two last years I get right data for it.

yamyam can you delete an archived data(not logs) from DB and rerun the archiving?


(Matthieu Aubry) #9

Can you reproduce such problem on the demo.piwik.org ? If you can show me exactly the bug then I will definitely look into fixing ti! thanks


(yamyam) #10

Yes, thats also a problem or “bug” as piwik should really only count real nations as countries and not “proxy” or “satellite user” and stuff like that. This is the reason why i had to take off the [nations] variable from counter plugin (as amount of nations was wrong), and hardcoded the real amount in the counter text until piwik fixes this issue.

[quote=Viper]
yamyam can you delete an archived data(not logs) from DB and rerun the archiving?[/quote]

Of course i can, if you tell me what exactly counts as archived data? :wink:

This is my current piwik database: http://www7.pic-upload.de/26.02.14/jtcn5dqbgum.png
137 tables, about 837K entries and 162 MB size.

What should i delete? And my archive.php is done by the cronjob hourly anyway, so i just wait until the next hour passed?

Greets


#11

Read this docs How do I force the reports to be re-processed from the logs? - Analytics Platform - Matomo


(yamyam) #12

[quote=Viper]

Read this docs How do I force the reports to be re-processed from the logs? - Analytics Platform - Matomo

Ok thanks fpr the link! I will do this, but what do you think will this improve?
You think the fluctuation of my counter numbers occurs because some old logs are wrong?
However, i will do it and tell you what happened, but i will need some hours before i havw time to check it.

Another thing is - maybe also a question to matt:
In piwik itself the first date i can select is 09.11.2013.
Which means for me piwik got the first data of my site at 09.11.2013, right?
So now the mysterious:
If i select 09.11.2013 in the counter plugin as start date i get some data.
If i select 01.11.2013 in the counter plugin, i get increased numbers (a little bit).
If i select 01.01.2013, i get ever higher numbers but it makes no difference if i change it to 01.01.2000 or 01.01.2005 … i think 01.01.2013 should include all data piwik got about my site. But WHY is piwiki showing me then that the first data he got is from 09.11.2013 and if i select 01.11.2013 as start date he is showing more (increased) data?

Greets and thanks you all for helping.


(yamyam) #13

It is becoming even more funny now :slight_smile:

As of 01.03.2014 all the missing nations, visits and pageviews are BACK! (:P)

All three numbers jumped extremly up in just one moment (can say exactly when), but amounbt if visitors, nations and pageviews ios now back where it should be, inlcuding all the thousands of hits and visitors who got “lost” the last days.

So how does this comes???

Could that be related to the monthly report/archiving of the database? or is it just by fortune that this happened at 01.03. ?

I am confused :wink:


(yamyam) #14

UPDATE!

One week later … the same happened again!
One week everything was increasing and going well, and BAM … just another drop from one day of another.

Had a simple manual log running since the first decrease and (back-)increase:

(date / time / visitors / nations / pageviews)
01.03. / 01:18 / 6.000 / 95 / 27.326
01.03. / 08:42 / 6.006 / 95 / 27.399
01.03. / 16:37 / 6.028 / 95 / 27.841
02.03. / 13:06 / 6.106 / 95 / 27.840
02.03. / 15:28 / 6.124 / 95 / 27.894
02.03. / 18:21 / 6.144 / 95 / 28.015
02.03. / 23:31 / 6.164 / 95 / 28.129
03.03. / 18:47 / 6.213 / 95 / 28.275
04.03. / 20:55 / 6.278 / 95 / 28.461
06.03. / 18:57 / 6.406 / 95 / 28.948
07.03. / 18:53 / 6.460 / 95 / 29.201
08.03. / 18:08 / 6.512 / 95 / 29.347
09.03. / 02:10 / 6.168 / 92 / 28.143

It is like happening once a week (every sunday on the weekly archiving maybe?!)

EDIT:
I have also noticed that every time this happens (seems to be weekly), that the counter data is not changing at all for several hours or even days (have to check deeper). Normaly the site has traffic hourly and the counter data is also increasing hourly too (every 1 hour archive.php is done). But as soon as this “data-dump” occurs, the data is like frozen for about 1 day … very suspicious :frowning:


(Matthieu Aubry) #15

@yamyam, if you are using Piwik 2.1.0 then there is definitely a bug.
thanks for reporting it.

can you create a ticket with your bug and what you see VS what you expect?
In general it would be super useful if you find the pattern in when the bug occurs. Then with the pattern we can try to replicate the bug. Then we could fix it. cheers