ERRORS after UPDATE

gcpiwik · October 23, 2014, 7:15pm

Guys,

I have an automatic script that uploads all my access logs onto the Piwik server every morning. Looking through my logs I see that all my files for yesterday were uploaded correctly without any issues. When I open my Piwik home page, I have it set to show yesterdays data because I always load yesterdays data today. So I am interested in delayed data. Now today, I see now data visible in the graphs.

I have run the

php /var/www/html/piwik/console core:archive --url=https://stats.globalcollect.com/piwik/ > /export/home/piwik/piwik-archive.log

I see the following ERRORS, will they affect the results being displayed?

ERROR CoreConsole[2014-10-23 13:40:00] [0863a] Error unserializing the following response from https://stats.globalcollect.com/piwik/index.php?module=API&method=API.get&idSite=1&period=month&date=last2&format=php&token_auth=9af4eb7c60305df5e38b22afcf226d74&trigger=archivephp: Got invalid response from API request: https://stats.globalcollect.com/piwik/index.php?module=API&method=API.get&idSite=1&period=month&date=last2&format=php&token_auth=9af4eb7c60305df5e38b22afcf226d74&trigger=archivephp. Response was ‘curl_exec: SSL read: errno -5961. Hostname requested was: stats.globalcollect.com’

INFO CoreConsole[2014-10-23 13:40:00] [0863a] Archived website id = 1, period = month, 0 visits in last last2 months, 0 visits this month, Time elapsed: 203.557s

ERROR CoreConsole[2014-10-23 13:44:01] [0863a] Error unserializing the following response from https://stats.globalcollect.com/piwik/index.php?module=API&method=API.get&idSite=1&period=year&date=last2&format=php&token_auth=9af4eb7c60305df5e38b22afcf226d74&trigger=archivephp: Got invalid response from API request: https://stats.globalcollect.com/piwik/index.php?module=API&method=API.get&idSite=1&period=year&date=last2&format=php&token_auth=9af4eb7c60305df5e38b22afcf226d74&trigger=archivephp. Response was ‘curl_exec: SSL read: errno -5961. Hostname requested was: stats.globalcollect.com’

I also saw the following in the logs.

INFO CoreConsole[2014-10-23 13:44:29] [0863a] Error: GeoIPAutoUpdater: failed to download ‘http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz’ to ‘/var/www/html/piwik/misc/GeoIPCity.dat.dat.gz’: Error while creating the file: /var/www/html/piwik/misc/GeoIPCity.dat.dat.gz

task,output
Piwik\Plugins\ExamplePlugin\Tasks.myTask,Time elapsed: 0.000s
Piwik\Plugins\UserCountry\GeoIPAutoUpdater.update,ERROR: GeoIPAutoUpdater: failed to download ‘http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz’ to ‘/var/www/html/piwik/misc/GeoIPCity.dat.dat.gz’: Error while creating the file: /var/www/html/piwik/misc/GeoIPCity.dat.dat.gz

very strange that this used to work fine, but now I see that the file being created has a (.dat.dat) in it.
Can someone help me query the database to verify that yesterdays data is present and try to advise on the other errors as well.
Thanks in advance
Lawrence

gcpiwik · October 23, 2014, 8:15pm

tried to update the title,

the data seems to be missing after the latest update. I am not sure that this is related.
The errors shown are when trying to update the database after an upload.

Lawrence

matthieu · October 24, 2014, 3:04am

it looks like your server has some connectivity problems eg.

Response was ‘curl_exec: SSL read: errno -5961. Hostname requested was: stats.globalcollect.com’

gcpiwik · October 29, 2014, 8:26pm

Matt

I have run multiple tests and I do not have an connectivity issues between the two servers.

The ipaddresses are as follows.
piwik application server = 10.171.99.166
piwik database server = 10.171.99.167

The response between the two servers is very good. There is very little latency.

[llamprec@piwik ~]$ ping 10.171.99.167
PING 10.171.99.167 (10.171.99.167) 56(84) bytes of data.
64 bytes from 10.171.99.167: icmp_seq=1 ttl=64 time=0.255 ms
64 bytes from 10.171.99.167: icmp_seq=2 ttl=64 time=0.255 ms
64 bytes from 10.171.99.167: icmp_seq=3 ttl=64 time=0.258 ms
^C
— 10.171.99.167 ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2195ms
rtt min/avg/max/mdev = 0.255/0.256/0.258/0.001 ms

[llamprec@piwik ~]$ traceroute 10.171.99.167
traceroute to 10.171.99.167 (10.171.99.167), 30 hops max, 60 byte packets
1 10.171.99.167 (10.171.99.167) 0.235 ms 0.220 ms 0.209 ms

[llamprec@piwik ~]$ nmap 10.171.99.167 -PN -p ssh | grep open
22/tcp open ssh

[llamprec@piwik ~]$ ssh -o BatchMode=yes -o ConnectTimeout=5 llamprec@10.171.99.167 echo ok 2>&1
ok

What is really strange is that looking at the stats graphs today I see that data has returned.

visitors over time shows the following.
mon 20 Oct - 125789 visits
tues 21 Oct - 109683 vists
wed 22 Oct - 0 vists
thurs 23 Oct - 0 visits
fri 24 Oct - 0 visits
sat 25 Oct - 0 visits
sun 26 Oct - 3867 visits
mon 27 Oct - 119011 visits
thues 28 Oct - 119234 visits

Looking at my upload scripts, all the days showing missing data have successfully been uploaded into the database.

below is a list of the logs that are generated from the results of uploads. As you can see all the files have exactly the same size meaning that all the data has been uploaded. Look specifically at the 21 Oct (data in graph) and 22 Oct. (no data)

https://stats.globalcollect.com/piwik/tmp/stats_gc_missing_data.jpg

-rw-r–r-- 1 piwik piwik 2880 Oct 20 04:53 20141019.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 21 04:59 20141020.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 22 05:24 20141021.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 23 04:56 20141022.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 24 04:00 20141023.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 25 04:02 20141024.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 26 04:02 20141025.piwik.stats.log
-rw-r–r-- 1 piwik piwik 2880 Oct 27 04:02 20141026.piwik.stats.log

Below is the content of the log file.
[piwik@piwik ~]$ cat 20141022.piwik.stats.log
00/22/10
10/23/14_04:00:01
00 hour/22 day/10 month has started
10/23/14_04:02:33
00 hour/22 day/10 month has completed
01/22/10
10/23/14_04:02:33
01 hour/22 day/10 month has started
10/23/14_04:04:21
01 hour/22 day/10 month has completed
02/22/10
10/23/14_04:04:21
02 hour/22 day/10 month has started
10/23/14_04:05:43
02 hour/22 day/10 month has completed
03/22/10
10/23/14_04:05:43
03 hour/22 day/10 month has started
10/23/14_04:06:49
03 hour/22 day/10 month has completed
04/22/10
10/23/14_04:06:49
04 hour/22 day/10 month has started
10/23/14_04:07:48
04 hour/22 day/10 month has completed
05/22/10
10/23/14_04:07:48
05 hour/22 day/10 month has started
10/23/14_04:08:44
05 hour/22 day/10 month has completed
06/22/10
10/23/14_04:08:44
06 hour/22 day/10 month has started
10/23/14_04:09:31
06 hour/22 day/10 month has completed
07/22/10
10/23/14_04:09:31
07 hour/22 day/10 month has started
10/23/14_04:10:30
07 hour/22 day/10 month has completed
08/22/10
10/23/14_04:10:30
08 hour/22 day/10 month has started
10/23/14_04:11:29
08 hour/22 day/10 month has completed
09/22/10
10/23/14_04:11:29
09 hour/22 day/10 month has started
10/23/14_04:12:46
09 hour/22 day/10 month has completed
10/22/10
10/23/14_04:12:46
10 hour/22 day/10 month has started
10/23/14_04:14:29
10 hour/22 day/10 month has completed
11/22/10
10/23/14_04:14:29
11 hour/22 day/10 month has started
10/23/14_04:16:37
11 hour/22 day/10 month has completed
12/22/10
10/23/14_04:16:37
12 hour/22 day/10 month has started
10/23/14_04:19:04
12 hour/22 day/10 month has completed
13/22/10
10/23/14_04:19:04
13 hour/22 day/10 month has started
10/23/14_04:21:41
13 hour/22 day/10 month has completed
14/22/10
10/23/14_04:21:41
14 hour/22 day/10 month has started
10/23/14_04:24:40
14 hour/22 day/10 month has completed
15/22/10
10/23/14_04:24:40
15 hour/22 day/10 month has started
10/23/14_04:27:58
15 hour/22 day/10 month has completed
16/22/10
10/23/14_04:27:58
16 hour/22 day/10 month has started
10/23/14_04:31:38
16 hour/22 day/10 month has completed
17/22/10
10/23/14_04:31:38
17 hour/22 day/10 month has started
10/23/14_04:35:26
17 hour/22 day/10 month has completed
18/22/10
10/23/14_04:35:26
18 hour/22 day/10 month has started
10/23/14_04:39:23
18 hour/22 day/10 month has completed
19/22/10
10/23/14_04:39:23
19 hour/22 day/10 month has started
10/23/14_04:43:09
19 hour/22 day/10 month has completed
20/22/10
10/23/14_04:43:09
20 hour/22 day/10 month has started
10/23/14_04:46:50
20 hour/22 day/10 month has completed
21/22/10
10/23/14_04:46:50
21 hour/22 day/10 month has started
10/23/14_04:50:22
21 hour/22 day/10 month has completed
22/22/10
10/23/14_04:50:22
22 hour/22 day/10 month has started
10/23/14_04:53:47
22 hour/22 day/10 month has completed
23/22/10
10/23/14_04:53:47
23 hour/22 day/10 month has started
10/23/14_04:56:33
23 hour/22 day/10 month has completed

How can I explain that with no changes at all, one day the data is visible and the next nothing.

gcpiwik · November 3, 2014, 8:17am

Is there anyone out there that can help and advise me as to why I am missing data in the graphs, is there a query that I can run against the database to see if teh data is in fact on the database. Might there have been an issue with the loading of the data into the database. Is it possible that the data is not present and that is why the graphs are not showing data (how can I prove or verify this)

Thanks
Lawrence

matthieu · November 7, 2014, 3:08am

@gcpiwik Thanks for asking for support.
Though I see we have spent a long time discussing with your company by email (many many emails) and gave you a lot of information. Unfortunately it didn’t lead to something yet but maybe now is a good time to reconsider the support contract?
Best regards,