Problem with Archiving in Piwik 2.8.3


#1

Hello there,

I just experienced the following problem.

Called from console, memory 2048M
The periods “day” and “week” went fine. Then the following message was recorded. It’s reproducable.

ERROR CoreConsole[2014-10-24 08:38:52] [b09c8] Got invalid response from API request: http://domain/piwik/index.php?module=API&method=API.get&idSite=1&period=month&date=last52&format=php&token_auth=tokenauth&trigger=archivephp. Response was ’ There is an error. Please report the message (Piwik 2.8.3) and full backtrace in the Piwik forums (please do a Search first as it might have been reported already!). Warning: Error while sending QUERY packet. PID=8115 in /paths/domain/piwik/libs/Zend/Db/Statement/Pdo.php on line 228 Backtrace -->
#0 Piwik\Error::errorHandler(…) called at [:]
#1 PDOStatement->execute(…) called at [/paths/domain/piwik/libs/Zend/Db/Statement/Pdo.php:228]
#2 Zend_Db_Statement_Pdo->_execute(…) called at [/paths/domain/piwik/libs/Zend/Db/Statement.php:300]
#3 Zend_Db_Statement->execute(…) called at [/paths/domain/piwik/core/Db/Adapter/Pdo/Mysql.php:230]
#4 Piwik\Db\Adapter\Pdo\Mysql->query(…) called at [/paths/domain/piwik/libs/Zend/Db/Adapter/Abstract.php:736]
#5 Zend_Db_Adapter_Abstract->fetchAll(…) called at [/paths/domain/piwik/core/Db/Adapter/Pdo/Mysql.php:175]
#6 Piwik\Db\Adapter\Pdo\Mysql->isConnectionUTF8(…) called at [/paths/domain/piwik/core/DbHelper.php:121]
#7 Piwik\DbHelper::isDatabaseConnectionUTF8(…) called at [/paths/domain/piwik/core/Db/BatchInsert.php:80]
#8 Piwik\Db\BatchInsert::tableInsertBatch(…) called at [/paths/domain/piwik/core/DataAccess/ArchiveWriter.php:225]
#9 Piwik\DataAccess\ArchiveWriter->insertBulkRecords(…) called at [/paths/domain/piwik/core/DataAccess/ArchiveWriter.php:98]
#10 Piwik\DataAccess\ArchiveWriter->insertBlobRecord(…) called at [/paths/domain/piwik/core/ArchiveProcessor.php:334]
#11 Piwik\ArchiveProcessor->insertBlobRecord(…) called at [/paths/domain/piwik/core/ArchiveProcessor.php:230]
#12 Piwik\ArchiveProcessor->aggregateDataTableRecords(…) called at [/paths/domain/piwik/plugins/Referrers/Archiver.php:220]
#13 Piwik\Plugins\Referrers\Archiver->aggregateMultipleReports(…) called at [/paths/domain/piwik/core/ArchiveProcessor/PluginsArchiver.php:102]
#14 Piwik\ArchiveProcessor\PluginsArchiver->callAggregateAllPlugins(…) called at [/paths/domain/piwik/core/ArchiveProcessor/Loader.php:122]
#15 Piwik\ArchiveProcessor\Loader->prepareAllPluginsArchive(…) called at [/paths/domain/piwik/core/ArchiveProcessor/Loader.php:72]
#16 Piwik\ArchiveProcessor\Loader->prepareArchive(…) called at [/paths/domain/piwik/core/Archive.php:801]
#17 Piwik\Archive->prepareArchive(…) called at [/paths/domain/piwik/core/Archive.php:614]
#18 Piwik\Archive->cacheArchiveIdsAfterLaunching(…) called at [/paths/domain/piwik/core/Archive.php:568]
#19 Piwik\Archive->getArchiveIds(…) called at [/paths/domain/piwik/core/Archive.php:508]
#20 Piwik\Archive->get(…) called at [/paths/domain/piwik/core/Archive.php:348]
#21 Piwik\Archive->getDataTableFromNumeric(…) called at [/paths/domain/piwik/plugins/VisitsSummary/API.php:59]
#22 Piwik\Plugins\VisitsSummary\API->get(…) called at [:]
#23 call_user_func_array(…) called at [/paths/domain/piwik/core/API/Proxy.php:211]
#24 Piwik\API\Proxy->call(…) called at [/paths/domain/piwik/plugins/API/API.php:403]
#25 Piwik\Plugins\API\API->get(…) called at [:]
#26 call_user_func_array(…) called at [/paths/domain/piwik/core/API/Proxy.php:211]
#27 Piwik\API\Proxy->call(…) called at [/paths/domain/piwik/core/API/Request.php:216]
#28 Piwik\API\Request->process(…) called at [/paths/domain/piwik/plugins/API/Controller.php:33]
#29 Piwik\Plugins\API\Controller->index(…) called at [:]
#30 call_user_func_array(…) called at [/paths/domain/piwik/core/FrontController.php:587]
#31 Piwik\FrontController->doDispatch(…) called at [/paths/domain/piwik/core/FrontController.php:95]
#32 Piwik\FrontController->dispatch(…) called at [/paths/domain/piwik/core/dispatch.php:34]
#33 require_once(…) called at [/paths/domain/piwik/index.php:46]
#34 require_once(…) called at [/paths/domain/piwik/core/CliMulti/RequestCommand.php:53]
#35 Piwik\CliMulti\RequestCommand->execute(…) called at [/paths/domain/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252]
#36 Symfony\Component\Console\Command\Command->run(…) calle

Sorry. More wasn’t there. The next line was:
Archived website id = 1, period = month, 0 visits in last last52 months, 0 visits this month, Time elapsed: 94.154s

Thanks, Fixedsys


(Matthieu Aubry) #2

Hi there, error message “Warning: Error while sending QUERY packet. PID=8115” should be a temporary issue

if you get it many times, let us know!


#3

Hi there,

thanks for the reply. I’m sorry to say that the error occurred on every single run over the weekend.

The error message is the same every time, only the PID changes.

Regards,
Fixedsys


#4

I just got an new error message:

ERROR CoreConsole[2014-10-27 14:25:16] [37171] Got invalid response from API request: http://domain/piwik/index.php?module=API&method=API.get&idSite=2&period=day&date=last2&format=php&token_auth=token_auth&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of ‘memory_limit’ in your php.ini file. Please check your Web server Error Log file for more details.
ERROR CoreConsole[2014-10-27 14:25:16] [37171] Empty or invalid response ‘’ for website id 2, Time elapsed: 7.843s, skipping
ERROR CoreConsole[2014-10-27 14:25:23] [37171] 2 total errors during this script execution, please investigate and try and fix these errors.

The memory limit is 2048M.


#5

The last one suggests more memory for PHP. Not sure if you are able to.

I personally use PHP-FPM, with pools. So my piwik pool has a lot of memory, where the other pools have low memory. This way I can give Piwik almost all memory, and my regular website low memory.

But else, I don’t have a clue…


#6

Thanks for your reply.

On the next run of the cronjob the error message about “memory_limit” didn’t appear anymore. Instead I received the same error message as I wrote in my first post of this thread.

I give PHP 2048M of memory for this and I would be quite surprised if this wouldn’t be enough. :wink:


#7

I agree, I only use 128 MB for PHP for processing 10.000.000+ loglines/requests a day.

Something webserver write PHP errors in the webserver logfiles. Maybe a clue there?


(Alan) #8

Hi,

May or may not be related to my issue 301 Moved Permanently but your new error message is the same as I am getting


#9

Hi Piwik Team,
I am using PIWIK for last 7 months but I didn’t face a single problem. I love this tool. Thanks a lot. I am also trying to contribute to this tool.

For the past 23 days I am getting this error during the archiving process.
OUTPUT:-
ERROR CoreConsole[2014-11-27 11:23:43] [6a516] Got invalid response from API request: Domain.com. Response was ‘a:2:{s:6:“result”;s:5:“error”;s:7:“message”;s:48:“Mysqli prepare error: MySQL server has gone away”;}’

ERROR CoreConsole[2014-11-27 11:50:19] [6a516] Got invalid response from API request: Domain.com. Response was ‘a:2:{s:6:“result”;s:5:“error”;s:7:“message”;s:48:“Mysqli prepare error: MySQL server has gone away”;}’

For this I update the cron time to run every 8 hours from every 2 hours.
I have PHP 5.6+ with memory limit = 3072M
MySQL 5.6+ with innodb memory limit = 7G
Nginx 1.6.1 + PHP-FPM
PIWIK 2.8.3

I tried:-
php.ini conf changes:-
date.timezone = "UTC"
max_execution_time = 0
max_input_time = -1
Nginx.conf changes:-
fastcgi_buffers 256 4k
fastcgi_read_timeout 600

I tried almost all the replies in this forum, but I am unable to rectify it. This is happening after 5 days of updating to PIWIK 2.8.3.

I also tried to get the response use wget. the output as follows:-

wget “Domain.com
–2014-11-27 17:31:11-- Domain.com
Resolving stats.indiaproperty.com… 10.72.177.157
Connecting to stats.indiaproperty.com|10.72.177.157|:80… connected.
HTTP request sent, awaiting response… 502 Bad Gateway
2014-11-27 17:33:25 ERROR 502: Bad Gateway.

I will be waiting for the precious reply from you all.

Thanks in advance. :slight_smile:


#10

MySQL timeouts? What are the settings of the /etc/my.cnf file? Could be that Piwik needs more time do its calculation, and that MySQL is closing the connection based on a timeout.

TheYOSH


#11

I did some research just yesterday and yes, it seems that there is a timeout in MySQL.

I can’t solve this because I don’t have permission to edit my.cnf.


#12

Hmm, then I don’t know how to fix this. I have my own dedicated MySQL server, so I was able to tune it to Piwik needs.

Could it help to run the archiver every 30 minutes. So the amount of data to update is smaller? Not sure if that will help.


#13

At first thanks for your replys! :slight_smile:

The cron job runs every hour. Yesterday I tried to run the Archiver manually from the console between the cronjobs. This seems to work but the cronjob still does have the timeouts.

I’m waiting for a statement from our hoster to see whether they can change my.cnf.


#14

In my setup I have the my.cnf timeouts as:-
wait_timeout= 7800
interactive_timeout = 7800


#15

Please can any one help me!!!