Archiving problem. Please help


#1

Hey there,

i use piwik 1.4 and i have a problem concerning the archiving process.

Since i changed from the archiving triggered by the webinterface to using a cronjob i have some problems. That’s not quite right. The problems started when i started adding more and more sites to piwik. I did some research and found out that i can’t execute the archive.sh at my server for larger installations. The cronjob stops at one point and doesnt finish execution. But that has nothing to do with piwik.

So i found a php alternative to the archive.sh in this forum. When i call this script in my browser the xml output seems to be what i need. But however no data is written to the database. Although i know that some sites had traffic today for example the output doesnt show that. In this way the data in my webinterface isn’t right. When i switsch back to the default archiving process (for small installations) everything works fine.

I attached the php script that was written by Dave (member of this forum):


<?php
define('PIWIK_INCLUDE_PATH', preg_replace('!^(.*)'.DIRECTORY_SEPARATOR.'misc'.DIRECTORY_SEPARATOR.'cron$!i', '$1', dirname(__FILE__)).DIRECTORY_SEPARATOR);
define('PIWIK_USER_PATH', PIWIK_INCLUDE_PATH);
define('PIWIK_ENABLE_DISPATCH', false);
define('PIWIK_ENABLE_ERROR_HANDLER', false);
define('PIWIK_ENABLE_SESSION_START', false);

$ini_file = PIWIK_INCLUDE_PATH.'config'.DIRECTORY_SEPARATOR.'config.ini.php';
if (!file_exists($ini_file))
{
	echo "Piwik configuration file $ini_file not found!\n";
	exit(1);
}


$PIWIK_SUPERUSER = 'xxx';
$PIWIK_SUPERUSER_MD5_PASSWORD = 'xxx';

if (empty($PIWIK_SUPERUSER) || empty($PIWIK_SUPERUSER_MD5_PASSWORD))
{
	echo "Could not extract PIWIK_SUPERUSER and/or PIWIK_SUPERUSER_MD5_PASSWORD!\n";
	exit(1);
}

require_once PIWIK_INCLUDE_PATH . "/index.php";
require_once PIWIK_INCLUDE_PATH . "/core/API/Request.php";

Piwik_FrontController::getInstance()->init();
$request = new Piwik_API_Request("method=UsersManager.getTokenAuth&userLogin=$PIWIK_SUPERUSER&md5Password=$PIWIK_SUPERUSER_MD5_PASSWORD&format=php&serialize=0" );
$TOKEN_AUTH = trim($request->process());
if (empty($TOKEN_AUTH))
{
	echo "Could not get TOKEN_AUTH, unable to continue.\n";
	exit(1);
}

$request = new Piwik_API_Request("method=SitesManager.getAllSitesId&token_auth=$TOKEN_AUTH&format=php&serialize=1" );
$ID_SITES = unserialize($request->process());
if (!count($ID_SITES))
{
	echo "ID_SITES does not contain any sites to archive!\n";
	exit(1);
}

$request = new Piwik_API_Request("method=CoreAdminHome.getKnownSegmentsToArchive&token_auth=$TOKEN_AUTH&format=php&serialize=1" );
$SEGMENTS_TO_ARCHIVE = unserialize($request->process());

$periods = array('day', 'week', 'month', 'year');
foreach ($ID_SITES as $id_sites_key => $id_sites_value)
{
	$idsite = trim($id_sites_value[0]);
	if (!$idsite || !ctype_digit($idsite))
	{
		echo "Skipping idsite = $idsite (must be a number to process)\n";
		continue;
	}
	
	foreach ($periods as $period)
	{
		echo "\n";
		echo "Archiving period = $period for idsite = $idsite...\n";
		$request = new Piwik_API_Request("method=VisitsSummary.getVisits&idSite=$idsite&period=$period&date=last52&format=xml&token_auth=$TOKEN_AUTH" );
		echo $request->process();
		echo "\n";
		
		
		if( !empty( $SEGMENTS_TO_ARCHIVE) )
		{
			foreach ($SEGMENTS_TO_ARCHIVE as $segments_to_archive_key => $segments_to_archive_value)
			{
				$segment = $segments_to_archive_value[0];
				if ($segment != 'value')
				{
					echo "\n";
					echo " - Archiving for visitor segment $segment...\n";
					$request = new Piwik_API_Request("method=VisitsSummary.getVisits&idSite=$idsite&period=$period&date=last52&format=xml&token_auth=$TOKEN_AUTH&segment=$segment" );
					echo $request->process();
					echo "\n";
				}
			}
		} else {
			echo "Segments are empty...\n\n";
		}
		
		
	}
	echo "Archiving for idsite = $idsite done!\n";
}

echo "Reports archiving finished.\n";
echo "---------------------------\n";
echo "Starting Scheduled tasks...\n";
echo "\n";

$request = new Piwik_API_Request("method=CoreAdminHome.runScheduledTasks&format=xml&convertToUnicode=0&token_auth=$TOKEN_AUTH" );
echo $request->process();
echo "\n";

echo "\n";
echo "Finished Scheduled tasks.\n";
echo "\n";

?>