Not all Websites are porcessed without --force-all-websites

Support & Bugs
#1

I have a cronjob for the archive-thing


3 *     * * *   www-data        php /var/www/piwik.ibumedia.de/misc/cron/archive.php --url=http://piwik.ibumedia.de --force-all-websites > /dev/null

Now the --force-all-websites is marked as deprecated and I tried it without the parameter. But without the parameter not all of my Websites are processed and after a day the half of them show zero visits, pageimpressions, progression and a zero chart in all piwik pages. After forcing all websites all information is back.

Why is this?

Thanks for Help

#2

the script should detect sits with new visits and automatically archive those

  • it should archive each site’s data at midnight in the site timezone

Is it not working for you?

if not can you send 24 logs showing the full log for 1 day (change > to >> in cron to append to file)

#3

the may be right about new visits but it should not flatline the entire history of the website.

log will follow on monday

#4

no it should definitely not flat line the entire history, I’m surprised it does that to you ?

#5

Here we go.

http://pastebin.de/33326

The log from Saturday til Monday. On Monday Morning half the log in “All Websites” were flattened (Visit 0, PI 0 chart down to 0)

All Archiving is done without the --force-all-websites except the last one.

So why are the charts flattened?

#6

Thanks. can you tell me, what websites IDs were appearing flattened in the all websites (or just a few of them)

and also at what time (corresponding UTC in the log file) did you see them with 0 values ?

#7

I can’t give you a exact time. If i remove --force-all-websites the stats are flattened in the morning.

try site-id 39

greetings
moe

#8

Thanks it helps. Maybe I found a bug after reviewing the code.

Could you try patching as follows:


--- core/ArchiveProcessing.php	(revision 750b789d51aef79f82dd322599ee4ad3e0ef2276)
+++ core/ArchiveProcessing.php	(revision )
@@ -450,7 +450,7 @@
 			if($this->isArchivingDisabled())
 			{
 				$timezone = $this->site->getTimezone();
-				$minDatetimeArchiveProcessedUTC = Piwik_Date::factory(Piwik_Date::factory('now', $timezone)->getDateStartUTC())->setTimezone($timezone)->getTimestamp();
+				$minDatetimeArchiveProcessedUTC = Piwik_Date::factory(Piwik_Date::factory('yesterday', $timezone)->getDateStartUTC())->setTimezone($timezone)->getTimestamp();
 			}
 		}
 		return $minDatetimeArchiveProcessedUTC;

wait 1 day or 2 and try again and see if it works better?

If so, please make sure to create a ticket at dev.piwik.org explaining it fixed for you so I don’t forget to commit the change, which I should be safe.

#9

I’m sorry to report

bug still exists.

while i created a few screenshots for you for easier understanding i dug a little deeper inside piwik and found funny stuff like there are 73 sites configured in Piwik > Administration > Websites but only 70 are shown in Piwik > All Webistes (logged in as admin)

For example site 36 which is shown in archive-run without --force-all-webistes but disapears in archive-run with --force-all-webistes. It also isn’t found on page 2 of Piwik > All Webistes in archive-run with --force-all-webistes

Very strange.

Log from yesterday till today.
http://pastebin.de/33492

affected id’s 73, 34, 35, 36, 2, 38, 25, 39, 23, 40, 43, 44, 1, 17, 16, 45, 47, 48, 49, 50, 10, 9, 53, 54, 55, 59, 61, 65, 37, 18

hope you’ll find the problem

thanks for help!

#10

so just double checked there are only 70 webs configured and displayed in Piwik > Administration > Websites.

but it still stands that site 36 is shown in archive-run without --force-all-webistes and not in archive-run with --force-all-webistes

I also noticed that site 73 is shown on both pages in piwik_with_force_all_webs

#11

73 is just the ID of the website. You probably had 3 websites that were deleted. IDs are not reused.

#12

[quote=“Fabian Becker”]
73 is just the ID of the website. You probably had 3 websites that were deleted. IDs are not reused.[/quote]

jop. as I said I rechecked that