Error with import_logs.py


#1

Hi, i have the following issue. I run the following command and the script tells me that everything is fine:


PS D:\www\piwik\misc\log-analytics> python ./import_logs.py --url="http://myserver.de/piwik" --replay-t
racking "D:/www/sorttest.log" --idsite=2 --recorders=8
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log D:/www/sorttest.log...
440 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
440 lines parsed, 91 lines recorded, 44 records/sec (avg), 91 records/sec (current)
440 lines parsed, 165 lines recorded, 54 records/sec (avg), 74 records/sec (current)
440 lines parsed, 165 lines recorded, 40 records/sec (avg), 0 records/sec (current)
440 lines parsed, 336 lines recorded, 66 records/sec (avg), 171 records/sec (current)
440 lines parsed, 367 lines recorded, 60 records/sec (avg), 31 records/sec (current)

Purging Piwik archives for dates: 2014-04-22

To re-process these reports with your newly imported data, execute the following command:
$ /path/to/piwik/console core:archive --url=http://example/piwik --force-all-websites --force-all-periods=315576000 --fo
rce-date-last-n=1000
Reference: http://piwik.org/docs/setup-auto-archiving/

Logs import summary
-------------------

    439 requests imported successfully
    0 requests were downloads
    1 requests ignored:
        0 HTTP errors
        0 HTTP redirects
        1 invalid log lines
        0 requests did not match any known site
        0 requests did not match any --hostname
        0 requests done by bots, search engines...
        0 requests to static resources (css, js, images, ico, ttf...)
        0 requests to file downloads did not match any --download-extensions

Website import summary
----------------------

    439 requests imported to 1 sites
        1 sites already existed
        0 sites were created:

    0 distinct hostnames did not match any existing site:



Performance summary
-------------------

    Total time: 6 seconds
    Requests imported per second: 70.24 requests per second

The problem is that i cannot see any of the requests in the webinterface. its like they havent been logged at all. i even started a new website/project to make sure.

Then i run this archive command to archive as advised:


PS D:\www\piwik> php ./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000
--url=http://myserver.de/piwik/
INFO CoreConsole[2014-11-28 15:49:06] [423ac] ---------------------------
INFO CoreConsole[2014-11-28 15:49:06] [423ac] INIT
INFO CoreConsole[2014-11-28 15:49:06] [423ac] Piwik is installed at: http://myserver.de/piwik/index.php

INFO CoreConsole[2014-11-28 15:49:06] [423ac] Running Piwik 2.9.1 as Super User
INFO CoreConsole[2014-11-28 15:49:06] [423ac] ---------------------------
INFO CoreConsole[2014-11-28 15:49:06] [423ac] NOTES
INFO CoreConsole[2014-11-28 15:49:06] [423ac] - If you execute this script at least once per hour (or more often) in a c
rontab, you may disable 'Browser trigger archiving' in Piwik UI > Settings > General Settings.
INFO CoreConsole[2014-11-28 15:49:06] [423ac]   See the doc at: http://piwik.org/docs/setup-auto-archiving/
INFO CoreConsole[2014-11-28 15:49:06] [423ac] - Reports for today will be processed at most every 150 seconds. You can c
hange this value in Piwik UI > Settings > General Settings.
INFO CoreConsole[2014-11-28 15:49:06] [423ac] - Reports for the current week/month/year will be refreshed at most every
3600 seconds.
INFO CoreConsole[2014-11-28 15:49:06] [423ac] - Archiving was last executed without error 23 min 20s ago
INFO CoreConsole[2014-11-28 15:49:06] [423ac] - Will process all 2 websites
INFO CoreConsole[2014-11-28 15:49:06] [423ac] ---------------------------
INFO CoreConsole[2014-11-28 15:49:06] [423ac] START
INFO CoreConsole[2014-11-28 15:49:06] [423ac] Starting Piwik reports archiving...
INFO CoreConsole[2014-11-28 15:49:15] [423ac] Archived website id = 1, period = day, 5371 visits in last last1000 days,
62 visits today, Time elapsed: 8.544s
INFO CoreConsole[2014-11-28 15:49:21] [423ac] Archived website id = 1, period = week, 5371 visits in last last1000 weeks
, 456 visits this week, Time elapsed: 6.111s
INFO CoreConsole[2014-11-28 15:49:30] [423ac] Archived website id = 1, period = month, 5371 visits in last last1000 mont
hs, 2261 visits this month, Time elapsed: 8.710s
INFO CoreConsole[2014-11-28 15:49:34] [423ac] Archived website id = 1, period = year, 5371 visits in last last1000 years
, 5371 visits this year, Time elapsed: 3.834s
INFO CoreConsole[2014-11-28 15:49:34] [423ac] Archived website id = 1, 4 API requests, Time elapsed: 27.223s [1/2 done]
INFO CoreConsole[2014-11-28 15:49:40] [423ac] Archived website id = 2, period = day, 0 visits in last last1000 days, 0 v
isits today, Time elapsed: 6.730s
INFO CoreConsole[2014-11-28 15:49:44] [423ac] Archived website id = 2, period = week, 0 visits in last last1000 weeks, 0
 visits this week, Time elapsed: 3.972s
INFO CoreConsole[2014-11-28 15:49:47] [423ac] Archived website id = 2, period = month, 0 visits in last last1000 months,
 0 visits this month, Time elapsed: 2.434s
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Archived website id = 2, period = year, 0 visits in last last1000 years, 0
 visits this year, Time elapsed: 1.022s
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Archived website id = 2, 4 API requests, Time elapsed: 14.185s [2/2 done]
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Done archiving!
INFO CoreConsole[2014-11-28 15:49:48] [423ac] ---------------------------
INFO CoreConsole[2014-11-28 15:49:48] [423ac] SUMMARY
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Total visits for today across archived websites: 62
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Archived today's reports for 2 websites
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Archived week/month/year for 2 websites
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Skipped 0 websites: no new visit since the last script execution
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Skipped 0 websites day archiving: existing daily reports are less than 150
 seconds old
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Skipped 0 websites week/month/year archiving: existing periods reports are
 less than 3600 seconds old
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Total API requests: 8
INFO CoreConsole[2014-11-28 15:49:48] [423ac] done: 2/2 100%, 62 vtoday, 2 wtoday, 2 wperiods, 8 req, 41423 ms, no error

INFO CoreConsole[2014-11-28 15:49:48] [423ac] Time elapsed: 41.424s
INFO CoreConsole[2014-11-28 15:49:48] [423ac] ---------------------------
INFO CoreConsole[2014-11-28 15:49:48] [423ac] SCHEDULED TASKS
INFO CoreConsole[2014-11-28 15:49:48] [423ac] Starting Scheduled tasks...
INFO CoreConsole[2014-11-28 15:49:48] [423ac]  No task to run
INFO CoreConsole[2014-11-28 15:49:48] [423ac] done
INFO CoreConsole[2014-11-28 15:49:48] [423ac] ---------------------------

Still the same result? anything i can try? does anyone have the same problem?

thanks in advance

Stephan


(Matthieu Aubry) #2

Hi, It looks like you have data for idsite=1 - dont you see the data in the repotrs?


#3

hi,
i have data in idsite=1, but that data was already there and does not come from the imported log. as you can see i replayed the logs to the idsite=2. this is a totally empty project in piwik. the data does not show there at all, not in this log, not via the webinterface.


#4

Do you mind answering this? I dont see the Data i should have just imported, plz help!


(Matthieu Aubry) #5

Hi there, Did you setup the cron job as explained in: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo ? this may help