Web site import log not showing stat in matomo


#1

Hello,

i configure 2 web site in piwik 3.4.0, when i’m importing the log file for the first site everything work find i get my stat showin in piwik and no error with the import part.

But when doing the import of the second site with no error also, i got nothing showing in piwik console for the stat.

What could be the problem ?

I have install piwik 3.5.0 but same result.

Hope you can help !

regards,

Stephane


#2

Hello,

i did not have any feedback on my question from anyone.

I try yesterday to update to the 3.5.1 version
Ialso empty allt stat and reload it , no change still have the second site not having any stats show in piwik.
I then try to delete and recrete the second site to generate a new Id and then load 1 month stats for the new site same rase after reloading the stats.

The first site show the stat in piwik but not the second one.

Hope someone can help me or i will have to stop doing my POC on piwik and use a software that’s works.

Regards,

Stephane


(Peterbo) #3

I highly doubt that this is a software issue. Rather more an issue with what you’re trying to accomplish or false usage. I will never get, why people who are using a free software demand free support on top of it or else “threaten” to abandon it and use something else with mostly paid support. This is utterly ridiculous when you think that through…

On topic… what are you importing exactly? Replay logs (with requests like “piwik.php?action_name=test123”) or “native” apache logfiles?

Try to tell us exactly what you did - we’re no shamans. What is your exact import command? Show us an example line from your logfile, etc.


#4

Hello Peterbo,

I don’t have problem not having support 7/24 on opensource product but when you want to develop the OpenSource community you have to at least take time to answer them.

I open this question 2 weeks ago !
i’m charging my client to implement piwik, and try to use the community to help me solve installation issue so i expect to have at least some feedback. I’m a big Opensource user but i can not loose too much time at a client site implementing a solution that they won’t have feedback from the community.

We can at least expect to have some feedback but i agree with you that the support can not be the same has paid support or proprietary software.

So that’s said here is what i’m trying to do.

I install the hosted version on Oracle Linux 6.8 (it’s the same Linux has Redhat)
i just install the 3.5.1 version
the web site that i’m trying to load the web log are on Miscroft IIS web server.

I create 2 site for which i imported the monthly log into piwik
the import gave me no error and seem to finish succesfully

i use this string to import one for each site

python /data/www/piwik-lab/misc/log-analytics/import_logs.py --url=https://piwik-lab.domainename --login=piwikadm@piwik-lab.domainename --idsite=12 /Log/*.log

python /data/www/piwik-lab/misc/log-analytics/import_logs.py --url=https://piwik-lab.domainename --login=piwikadm@piwik-lab.domainename --idsite=3 /Log/*.log

after that i do the refresh script and it’s finish with no error also

./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=‘https://piwik-lab.domainename

when i look at the site id=3 i got data loading
Archived website id = 3, period = day, 0 segments, 271132 visits in last 1000 days, 0 visits today, Time elapsed: 2.510s

but when i look at the site id=12 i got nothnig but i know their is
Archived website id = 12, period = day, 0 segments, 0 visits in last 1000 days, 0 visits today, Time elapsed: 0.718s

it’s was working at first ant then it’s stop for no reason

i start with the version 3.3.0 and got no problem with this one but since version 3.4.0 up to 3.5.1 it’s not working anymore, only for the first site.

Hope you can put me in the right direction to solve this issue.

Regards,

Stephane


(Peterbo) #5

This is not something the community must provide in any case (you’re earning money with free software and a volunteer is losing opportunity cost, helping you here?). This is all a “can” not a “must”. If you need urgent support, we also offer fast paid consultancy. Your reaction is just contra productive on any level.

On topic now.

Please post a sample log line, as well. I’m sure there is a reason why there is no import to site-id 12. Do you have a full import result report to post here?


#6

Here is the full import result (i hadd to remove some line because of the restriction of number of caracter to post here)

0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log /LogDcom/ex171201.log...
17462 lines parsed, 200 lines recorded, 199 records/sec (avg), 200 records/sec (current)
25521 lines parsed, 1000 lines recorded, 499 records/sec (avg), 800 records/sec (current)
31504 lines parsed, 1600 lines recorded, 532 records/sec (avg), 600 records/sec (current)
36936 lines parsed, 2400 lines recorded, 599 records/sec (avg), 800 records/sec (current)
47814 lines parsed, 3400 lines recorded, 679 records/sec (avg), 1000 records/sec (current)
59092 lines parsed, 4000 lines recorded, 665 records/sec (avg), 600 records/sec (current)
60057 lines parsed, 5000 lines recorded, 713 records/sec (avg), 1000 records/sec (current)
66624 lines parsed, 5800 lines recorded, 723 records/sec (avg), 800 records/sec (current)
71796 lines parsed, 6600 lines recorded, 732 records/sec (avg), 800 records/sec (current)
81043 lines parsed, 7400 lines recorded, 739 records/sec (avg), 800 records/sec (current)
89626 lines parsed, 8200 lines recorded, 744 records/sec (avg), 800 records/sec (current)
107161 lines parsed, 9000 lines recorded, 749 records/sec (avg), 800 records/sec (current)
117176 lines parsed, 9800 lines recorded, 752 records/sec (avg), 800 records/sec (current)
130215 lines parsed, 10400 lines recorded, 741 records/sec (avg), 600 records/sec (current)
137858 lines parsed, 11200 lines recorded, 745 records/sec (avg), 800 records/sec (current)
143971 lines parsed, 12000 lines recorded, 749 records/sec (avg), 800 records/sec (current)
150269 lines parsed, 12800 lines recorded, 752 records/sec (avg), 800 records/sec (current)
151903 lines parsed, 13800 lines recorded, 765 records/sec (avg), 1000 records/sec (current)
152968 lines parsed, 14800 lines recorded, 778 records/sec (avg), 1000 records/sec (current)
154499 lines parsed, 16000 lines recorded, 799 records/sec (avg), 1200 records/sec (current)
157075 lines parsed, 16800 lines recorded, 799 records/sec (avg), 800 records/sec (current)
168343 lines parsed, 17800 lines recorded, 808 records/sec (avg), 1000 records/sec (current)
174040 lines parsed, 18600 lines recorded, 807 records/sec (avg), 800 records/sec (current)
174883 lines parsed, 19400 lines recorded, 807 records/sec (avg), 800 records/sec (current)
182878 lines parsed, 20400 lines recorded, 815 records/sec (avg), 1000 records/sec (current)
187874 lines parsed, 21200 lines recorded, 814 records/sec (avg), 800 records/sec (current)
198884 lines parsed, 22000 lines recorded, 813 records/sec (avg), 800 records/sec (current)
213627 lines parsed, 22800 lines recorded, 813 records/sec (avg), 800 records/sec (current)
226611 lines parsed, 23400 lines recorded, 805 records/sec (avg), 600 records/sec (current)
Parsing log /LogDcom/ex171202.log...
236527 lines parsed, 24114 lines recorded, 802 records/sec (avg), 714 records/sec (current)
246354 lines parsed, 24914 lines recorded, 802 records/sec (avg), 800 records/sec (current)
260288 lines parsed, 25514 lines recorded, 796 records/sec (avg), 600 records/sec (current)
270969 lines parsed, 26114 lines recorded, 790 records/sec (avg), 600 records/sec (current)
287523 lines parsed, 26914 lines recorded, 790 records/sec (avg), 800 records/sec (current)
299516 lines parsed, 27514 lines recorded, 785 records/sec (avg), 600 records/sec (current)
304907 lines parsed, 28314 lines recorded, 785 records/sec (avg), 800 records/sec (current)
319242 lines parsed, 29114 lines recorded, 785 records/sec (avg), 800 records/sec (current)
331837 lines parsed, 29714 lines recorded, 781 records/sec (avg), 600 records/sec (current)
346219 lines parsed, 30314 lines recorded, 776 records/sec (avg), 600 records/sec (current)
360672 lines parsed, 31114 lines recorded, 776 records/sec (avg), 800 records/sec (current)
Parsing log /LogDcom/ex171203.log...
369686 lines parsed, 31821 lines recorded, 775 records/sec (avg), 707 records/sec (current)
380361 lines parsed, 32421 lines recorded, 771 records/sec (avg), 600 records/sec (current)
397397 lines parsed, 33221 lines recorded, 771 records/sec (avg), 800 records/sec (current)
408866 lines parsed, 34021 lines recorded, 772 records/sec (avg), 800 records/sec (current)
425743 lines parsed, 34621 lines recorded, 768 records/sec (avg), 600 records/sec (current)
441955 lines parsed, 35221 lines recorded, 764 records/sec (avg), 600 records/sec (current)
453891 lines parsed, 36021 lines recorded, 765 records/sec (avg), 800 records/sec (current)
462182 lines parsed, 36621 lines recorded, 762 records/sec (avg), 600 records/sec (current)
479352 lines parsed, 37421 lines recorded, 762 records/sec (avg), 800 records/sec (current)
492857 lines parsed, 38021 lines recorded, 759 records/sec (avg), 600 records/sec (current)
510750 lines parsed, 38821 lines recorded, 760 records/sec (avg), 800 records/sec (current)
Parsing log /LogDcom/ex171204.log...
525950 lines parsed, 39421 lines recorded, 757 records/sec (avg), 600 records/sec (current)
537401 lines parsed, 40171 lines recorded, 757 records/sec (avg), 750 records/sec (current)
541870 lines parsed, 40771 lines recorded, 754 records/sec (avg), 600 records/sec (current)
543512 lines parsed, 41571 lines recorded, 754 records/sec (avg), 800 records/sec (current)
548827 lines parsed, 42571 lines recorded, 759 records/sec (avg), 1000 records/sec (current)
557520 lines parsed, 43571 lines recorded, 763 records/sec (avg), 1000 records/sec (current)
560369 lines parsed, 44171 lines recorded, 760 records/sec (avg), 600 records/sec (current)
569126 lines parsed, 45171 lines recorded, 764 records/sec (avg), 1000 records/sec (current)
576430 lines parsed, 45971 lines recorded, 765 records/sec (avg), 800 records/sec (current)
582989 lines parsed, 46771 lines recorded, 765 records/sec (avg), 800 records/sec (current)
587314 lines parsed, 47571 lines recorded, 766 records/sec (avg), 800 records/sec (current)
593921 lines parsed, 48371 lines recorded, 766 records/sec (avg), 800 records/sec (current)
601524 lines parsed, 49171 lines recorded, 767 records/sec (avg), 800 records/sec (current)
607595 lines parsed, 49971 lines recorded, 767 records/sec (avg), 800 records/sec (current)
612142 lines parsed, 50771 lines recorded, 768 records/sec (avg), 800 records/sec (current)
613433 lines parsed, 51571 lines recorded, 768 records/sec (avg), 800 records/sec (current)
620641 lines parsed, 52571 lines recorded, 772 records/sec (avg), 1000 records/sec (current)
634225 lines parsed, 53371 lines recorded, 772 records/sec (avg), 800 records/sec (current)
646598 lines parsed, 53971 lines recorded, 770 records/sec (avg), 600 records/sec (current)
661485 lines parsed, 54571 lines recorded, 767 records/sec (avg), 600 records/sec (current)
663853 lines parsed, 55371 lines recorded, 768 records/sec (avg), 800 records/sec (current)
668460 lines parsed, 56171 lines recorded, 768 records/sec (avg), 800 records/sec (current)
675040 lines parsed, 56971 lines recorded, 769 records/sec (avg), 800 records/sec (current)
681636 lines parsed, 57771 lines recorded, 769 records/sec (avg), 800 records/sec (current)
688740 lines parsed, 58771 lines recorded, 772 records/sec (avg), 1000 records/sec (current)
699485 lines parsed, 59571 lines recorded, 772 records/sec (avg), 800 records/sec (current)
705961 lines parsed, 60371 lines recorded, 773 records/sec (avg), 800 records/sec (current)
711680 lines parsed, 60971 lines recorded, 770 records/sec (avg), 600 records/sec (current)
717244 lines parsed, 61771 lines recorded, 771 records/sec (avg), 800 records/sec (current)
724429 lines parsed, 62771 lines recorded, 773 records/sec (avg), 1000 records/sec (current)
728390 lines parsed, 63571 lines recorded, 774 records/sec (avg), 800 records/sec (current)
738680 lines parsed, 64371 lines recorded, 774 records/sec (avg), 800 records/sec (current)
740255 lines parsed, 65171 lines recorded, 774 records/sec (avg), 800 records/sec (current)
749824 lines parsed, 65971 lines recorded, 775 records/sec (avg), 800 records/sec (current)
766285 lines parsed, 66771 lines recorded, 775 records/sec (avg), 800 records/sec (current)
Parsing log /LogDcom/ex171205.log...
773971 lines parsed, 67371 lines recorded, 773 records/sec (avg), 600 records/sec (current)
780903 lines parsed, 68006 lines recorded, 771 records/sec (avg), 635 records/sec (current)
790720 lines parsed, 68806 lines recorded, 772 records/sec (avg), 800 records/sec (current)
793452 lines parsed, 69606 lines recorded, 772 records/sec (avg), 800 records/sec (current)
796102 lines parsed, 70406 lines recorded, 772 records/sec (avg), 800 records/sec (current)
797060 lines parsed, 71406 lines recorded, 775 records/sec (avg), 1000 records/sec (current)
802788 lines parsed, 72206 lines recorded, 775 records/sec (avg), 800 records/sec (current)
807140 lines parsed, 73006 lines recorded, 775 records/sec (avg), 800 records/sec (current)
810465 lines parsed, 73806 lines recorded, 775 records/sec (avg), 800 records/sec (current)
814295 lines parsed, 74606 lines recorded, 776 records/sec (avg), 800 records/sec (current)
819567 lines parsed, 75406 lines recorded, 776 records/sec (avg), 800 records/sec (current)
833459 lines parsed, 76406 lines recorded, 778 records/sec (avg), 1000 records/sec (current)
841946 lines parsed, 77006 lines recorded, 776 records/sec (avg), 600 records/sec (current)
844741 lines parsed, 77806 lines recorded, 777 records/sec (avg), 800 records/sec (current)
849767 lines parsed, 78806 lines recorded, 779 records/sec (avg), 1000 records/sec (current)
860051 lines parsed, 79606 lines recorded, 779 records/sec (avg), 800 records/sec (current)
867099 lines parsed, 80406 lines recorded, 779 records/sec (avg), 800 records/sec (current)
880738 lines parsed, 81206 lines recorded, 779 records/sec (avg), 800 records/sec (current)
881792 lines parsed, 81806 lines recorded, 778 records/sec (avg), 600 records/sec (current)
886959 lines parsed, 82806 lines recorded, 780 records/sec (avg), 1000 records/sec (current)
896477 lines parsed, 83606 lines recorded, 780 records/sec (avg), 800 records/sec (current)
899186 lines parsed, 84406 lines recorded, 780 records/sec (avg), 800 records/sec (current)
909368 lines parsed, 85206 lines recorded, 780 records/sec (avg), 800 records/sec (current)
915900 lines parsed, 86006 lines recorded, 780 records/sec (avg), 800 records/sec (current)
924779 lines parsed, 86606 lines recorded, 779 records/sec (avg), 600 records/sec (current)
927295 lines parsed, 87406 lines recorded, 779 records/sec (avg), 800 records/sec (current)
930193 lines parsed, 88406 lines recorded, 781 records/sec (avg), 1000 records/sec (current)
931133 lines parsed, 89206 lines recorded, 781 records/sec (avg), 800 records/sec (current)
938002 lines parsed, 90206 lines recorded, 783 records/sec (avg), 1000 records/sec (current)
940284 lines parsed, 91006 lines recorded, 783 records/sec (avg), 800 records/sec (current)
944644 lines parsed, 91806 lines recorded, 783 records/sec (avg), 800 records/sec (current)
951262 lines parsed, 92606 lines recorded, 783 records/sec (avg), 800 records/sec (current)
959663 lines parsed, 93406 lines recorded, 783 records/sec (avg), 800 records/sec (current)
973366 lines parsed, 94206 lines recorded, 784 records/sec (avg), 800 records/sec (current)
978863 lines parsed, 95006 lines recorded, 784 records/sec (avg), 800 records/sec (current)
982808 lines parsed, 95806 lines recorded, 784 records/sec (avg), 800 records/sec (current)
985227 lines parsed, 96606 lines recorded, 784 records/sec (avg), 800 records/sec (current)
992370 lines parsed, 97606 lines recorded, 786 records/sec (avg), 1000 records/sec (current)
999887 lines parsed, 98206 lines recorded, 784 records/sec (avg), 600 records/sec (current)
1015350 lines parsed, 99206 lines recorded, 786 records/sec (avg), 1000 records/sec (current)
1019218 lines parsed, 99806 lines recorded, 784 records/sec (avg), 600 records/sec (current)
1022704 lines parsed, 100806 lines recorded, 786 records/sec (avg), 1000 records/sec (current)
1034284 lines parsed, 101606 lines recorded, 786 records/sec (avg), 800 records/sec (current)
1037745 lines parsed, 102406 lines recorded, 786 records/sec (avg), 800 records/sec (current)
Parsing log /LogDcom/ex171206.log...
1047212 lines parsed, 103206 lines recorded, 786 records/sec (avg), 800 records/sec (current)
1057933 lines parsed, 103831 lines recorded, 785 records/sec (avg), 625 records/sec (current)
1064616 lines parsed, 104431 lines recorded, 784 records/sec (avg), 600 records/sec (current)
1072077 lines parsed, 105231 lines recorded, 784 records/sec (avg), 800 records/sec (current)
1079981 lines parsed, 106031 lines recorded, 784 records/sec (avg), 800 records/sec (current)
1086907 lines parsed, 106831 lines recorded, 784 records/sec (avg), 800 records/sec (current)
1089320 lines parsed, 107831 lines recorded, 786 records/sec (avg), 1000 records/sec (current)
1091537 lines parsed, 108831 lines recorded, 787 records/sec (avg), 1000 records/sec (current)
1096872 lines parsed, 109831 lines recorded, 789 records/sec (avg), 1000 records/sec (current)
1105186 lines parsed, 110631 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1117515 lines parsed, 111431 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1121089 lines parsed, 112231 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1130106 lines parsed, 113031 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1136589 lines parsed, 113831 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1151343 lines parsed, 114631 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1167690 lines parsed, 115231 lines recorded, 788 records/sec (avg), 600 records/sec (current)
1175562 lines parsed, 115831 lines recorded, 786 records/sec (avg), 600 records/sec (current)
1184581 lines parsed, 116631 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1192973 lines parsed, 117431 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1198448 lines parsed, 118231 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1203416 lines parsed, 119031 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1211893 lines parsed, 119831 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1213504 lines parsed, 120831 lines recorded, 788 records/sec (avg), 1000 records/sec (current)
1220306 lines parsed, 121831 lines recorded, 790 records/sec (avg), 1000 records/sec (current)
1234727 lines parsed, 122631 lines recorded, 790 records/sec (avg), 800 records/sec (current)
1240096 lines parsed, 123231 lines recorded, 788 records/sec (avg), 600 records/sec (current)
1244285 lines parsed, 124031 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1252078 lines parsed, 125031 lines recorded, 790 records/sec (avg), 1000 records/sec (current)
1256274 lines parsed, 125631 lines recorded, 789 records/sec (avg), 600 records/sec (current)
1269367 lines parsed, 126631 lines recorded, 790 records/sec (avg), 1000 records/sec (current)
1284900 lines parsed, 127231 lines recorded, 789 records/sec (avg), 600 records/sec (current)
Parsing log /LogDcom/ex171207.log...
1296722 lines parsed, 128031 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1304012 lines parsed, 128731 lines recorded, 788 records/sec (avg), 700 records/sec (current)
1310014 lines parsed, 129331 lines recorded, 787 records/sec (avg), 600 records/sec (current)
1318790 lines parsed, 130331 lines recorded, 788 records/sec (avg), 1000 records/sec (current)
1324674 lines parsed, 131131 lines recorded, 788 records/sec (avg), 800 records/sec (current)
1329812 lines parsed, 131931 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1342728 lines parsed, 132731 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1345883 lines parsed, 133331 lines recorded, 787 records/sec (avg), 600 records/sec (current)
1358256 lines parsed, 134331 lines recorded, 789 records/sec (avg), 1000 records/sec (current)
1370815 lines parsed, 135131 lines recorded, 789 records/sec (avg), 800 records/sec (current)
1379547 lines parsed, 135731 lines recorded, 788 records/sec (avg), 600 records/sec (current)
1391480 lines parsed, 136531 lines recorded, 788 records/sec (avg), 800 records/sec (current)
1402913 lines parsed, 137331 lines recorded, 788 records/sec (avg), 800 records/sec (current)
1411036 lines parsed, 137931 lines recorded, 787 records/sec (avg), 600 records/sec (current)
1418351 lines parsed, 138731 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1422948 lines parsed, 139531 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1433165 lines parsed, 140531 lines recorded, 788 records/sec (avg), 1000 records/sec (current)
1443032 lines parsed, 141331 lines recorded, 788 records/sec (avg), 800 records/sec (current)
1454327 lines parsed, 142131 lines recorded, 788 records/sec (avg), 800 records/sec (current)
1458300 lines parsed, 142731 lines recorded, 787 records/sec (avg), 600 records/sec (current)
1467359 lines parsed, 143531 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1478772 lines parsed, 144331 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1486258 lines parsed, 145131 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1497017 lines parsed, 145931 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1507693 lines parsed, 146731 lines recorded, 787 records/sec (avg), 800 records/sec (current)
1515329 lines parsed, 147331 lines recorded, 786 records/sec (avg), 600 records/sec (current)
Parsing log /LogDcom/ex171208.log...
......

Parsing log /LogDcom/ex171229.log...
4767444 lines parsed, 477668 lines recorded, 778 records/sec (avg), 800 records/sec (current)
4778610 lines parsed, 478212 lines recorded, 777 records/sec (avg), 544 records/sec (current)
4789903 lines parsed, 479012 lines recorded, 777 records/sec (avg), 800 records/sec (current)
4803861 lines parsed, 479812 lines recorded, 777 records/sec (avg), 800 records/sec (current)
4816343 lines parsed, 480412 lines recorded, 777 records/sec (avg), 600 records/sec (current)
4827332 lines parsed, 481012 lines recorded, 777 records/sec (avg), 600 records/sec (current)
4838360 lines parsed, 481812 lines recorded, 777 records/sec (avg), 800 records/sec (current)
4851672 lines parsed, 482612 lines recorded, 777 records/sec (avg), 800 records/sec (current)
4865185 lines parsed, 483212 lines recorded, 777 records/sec (avg), 600 records/sec (current)
Parsing log /LogDcom/ex171230.log...
4875416 lines parsed, 483812 lines recorded, 776 records/sec (avg), 600 records/sec (current)
4887912 lines parsed, 484511 lines recorded, 776 records/sec (avg), 699 records/sec (current)
4899088 lines parsed, 485311 lines recorded, 776 records/sec (avg), 800 records/sec (current)
4911023 lines parsed, 485911 lines recorded, 776 records/sec (avg), 600 records/sec (current)
4920183 lines parsed, 486711 lines recorded, 776 records/sec (avg), 800 records/sec (current)
4928609 lines parsed, 487311 lines recorded, 776 records/sec (avg), 600 records/sec (current)
4939898 lines parsed, 488111 lines recorded, 776 records/sec (avg), 800 records/sec (current)
4953720 lines parsed, 488711 lines recorded, 776 records/sec (avg), 600 records/sec (current)
4962175 lines parsed, 489511 lines recorded, 776 records/sec (avg), 800 records/sec (current)
Parsing log /LogDcom/ex171231.log...
4975718 lines parsed, 490196 lines recorded, 775 records/sec (avg), 685 records/sec (current)
4986096 lines parsed, 490996 lines recorded, 775 records/sec (avg), 800 records/sec (current)
4990250 lines parsed, 491596 lines recorded, 775 records/sec (avg), 600 records/sec (current)
4998451 lines parsed, 492396 lines recorded, 775 records/sec (avg), 800 records/sec (current)
5001475 lines parsed, 493196 lines recorded, 775 records/sec (avg), 800 records/sec (current)
5014681 lines parsed, 493996 lines recorded, 775 records/sec (avg), 800 records/sec (current)
5021788 lines parsed, 494596 lines recorded, 775 records/sec (avg), 600 records/sec (current)
5026447 lines parsed, 495396 lines recorded, 775 records/sec (avg), 800 records/sec (current)
5035549 lines parsed, 496396 lines recorded, 775 records/sec (avg), 1000 records/sec (current)
5040368 lines parsed, 496996 lines recorded, 775 records/sec (avg), 600 records/sec (current)
5043306 lines parsed, 497796 lines recorded, 775 records/sec (avg), 800 records/sec (current)

Logs import summary
-------------------

    498311 requests imported successfully
    375541 requests were downloads
    4544995 requests ignored:
        61923 HTTP errors
        8766 HTTP redirects
        124 invalid log lines
        0 filtered log lines
        0 requests did not match any known site
        0 requests did not match any --hostname
        177395 requests done by bots, search engines...
        4296787 requests to static resources (css, js, images, ico, ttf...)
        0 requests to file downloads did not match any --download-extensions

Website import summary
----------------------

    498311 requests imported to 1 sites
        1 sites already existed
        0 sites were created:

    0 distinct hostnames did not match any existing site:



Performance summary
-------------------

    Total time: 642 seconds
    Requests imported per second: 775.45 requests per second

Processing your log data
------------------------

#7

here is the resul for the refresh of the stats

./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=‘https://piwik-lab.domainename
INFO [2018-06-07 20:52:11] 58754 ---------------------------
INFO [2018-06-07 20:52:11] 58754 INIT
INFO [2018-06-07 20:52:11] 58754 Running Matomo 3.5.1 as Super User
INFO [2018-06-07 20:52:11] 58754 ---------------------------
INFO [2018-06-07 20:52:11] 58754 NOTES
INFO [2018-06-07 20:52:11] 58754 - If you execute this script at least once per hour (or more often) in a crontab, you may disable ‘Browser trigger archiving’ in Matomo UI > Settings > General Settings.
INFO [2018-06-07 20:52:11] 58754 See the doc at: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo
INFO [2018-06-07 20:52:11] 58754 - Reports for today will be processed at most every 150 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2018-06-07 20:52:11] 58754 - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2018-06-07 20:52:11] 58754 - Archiving was last executed without error 1 hours 56 min ago
INFO [2018-06-07 20:52:11] 58754 - Will process all 2 websites
INFO [2018-06-07 20:52:11] 58754 ---------------------------
INFO [2018-06-07 20:52:11] 58754 START
INFO [2018-06-07 20:52:11] 58754 Starting Matomo reports archiving…
INFO [2018-06-07 20:52:11] 58754 Will pre-process for website id = 3, period = day, date = last1000
INFO [2018-06-07 20:52:11] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:13] 58754 Archived website id = 3, period = day, 0 segments, 271132 visits in last 1000 days, 0 visits today, Time elapsed: 2.510s
INFO [2018-06-07 20:52:13] 58754 Will pre-process for website id = 3, period = week, date = last1000
INFO [2018-06-07 20:52:13] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:14] 58754 Archived website id = 3, period = week, 0 segments, 271132 visits in last 1000 weeks, 0 visits this week, Time elapsed: 0.856s
INFO [2018-06-07 20:52:14] 58754 Will pre-process for website id = 3, period = month, date = last1000
INFO [2018-06-07 20:52:14] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:15] 58754 Archived website id = 3, period = month, 0 segments, 271132 visits in last 1000 months, 0 visits this month, Time elapsed: 0.637s
INFO [2018-06-07 20:52:15] 58754 Will pre-process for website id = 3, period = year, date = last1000
INFO [2018-06-07 20:52:15] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:15] 58754 Archived website id = 3, period = year, 0 segments, 271132 visits in last 1000 years, 0 visits this year, Time elapsed: 0.528s
INFO [2018-06-07 20:52:16] 58754 Archived website id = 3, 4 API requests, Time elapsed: 4.563s [1/2 done]
INFO [2018-06-07 20:52:16] 58754 Will pre-process for website id = 12, period = day, date = last1000
INFO [2018-06-07 20:52:16] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:16] 58754 Archived website id = 12, period = day, 0 segments, 0 visits in last 1000 days, 0 visits today, Time elapsed: 0.718s
INFO [2018-06-07 20:52:16] 58754 Will pre-process for website id = 12, period = week, date = last1000
INFO [2018-06-07 20:52:16] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:17] 58754 Archived website id = 12, period = week, 0 segments, 0 visits in last 1000 weeks, 0 visits this week, Time elapsed: 0.616s
INFO [2018-06-07 20:52:17] 58754 Will pre-process for website id = 12, period = month, date = last1000
INFO [2018-06-07 20:52:17] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:17] 58754 Archived website id = 12, period = month, 0 segments, 0 visits in last 1000 months, 0 visits this month, Time elapsed: 0.517s
INFO [2018-06-07 20:52:17] 58754 Will pre-process for website id = 12, period = year, date = last1000
INFO [2018-06-07 20:52:17] 58754 - pre-processing all visits
INFO [2018-06-07 20:52:18] 58754 Archived website id = 12, period = year, 0 segments, 0 visits in last 1000 years, 0 visits this year, Time elapsed: 0.525s
INFO [2018-06-07 20:52:18] 58754 Archived website id = 12, 4 API requests, Time elapsed: 2.378s [2/2 done]
INFO [2018-06-07 20:52:18] 58754 Done archiving!
INFO [2018-06-07 20:52:18] 58754 ---------------------------
INFO [2018-06-07 20:52:18] 58754 SUMMARY
INFO [2018-06-07 20:52:18] 58754 Total visits for today across archived websites: 0
INFO [2018-06-07 20:52:18] 58754 Archived today’s reports for 2 websites
INFO [2018-06-07 20:52:18] 58754 Archived week/month/year for 2 websites
INFO [2018-06-07 20:52:18] 58754 Skipped 0 websites
INFO [2018-06-07 20:52:18] 58754 - 0 skipped because no new visit since the last script execution
INFO [2018-06-07 20:52:18] 58754 - 0 skipped because existing daily reports are less than 150 seconds old
INFO [2018-06-07 20:52:18] 58754 - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2018-06-07 20:52:18] 58754 Total API requests: 8
INFO [2018-06-07 20:52:18] 58754 done: 2/2 100%, 0 vtoday, 2 wtoday, 2 wperiods, 8 req, 7086 ms, no error
INFO [2018-06-07 20:52:18] 58754 Time elapsed: 7.086s
INFO [2018-06-07 20:52:18] 58754 ---------------------------
INFO [2018-06-07 20:52:18] 58754 SCHEDULED TASKS
INFO [2018-06-07 20:52:18] 58754 Starting Scheduled tasks…
INFO [2018-06-07 20:52:18] 58754 Scheduler: executing task Piwik\Plugins\CustomPiwikJs\Tasks.updateTracker…
INFO [2018-06-07 20:52:18] 58754 Scheduler: finished. Time elapsed: 0.001s
INFO [2018-06-07 20:52:18] 58754 Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.anonymizePastData…
INFO [2018-06-07 20:52:18] 58754 Scheduler: finished. Time elapsed: 0.017s
INFO [2018-06-07 20:52:18] 58754 done
INFO [2018-06-07 20:52:18] 58754 ---------------------------


#8

Here is a sample log line in one of the log if you need more let me know

#Software: Microsoft Log Parser
#Version: 1.0
#Date: 2018-05-08 13:50:15
#Fields: LogFilename LogRow date time c-ip cs-username s-sitename s-computername s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken cs-version cs-host cs(User-Agent) cs(Cookie) cs(Referer) s-event s-process-type s-user-time s-kernel-time s-page-faults s-total-procs s-active-procs s-stopped-procs
s:\W3SVC402628196\ex171201.log 5 2017-12-01 05:00:32 142.41.53.108 - - - 142.41.245.17 80 GET /images/acc_12.gif - 200 0 0 913 904 15 HTTP/1.1 www2.mern.gouv.qc.ca Mozilla/5.0+(iPad;+CPU+OS+9_3_5+like+Mac+OS+X)+AppleWebKit/601.1.46+(KHTML,+like+Gecko)+Version/9.0+Mobile/13G36+Safari/601.1 -

Regards,

Stephane


(Peterbo) #9

Thanks for the additional Info. I’ll have a look at it on the weekend.


#10

thanks really appreciated

Regards

Stephane


(Peterbo) #11

Doesn’t look too bad so far. More questions:

  • When you setup the website in the Matomo GUI, did you include the hostname(s) of the tracked actions in the logfile (for each Site) correctly? Did you check the option to “Only track visits and actions when the action URL starts with one of the above URLs.”?
  • Is the IDSite definitely correct?
  • Did you use the super-user login credentials?

You can check if data was imported without archiving it. Just have a look in the visitor log on a date where must be some data for sure. The visitor log is created from raw data.


#12

Hello Peterbo,

on friday i decide to try something else.

I delete my database
i delete both web site in piwik
delete piwik 3.5.0 and install 3.5.1
Configure Matomo 3.5.1
recreate both website wich recreate new Id site ==now 3 and 6
did an import of one month of stat for each web site

Now it’s working again !, i will reimport all stats for both site and see if the problem comeback again.

I will keep you post on this one.

Regards,

Stephane


(Peterbo) #13

Sounds good! Nevertheless I still think that it was just a minor configuration issue. This is a heavily used feature and there are no (bigger) issues known with this (or the last) major version(s).


#14

I did’nt change anything in the configuration and I have documented every step of my installation process.
But i will do more testing and gave you some feedback, if i am able to replicate what causing the import to stop working for this site.

Thanks for the help , really appreciate it.

regards,

Stephane