Greetings,
I am a newbie to Piwik. I seem to be having an issue with loading data from access files into the application.
Let me start by saying that this all used to work, then the last days it seems not to be visible importing the data and running the archive.
Below is my import method that I have been using. I have this set as a cronjob that runs everyday at 4 in the morning. after all the previous days files have been placed into the local $LOGDIR location.
#!/bin/bash
YEAR=$(date --date=‘yesterday’ ‘+%Y’)
MONTH=$(date --date=‘yesterday’ ‘+%m’)
DAY=$(date --date=‘yesterday’ ‘+%d’)
LOGDIR=/tmp/parselogs.$DAY.$MONTH
LOG=$YEAR.$MONTH.$DAY.piwik.stats.log
echo “Parsing started” >> $LOG
date >> $LOG
for HOUR in 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23; do
sudo /var/www/piwik/misc/log-analytics/import_logs.py
–enable-http-errors --enable-http-errors --enable-http-redirects
–exclude-path-from=/home/piwik/excludepaths.txt --url=https://stats.globalcollect.com/piwik/ --idsite=1
$LOGDIR/access_na.gcsip.com*${YEAR}${MONTH}${DAY}${HOUR}*
sleep 10
done
echo “Parsing Completed” >> $LOG
date >> $LOG
Looking through the log files I cannot see any issues with mysql, php, or system logs. After the archive is completed I do not see the that the data uploaded is visible within the stats pages?
I am completely lost as to why this is not visible.
Can anyone please point me into the correct direction.
Below are my server specs.
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU E7330 @ 2.40GHz
stepping : 11
microcode : 0xb6
cpu MHz : 2393.890
cache size : 3072 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx lm constant_tsc arch_perfmon pebs bts nopl tsc_reliable aperfmperf pni ssse3 cx16 hypervisor lahf_lm dtherm
bogomips : 4787.78
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU E7330 @ 2.40GHz
stepping : 11
microcode : 0xb6
cpu MHz : 2393.890
cache size : 3072 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx lm constant_tsc arch_perfmon pebs bts nopl tsc_reliable aperfmperf pni ssse3 cx16 hypervisor lahf_lm dtherm
bogomips : 4787.78
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU E7330 @ 2.40GHz
stepping : 11
microcode : 0xb6
cpu MHz : 2393.890
cache size : 3072 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx lm constant_tsc arch_perfmon pebs bts nopl tsc_reliable aperfmperf pni ssse3 cx16 hypervisor lahf_lm dtherm
bogomips : 4787.78
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel® Xeon® CPU E7330 @ 2.40GHz
stepping : 11
microcode : 0xb6
cpu MHz : 2393.890
cache size : 3072 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx lm constant_tsc arch_perfmon pebs bts nopl tsc_reliable aperfmperf pni ssse3 cx16 hypervisor lahf_lm dtherm
bogomips : 4787.78
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management:
piwik@Piwik:~$ free
total used free shared buffers cached
Mem: 16435500 3046336 13389164 0 50516 272644
-/+ buffers/cache: 2723176 13712324
Swap: 524284 0 524284
piwik@Piwik:~$ cat /proc/meminfo
MemTotal: 16435500 kB
MemFree: 13389140 kB
Buffers: 50524 kB
Cached: 272656 kB
SwapCached: 0 kB
Active: 2640264 kB
Inactive: 230848 kB
Active(anon): 2580712 kB
Inactive(anon): 800 kB
Active(file): 59552 kB
Inactive(file): 230048 kB
Unevictable: 0 kB
Mlocked: 0 kB
SwapTotal: 524284 kB
SwapFree: 524284 kB
Dirty: 4 kB
Writeback: 0 kB
AnonPages: 2547972 kB
Mapped: 54524 kB
Shmem: 33576 kB
Slab: 36204 kB
SReclaimable: 20952 kB
SUnreclaim: 15252 kB
KernelStack: 1480 kB
PageTables: 12276 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 8742032 kB
Committed_AS: 6156380 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 303504 kB
VmallocChunk: 34359431420 kB
HardwareCorrupted: 0 kB
AnonHugePages: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 61440 kB
DirectMap2M: 16715776 kB
piwik@Piwik:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 12.04.3 LTS
Release: 12.04
Codename: precise
mysql> SHOW VARIABLES LIKE “%version%”;
±------------------------±------------------------+
| Variable_name | Value |
±------------------------±------------------------+
| innodb_version | 5.5.34 |
| protocol_version | 10 |
| slave_type_conversions | |
| version | 5.5.34-0ubuntu0.12.04.1 |
| version_comment | (Ubuntu) |
| version_compile_machine | x86_64 |
| version_compile_os | debian-linux-gnu |
±------------------------±------------------------+
I hope that I have covered all the aspects.
Thanks
Lawrence
