1 MB a day?

Support & Bugs
1

Hi!

I running piwik at my site http://www.towerdefensehq.de/
There are above 3.000 Visitors a day. Tracking is fine and seem to have data of the same quality as Google Analytics which I used before.

But the amount of collected data bugs me a bit. TDHGQ collects steadily 1MB each day. 30 Days equals 30 MB.

I fear if I would add all of our customers to the piwik database I would collect a too large amount of data.

Is this normal? Can this amount be reduced? More sites will hash more data and the database will be smaller or something?

2

Hi there,
Currently Piwik keeps all logs in the database. This is because Piwik is still beta, and we are still adding new reports in the UI, we need the raw data to process these reports. In the future (before being stable) we will add automatic log pruning, which will greatly decrease database usage by getting rid of old logs.

In the meantime, you can of course manually delete old records in the tables piwik_log_* but this will require some sql magic. Otherwise, stay tuned, expect an update early 2009.

3

Ah! relief…

I wasn’t the only one complaining about usage volume style_emoticons/<#EMO_DIR#>/tongue.gif

That’s why Google is huge on data storage…

George

4

piwik_access 1,0 KiB
piwik_archive_blob_2008_11 4,6 MiB
piwik_archive_blob_2008_12 736,0 KiB
piwik_archive_numeric_2008_11 38,5 KiB
piwik_archive_numeric_2008_12 11,1 KiB
piwik_logger_api_call 1,0 KiB
piwik_logger_error 1,0 KiB
piwik_logger_exception 1,0 KiB
piwik_logger_message 1,0 KiB
piwik_log_action 89,7 KiB
piwik_log_link_visit_action 9,5 MiB
piwik_log_profiling 1,0 KiB
piwik_log_visit 18,6 MiB
piwik_option 2,9 KiB
piwik_site 2,1 KiB
piwik_site_url 1,0 KiB
piwik_user 3,1 KiB
piwik_user_dashboard 2,3 KiB
piwik_user_language 1,0 KiB
Sum: 33,6 MiB

piwik_log_action 89,7 KiB
piwik_log_link_visit_action 9,5 MiB
piwik_log_profiling 1,0 KiB
piwik_log_visit 18,6 MiB

Sum: around 28,2 MiB

33,6 MiB - 28,2 Mib = 5,4 MiB

Okay this will make the data a LOT smaller style_emoticons/<#EMO_DIR#>/smile.gif
Thanks for the answer!

Karsten

5

The feature of automatically deleting old older than 7/30/N days is now available in Piwik, under Settings > Privacy > Delete old logs from the database.

This is available in the latest 1.5 RC release, check it out now and report if you have suggestions, directly in this post: 301 Moved Permanently

6

piwik v1.4
not directly related, but i have some strange database anomaly that i cant wrap my head around.

piwik_archive_blob_2011_01 5,4 K 4 Mb 425 Kb 4,5 Mb
piwik_archive_blob_2011_02 7,1 K 2,5 Mb 210 Kb 2,7 Mb
piwik_archive_blob_2011_03 365,5 K 111,6 Mb 10,3 Mb 121,9 Mb (<-)
piwik_archive_blob_2011_04 58 K 18,8 Mb 1,6 Mb 20,4 Mb
piwik_archive_blob_2011_05 17,7 K 6 Mb 527 Kb 6,5 Mb

I started tracking at the end of february (i dont understand why there even is a 2011_01 but thats another issue) with around 15k hits in feb and around 40k in every month since then. March had 44k hits and May had 39k hits. How is this possible that piwik_archive_blob_2011_03 takes nearly half the space of the whole database (260,5 MiB) with near identical visits?

edit:
march 374,260 data records
may 18,123 data records
? ^^

7

Rlyeh, you can delete the table and it will be built again