Segmented Vistor Log CPU usage


(Samuele Kaplun) #1

I am trying to generate "Segmented Visitor Log"s for milldy accessed URLs, say 300 visit today. The server fails to reply on time. I checked and, while the database seems barely used for the response, the corresponding PHP(-fpm) process that is tasked with the reply is taking up 100% CPU and is not able to reply in time (I guess with respect to the Ngnix timeout I have set up).

I checked with strace to see what the process is busy doing and to my surprise I have seen that is opening thousands of times the same files:

access("/var/www/html/piwik/tmp/assets", W_OK) = 0
access("/var/www/html/piwik/tmp/assets/asset_manager_global_css.css", R_OK) = 0
open("/var/www/html/piwik/tmp/assets/asset_manager_global_css.css", O_RDONLY) = 11
fstat(11, {st_mode=S_IFREG|0644, st_size=351893, ...}) = 0
lseek(11, 0, SEEK_CUR)                  = 0
fstat(11, {st_mode=S_IFREG|0644, st_size=351893, ...}) = 0
read(11, "/* compile_me_once=9c0304c15a2c4"..., 8192) = 8192
read(11, ": \"\\e62c\";\n}\n\n.icon-newtab:befor"..., 8192) = 8192
read(11, "1px}.ui-datepicker .ui-datepicke"..., 8192) = 8192
read(11, "lass_75_e6e6e6_1x400.png) 50% 50"..., 8192) = 8192
read(11, "ition:-96px -112px}.ui-icon-calc"..., 8192) = 8192
read(11, "background-color:#ec407a !import"..., 8192) = 8192
read(11, "mportant}.light-blue-text.text-a"..., 8192) = 8192
read(11, "xt.text-darken-2{color:#fbc02d !"..., 8192) = 8192
read(11, " !important}.grey-text.text-dark"..., 8192) = 8192
read(11, "ntent:\"\\00a0\"}table.responsive-t"..., 8192) = 8192
read(11, "66.6666666667%;margin-left:auto;"..., 8192) = 8192
[...]

This is repeated over and over again. I think there is some space for caching or moving this operation out of a loop.

Since I think this could be due to a bug I have opened also a corresponding issue on Github:

P.s. I have tried enabling also the redis cache but the behavior is the same.