Same here, in my apache install (linux gentoo) no htaccess was created automatically. only some in config folder to blacklist access. but no htaccess in htdocs to whitelist files.
i need to read the code with the installer-wizard if it tries to copy or create this file.
as far as i can see this is all manual. and only the config is protected. but that’s not the discussion here.
I it was there, i would not even have started this discussion.
Some search in github shows there is indeed a file to create and put a htaccess there.
But when i used the wizard it connected DB etc. but didnt write the .htaccess file to my server.
So it seems to be a bug, which i was lucky to trigger with my server setup. (Shared Hosting)
Still odd we are now down 21 comments in this thread, and it seems all I question already exists, and its a real bug in the installer wizard, maybe problems with file access rights to save to disk.
The second answer should have been in this thread.
“… it already exists, you have a bug in your installation …”
I need to make a new Install on another server and see if the htaccess is created, and why it is not on my main server.
Then it just would be a bug report if the wizard cannot write to disk or similar .
Hi @Unterberger.Media
In case of discovered bug, you can create an issue at:
Hi!
I’m following this discussion since the start as I found it interesting that no .htaccess was created in the Matomo install directory three month ago when I first installed it on the server.
I do have it in:
All other directories and the main install directory do not have one.
I’m still not clear if they are missing or not and how to create them after the installation if necessary.
I would be grateful if this could be explained better.
@innocraft @matthieu do you have some clues for Xavier or Daniel?
Any news on this topic?
I have other things higher priority but i will make some tests in a few days. new install and debug what’s going on with the missing htaccess. (2 different hosters, to see if its some funny side-effect)
Thanks for the quick feedback!
Finally i have time to debug and analyse the matomo installation.
Quick Result is:
One on Server there was no .htaccess in main folder, also not in the node_modules. Probably some issue with file permissions during installation.
I try to make a new test installation and watch for file permissions or other errors, why the files are not generated.
eg. /node_modules/jquery/README.md
can be read in browser, which is a Bug as discussed in this thread.
the reason is
a) mistakes in following the installation (wrong file permissions and using a .zip packaged on linux)
b) bugs in the software, that the .htaccess is not written to disk, but does not give a error-message, that something went wrong.
then one has a incomplete (potentially insecure) installation without knowing, or opens a discussion in the forum to figure out what is going wrong as this is not the expected behaviour.
On the other hoster, matomo can be installed with an auto-installer from the backend. The installation works. This is supported by the hoster, so all should be correct here ideally and no risk of making errors during the installation skipping some steps or using wrong file permissions.
and the node_modules and config folder has a .htaccess which prevents the file to be readable in browser.
but the main .htaccess is only 2 lines and not preventing access to other files and folders.
but according to this discussion here, the htaccess in the main folder by default should have a blacklist for some files.
i am using latest 4.14.1 of matomo.
i will do more debugging, what is the “expected behaviour” according to the installer-wizard, and which files are not behaving properly using the comments from the discussion here, which suggests some .htaccess should have been generated automatically.
Also the systemCheckPage is not fully correct on my first installation, as it states “All private directories are inaccessible from the internet” , but i can read some files from node_modules as the htaccess was never created and was not part of the installer zip package (4.13.3)
Reading the code, the strategy is as following.
a) nothing changed with .htaccess in root folder. (has to be done manually by admin if you want this)
b) in the private directories a file is put (also detects IIS on Windows with other files)
$directoriesWithAutoHtaccess = array(
'/js',
'/libs',
'/vendor',
'/plugins',
'/misc/user',
'/node_modules',
'/config',
'/core',
'/lang',
'/tmp',
);
writing suppresses errors instead of handling the exception and warning the user that something went wrong. thats bad. It’s about security and missing these .htaccess files is not trivial.
@file_put_contents($file, $content, LOCK_EX);
The @ means don’t show errors and is bad Practice in PHP , and i think even turned off in PHP8+ , means the @ is ignored in future and will show errors again.
In my First installation it is sill PHP7 , so errors are not shown.
So in my combination of PHP7 + wrong file permissions (my guess) it’s possible to hide the error.
on a Server with PHP8 + wrong file permission (cannot write) it probably fails. And shows an error or breaks the script.
my second hosting has php8 and the .htaccess files were created. So it might be the expected behaviour.
Hi Daniel, thanks for coming back on this topic.
My situation is as follows:
The .htaccess situation is as follows:
So that’s quite a mixed bag.
As we installed the previous version freshly, I really can’t understand how this can be so random.
We would be interested to get the content of the missing .htaccess files so we can manually correct the situation.
Thanks!
A “Repair” function to fix broken htaccess could be done like in a 3.0.0-b1.php update.
/core/Updates/3.0.0-b1.php
// added .woff and woff2 allowlisted file for apache webserver
ServerFilesGenerator::deleteHtAccessFiles();
ServerFilesGenerator::createHtAccessFiles();
This would delete and recreate htaccess files managed by matomo. and not touch the root htaccess which might have manual changes.
I have created some changes to the “System Check” page using the delete + create function , adding a check for missing files.
You can see the modified files and use it manually, or it might end up in the github version sometime.
Opening the “System Check” Full System Report it will show missing files and automatically repair them.
After a reload the warning of missing files should disappear.
Warning Temporary:
This is great!
Does your github push block these as well? or is it something we should do manually?
It’d be great if someone from Matomo dev team would take a look at your github push, vet it, and update it for full Matomo installation and get pushed to all Matomo users in an update. I do get your point on not having it in the source code due to diff types of servers.
awesome thx Daniel, got it!
“EDIT”:
I deleted some posts, discussion about CLI / CGI-BIN Files now in BugBounty.
Copy from deleted Post/Comment. Still useful infos how the bug is “random”. :
did some more analysis. I have installed matomo on 2 different servers at different hosters (different Linux Versions etc.) .
On the newer one it seems all the htaccess file were created,
but on the old server half the htaccess files are missing.
So my first assumption that the file permissions were wrong and “no” htaccess was created is wrong. So like in your case randomly some files were not written, and the errors were suppressed by the @ Operator.
The file checker only tests a few files and does not check the directories. Also it does try to load the Files via http if its loadable, but not if the files are on the disc directly.
Also now i understand the logic of the htaccess . Basically its a mix of whitelist + blacklist in the folder.
Often its
a) Deny from All
b) Allow file extensions “static like .jpg, .css” etc. which are not executed by Server.
This will hide “.md” Documentation for example, as its not in the allowed list.
Also the node_modules, are actually not node.js Server Code, but its just used as a package-manager to install client-side JS Libraries (angular, vue, jquery) which again is used by plugins in the GUI of the Backend, some are moved or copied, some are used directly.
in /plugins/Widgets/CoreHome.php :: getJsFiles() is a listing which files are public and in usage.
there are also many node_modules files which are accessed directly from the browser, so these are not some development and deployment files, but public distributed files in the client.
This means the node_modules is a PUBLIC folder, as some files like …/jquery.js are loaded directly from it. My first assumption was that node_modules are only server-side scripts needed for node.js server.
The htaccess in the root folder probably is not generated automatically, but some providers create a default one, or in my case i use some settings for cache-expire, but its not managed by matomo.
you can search for existing htaccess files with the shell command in your matomo root directory:
find . -name “.htaccess”
On my “good” server i have .htaccess in following directories and they 90% work as intended.
./lang/.htaccess
./plugins/.htaccess
./tmp/.htaccess
./js/.htaccess
./vendor/.htaccess
./core/.htaccess
./config/.htaccess
./libs/.htaccess
./.htaccess
./node_modules/.htaccess
./misc/cron/.htaccess
./misc/user/.htaccess
On my “bad” server i have these .htaccess files, and one can compare what is “randomly” missing.
./plugins/.htaccess
./config/.htaccess
./.htaccess
./misc/cron/.htaccess
./misc/user/.htaccess
./lang/.htaccess
./js/.htaccess
./tmp/.htaccess
These .htaccess are all a different, so one should copy each from a good installation, for repairing.
But the question what’s the best way to “repair”. Is more philosophical how to handle with “this never happens” bugs .
The System-Check page could make a file-integrity check and see if files are missing, in reverse to the file-integrity if there are foreign files which should not be there:
The “good files” are defined in config/manifest.inc.php and has very detailed all essential files of matomo.
In reverse one could define a “missing” files list in the same way and check if the .htaccess files in the folders are missing and give a warning and maybe even allow to “repair” the files and do the creation of these files like done in the installation.
This should not be needed, but we have a sample size of 2 times, where this “cannot happen” bug is a serious problem, as now folders are public, which “in theory” are secured. but in practice are not.
As these bug is “randomly” skipping generation of .htaccess files , i am not even sure if my “good server” has all files .
I have opened a Ticket in BugBounty for Security-Discussion about CLI/CGI-BIN files.
I removed comments from public forum, just to be safe.
This thread is about "missing .htaccess randomly and how to fix it. "
i deleted the old pull-request and moved it into a feature branch.
One Idea would also be to use the new function “ServerFilesGenerator::checkHtaccessFiles()” after finished installation, as a double-check to catch the < 1% random bug where files are not created correctly.
also i don’t know the architecture so well. eg. if “use Piwik\Plugins\Installation\ServerFilesGenerator;” from the Installer Package in the “SystemCheck” (after installation, normal runtime) is so good.
Maybe the installer should be independent from other runtime.
…
For myself the bug is now “resolved” as i can use the Bugfix (Pullrequest Files) and i repaired my installation.
Other findings in CLI/CGI-Bin are discussed elsewhere.
I now understand the code better, and “node_modules” confused me, because it’s not actually node.js runtime-code, but just a package-manager to download jquery. i would suggest to rename the file “node_modules” to remove the confusion.
This folder is set to “static files only” in the .htaccess, so it’s secure, even if it would actually contain node.js files. or by some random accident a bad file ends up there.
Also the whole discussion is caused by a “real bug” which happens randomly very seldom, and matomo normally is “safe enough”, as the .htaccess files protect the folders enough.
I would like to reiterate Eastons question, are those changes going to be reflected in a normal Matomo update or do they have to be applied manually?
I’m not very fond of doing it manually, as Matomo will show those changes as not being core, right?
its unknown if the pull request will be included, or when.
if you want to fix it you can copy the code and repair the missing files , and yes it will show up as modified files, but you only have to run the patch once.
but if the pull request is integrated into the check page, it will catch the error for the few people where the files were not created properly.
as it is now fixed for me, and the issue is understandable, i will not put more energy into this.
only support for the pull request changes if needed.
