Correct command and syntax to import web server logs?

Dear Experts,

I have a website hosted at A2 Hosting.

Recently, using CPanel, and Softaculous, I installed Matomo onto the server.
NOT Wordpress.

The idea is to use Matomo to track all the traffic,
via the web server logs, as described here

The scripts I tried to import the web server logs
ran, but did not work. ie.

/bin/python3.6 /home/the_website/public_html/misc/log-analytics/import_logs.py --url=https://the_website.com file=/home/the_website/logs/unzipped/writing.the_website.com-ssl_log-Dec-2022

Output below.

What is the correct command to import the web server logs?

Thanks


0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)

Logs import summary

0 requests imported successfully
0 requests were downloads
0 requests ignored:
    0 HTTP errors
    0 HTTP redirects
    0 invalid log lines
    0 filtered log lines
    0 requests did not match any known site
    0 requests did not match any --hostname
    0 requests done by bots, search engines...
    0 requests to static resources (css, js, images, ico, ttf...)
    0 requests to file downloads did not match any --download-extensions

Website import summary

0 requests imported to 0 sites
    0 sites already existed
    0 sites were created:

0 distinct hostnames did not match any existing site:

Performance summary

Total time: 0 seconds
Requests imported per second: 0.0 requests per second

Processing your log data

In order for your logs to be processed by Matomo, you may need to run the following command:
 ./console core:archive --force-all-websites --url='https://the_website.com'

I have exactly the same problem. After spending hours to get everything working, I am now stuck at the same point here.

After days of struggle, I’ve concluded that the
only way for Matomo to read web server logs,
is to do it on a local server on which you have
full superuser rights.

See:

Log Analytics in Matomo/Piwik - Learn and see how to import server logs
Ronan Chardonneau

Procedure:

Download logs from server via CPanel
On Local Linux/Ubuntu server, import the logs

Although I haven’t tried/tested it yet
Perhaps sometime in the future

It would be a batch process.
Lagged perhaps by a whole month
As I only can only download after month end.


What really bothers me, like so much software and open source,
is the “figure it out yourselves” attitude.
Waste your own time.

So many (error) messages have NOTHING to do with the problem, and
NOTHING to do with the solution.

Why no output message to say how many rows were in the log file?
Even though zero rows were loaded?

Why no scripts to just analyze the file, to give ideas on what the
correct script parameters would be?

Why not 15 clear examples that actually work with correct syntax?
Detailing when you would use one script over another?

Input
script
output
with screen shots

Etc.

Sigh.

Something obscure I found on Github


Matomo Server Log Analytics

Import your server logs in Matomo with this powerful and easy to use tool.
Requirements

Python 3.5, 3.6, 3.7, 3.8, 3.9, 3.10.
Matomo On-Premise >= 4.0.0 or Matomo Cloud. Doesn’t work when Matomo for WordPress is used.

----->
Doesn’t work when Matomo for WordPress is used.
<-----

Although, I only installed Matomo on my hosted server.

@innocraft , do you have any clue?

I tried the following and it works (well it is not 100% perfect for the pageview):
./import_logs.py --url=https://my-matomo.com --idsite=1 --token-auth=mytoken --log-format-name=ovh --recorders=8 file.log
ticket to support A2 hosting log format is here https://github.com/matomo-org/matomo-log-analytics/issues/342

1 Like

in the meantime I wrote a mini book about log analytics… you will find it on Amazon.

1 Like

Something else I found:

Log Analytics cannot be used with Matomo for WordPress.