Background:
I have developed a script that imports log files every 5 minutes. The script tracks the inode of the log files and correctly handles log rotation, ensuring that it reads any remaining lines from the rotated log file before switching to the new one.
To efficiently continue from where import_logs.py last processed, I use the --skip parameter.
The Problem:
To determine the last processed line, I have been relying on the “%d lines parsed” output. However, this line often does not appear, even though import_logs.py processes lines.
Feature Request:
Could you please modify import_logs.py to always print the value of stats.count_lines_recorded.value as the final output? This would provide a reliable way to determine the last successfully processed line, making incremental imports much more robust.