It is possible to ignore/hide system check entries:
The check responsible for LOAD DATA INFILE is Piwik\Plugins\Diagnostics\Diagnostic\DatabaseAbilitiesCheck. But keep in mind that this also removes the check for Utf8mb4, TemporaryLabels and Transaction check.
Also I just noticed that the check is already hidden if you set enable_load_data_infile=0 (this was added in Matomo 4 after @progger commented above)