Not able to get all visits by looping with filter_offset

Hello all,

I am trying to use the Live.getLastVisitsDetails method to fetch all the visit information from Matomo. I have a loop over two-week periods and an inner loop using filter_limit=500 and changing filter_offset 0, 500, … until no more data is returned. I understand this should fetch all the visits.

I am seeing something very strange, though. For instance, with parameters
filter_limit=500&filter_offset=1000&period=range&date=2019-12-15,2019-12-29
the last visit I get has timestamp 2019-12-18 19:39:28, but when I increase filter_offset by 500,
filter_limit=500&filter_offset=1500&period=range&date=2019-12-15,2019-12-29
the first visit has timestamp 2019-12-16 07:02:58, leaving hundreds of visits between these two times unaccounted for. This seems to happen whenever I am fetching the last batch of a time period, but I’m not completely sure about that.

I noticed that if I increase filter_limit to 1000 in that latter call, I get the correct results back, continuing immediately after the previous batch, but that’s a cumbersome workaround to implement and since I don’t understand this situation, I can’t be sure it works.

Has anyone seen something like this? Any advice on what I could do to fix my problem? I cannot increase filter_limit because in general I start hitting server resource limits if I go much above 500 (that’s another reason why I’m not confident that increasing it to 1000 temporarily would be a good idea).

The Matomo version is 3.13.1.

I experimented with the demo.matomo.org server and was able to reproduce this problem there. When I run this query:
https://demo.matomo.org/?module=API&method=Live.getLastVisitsDetails&idSite=62&date=2020-03-07,2020-03-08&period=range&format=json&filter_limit=5000&filter_offset=10000
the last result returned has server timestamp 2020-03-07 16:16:44. When I increase the filter_offset by 4999, the first result of this query should, by my understanding, be the last result of the previous query:
https://demo.matomo.org/?module=API&method=Live.getLastVisitsDetails&idSite=62&date=2020-03-07,2020-03-08&period=range&format=json&filter_limit=5000&filter_offset=14999
but it is not, it has server timestamp 2020-03-07 01:51:08. So there is quite a big gap between the two queries. If I try the workaround of increasing filter_limit, which worked with my data, I still don’t get all the results. With the query:
https://demo.matomo.org/?module=API&method=Live.getLastVisitsDetails&idSite=62&date=2020-03-07,2020-03-08&period=range&format=json&filter_limit=10000&filter_offset=14999
the first result has server timestamp 2020-03-07 12:07:08, so I got some more results but not all of them.