I can’t seem to find a reference in the docs about paginating through results. I’m using the Live.getLastVisitsDetails module. I figured I could segment by visitId>X but I just get a {"result":"error","message":"The segment condition 'visitId&gt' is not valid."}
What’s the recommended way of fetching the next set of records from this API given the previous set? My full query looks something like:
Hmm I see. So there’s no indication how many records are in the entire collection or what “page” or “cursor” I’m looking at. Is the intention is that we start from filter_offset=0 and advance by the number of records returned for each call until no more records are returned?
I had hope to just be able to say something like:
get all records where idVisit > X
This would allow me to run a cronjob for instance that knows about the most recent idVisit I’ve ingested and I can then start from the next set of visits after that.
But it doesn’t seem you can actually query the API to that level of detail?
Ya I understand what the filter_limit is doing. Maybe I’m not clear in what I"m trying to do.
I basically just want to periodically scrape through all of the new visit information that’s being collected by Matomo so I can ingest it into another system. I imagined this as “paging through data sets”.
So I understand now that the segment option is not meant for this, but the filter options don’t seem to offer a very robust way of periodically checking for new visits from this endpoint.
Here’s a precise example:
I’d like to check for new results every hour in Matomo and ingest them into my system
It seems to do this, I always have to start at filter_offset=0 and iterate until the number of returned records is < filter_limit=500 (or whatever limit I’ve chosen)
The next time (ie: the next hour) when I go to find the new records, I don’t seem to have any way of actually doing this, other than to start again from 0, and iterate over all of the records that I’ve already seen, in the hopes that there are new ones tacked onto the end of the collection.
What I really want, is to be able to say "show me records whose idVisit > X" where X would be the last visit ID that I ingested. Is this type of query option available on Matomo?
Thanks so much for your responses, they’ve already been a big help!
If it would work, the response may not be the one you expect…
For example if you get the list of last visits… You get for example visit 1, visit 2, and visit 3 and their interactions (page views, events, etc): v1-pv1, v1-pv2, v2-pv1, v3-pv1. OK, that sounds good
One hour later same request, then you get visit 4, visit 5, and visit 6 + v4-pv1 + v5-pv1 + v6-pv1…
But you’ll never receive the interactions of visits 1, 2 and 3 that took place between your 2 snapshots!
Oh! Ok I think I’m starting to understand the API a bit. So each record from this API is actually updated with more interactions. These records aren’t like an append only “log” of each change/visit, they’re actually updated each time as more interactions from the user come in.
I don’t suppose there’s an “events”-like feed is there? That’s append only that I can just consume 1 by 1 ?
If you get each morning the data of the previous day, you should get what you need…
But if the previous day is not what you want, you’ll have to develop your own plugin: https://developer.matomo.org/develop
Right, ya that makes sense thanks. Does the visitId change for the same visitor that visits on multiple days? It’s an auto-incrementing bigint but I can’t tell at what point new records are inserted (vs updated old ones). Are there docs somewhere around that?