In matomo can we push custom Js events(Add to cart, view etc) on their specific Kafka topics or Hadoop?

General questions
#1

Hi team in matomo can we push custom Js events(Add to cart, view etc) on their specific Kafka topics or Hadoop ?
I am unable to find out to sink these real time payloads to Kafka or Hadoop

#2

@V.K sorry I am tagging you here ,
but I am struggling to find out this from a long,
can you please redirect me to the correct person for this issue.

Thanks

#3

Hi @heurteph-ei
Sorry to tag you directly here, actually im stuck in this from a long.
I want to know that how to send this raw data (visitorId, Screen size, revisit or not etc) to kafka for further processing. whatever raw data captured by Matomo Analytics, i want a real time dump into kafka for further processing.
how can we achieve this in matomo?
im using matomo piwik (4.14.1 version)
is there any native availability to do this ? because kafa extension of matomo is completely obsoleted .

Thanks

#4

Hi @pkr2
Sorry I don’t know kafka at all (I only know the writer! :wink: )
Maybe you could try to fork the kafka extension in GitHub and update your fork in order to work with latest version of Matomo?

#5

hi @heurteph-ei
Thanks for giving me your time to reply,
I saw this guide for migrating extensions for suporting newer version of Piwik
https://developer.matomo.org/guides/migrate-piwik-2-to-3
but it seems to me this will be too complex

https://github.com/gzilt-playground/piwik-plugin-kafkapush
Can you please please guide me on this :
1. right now this extension is developed on which piwik version 1 or 2 or 3 ?
2. and on which section do i need to change the code .?

#6

If we dont choose this complex route. do we have any other options like Pub/Sub… etc ?

#7

Hi @pkr2,
For linking purposes, link to your GitHub question:

Looking at the Marketplace, it seems this plugin was developed until Matomo version 2 (included).
For the code, sorry, I am not an expert on this field… :worried:

#8

hi @heurteph-ei

Thanks for your response.
Is there any other native way in Matomo (on premise or on cloud) through which I can send the raw data in real time to my own warehouse?
I saw this one
https://matomo.org/faq/how-to/faq_24536/
But this is completely API based which is providing you a day, month, year based raw data. I want this to be in Real Time subscription based manner .

Thanks

#9

Hi @pkr2
I have no info ahout this.
For real-time, except looking directly in the DB, I don’t see anything else…
https://developer.matomo.org/guides/database-schema
I found some example videos, I don’t know if it could give you some tips?