B) First Party Cookies


(Xaver) #1

I’m running Piwik as a hosted service for some of my websites and I like to have the possibility to set first party cookies (Implement first party cookie in Piwik · Issue #409 · matomo-org/matomo · GitHub)

I really don’t know how to start, because I know that cookies are generated server side. So there is no way to have fpc on different domains.

We have to create cookies on the client side via javascript (setCookie method allready exists)

My thoughts:

  1. Check if a cookie is set
  2. if no cookie, generate one
  3. else use existing.
  • the cookiename should be 'piwik'
  • the value should be an unique string for each unique visitor
Is it possible to just add this unique string and to verify the user server side?

I think it’s easy to solve the cookiecreation on the client (js), but it is much harder to solve the problem on the server.

Hope we find a way to solve this.


(vipsoft) #2

There are two problem areas in #409.

The first is a cookie store … this will map first party and third-party cookies to a unique visitor id per site.

The second is generating first party cookies from the browser which would be passed to the server via a parameter. We’ll need a good uuid algorithm. We may also need API methods to specify the cookie name and lifetime.


(vipsoft) #3

Don’t understand, what you mean. Should the old unique visitor id look like the new one?
Plz explain that for me.

We need the data store so that users use a single cookie across all the sites they’re tracking (vs one cookie per site, now). This is to avoid hitting the Cookie length limit in the http request. We also want to store less information in the cookie.

Why do we need an API method for the cookie name? Lets define it as an variable

Adding setCookieName() and setCookieExpire() methods won’t pollute the global namespace like a variable would.

p.s. please don’t post non-free code here. (I’ve had to unapprove your post.)


(Xaver) #4

I finally get more into the GA cookie setting system (GACSS style_emoticons/<#EMO_DIR#>/tongue.gif )

GA don’t use just a single cookie to verify user, it use several cookies. and thats a really good approach for us to.

GA sets basically 4 cookies:

_utma, _utmb, _utmc, _utmz

I wont to go a little into the first three ones (more details on http:\Google Analytics Cookie Usage on Websites  |  Analytics for Web (analytics.js)  |  Google Developers)

But previously i go over the structure of those. The cookies all start with an id (unique id) which gets generated only once - when no cookie is set.
I found out that GA always generates the same id, even on different browsers and even with different IPs, so our getUniqueId method maybe can’t use only some Math.random…

If more values necessary a single dot joins the values:

id.value2.value3

_utma:

Example: _utma: 183050102.1945398870.1291196605.1291196605.1291196605.1

Expires: 730 days (2 years)

I call it the “main cookie” which has the important part.
As you can see we have six values in this cookie (separeted by dots):

  • ID
  • timestamp
  • timestamp
  • timestamp
  • timestamp
  • Integer

The first timestamp is pointing very far in the future (in this example 2031-08-25 04:34:30 GMT) and don’t change
Don’t think we need something like this.

Second and third one are timestamps of creating the cookie/first visit
I think we need just one of them

The fourth timestamp is reset after closing the browser.

The last Number is increased by one after closing the browser

I think our cookie for storing unique vistors should look like this:

_piwika = id.timestampFirstVisit.timestampLastVisit.totalVisits

I guess 2 years is a good expire time but also saw 1 and even 10 years on other tracking solutions

_utmb:

Example: 183050102.1.10.1291198913

Expires: 30min

Its basically a session cookie

  • id
  • int
  • int
  • timestamp

Every trackaction the first int get increased by one
The timestamp gets an update after closing Browser
don’t know whats the second int (10) is all about

Our cookie:

_piwika = id.numberOfActions.timestampOfFirstViewAfterBrowserClosing

Expire time should be the session time (30min)

_utmc:
Example: 183050102

Expires: after closing the browser (no date set)
A second session cookie, which plays with _utmb. just to check if the browser has been closed.

Our cookie:

_piwikc = id

Expire time shouldn’t be set

_utmz:

Don’t won’t to get more into this. Its basically responsible for der referer handling

I’ve modified the getRequest function on piwik.js to generate cookies and ad them to the request:

function getRequest(customData, pluginMethod) {
    /*
    ...
    *7

    // set the cookie
    if (browserHasCookies) {
        var uniqueId;
        var timestamp = new Date().getTime();
        
        //Format: id.timestampFirstVisit.timestampLastVisit.totalVisits
        //Expires: 730 days (2 years)
        var _piwika = getCookie('_piwika');
        
        //Format: id.numberOfActions.timestampOfFirstViewAfterBrowserClosing
        //Expires: 1/48 day (30min)
        var _piwikb = getCookie('_piwikb');
        
        //Format: id
        //Expires: after Browser closing
        var _piwikc = getCookie('_piwikc');
        
                
        
        //main cookie not set => new Visitor
        if(!_piwika){
            //get new unique ID
            uniqueId = getUniqueId();
            //generate string
            _piwika = uniqueId+'.'+timestamp+'.'+timestamp+'.1';
        }else{
            
            _piwika = _piwika.split('.');
            //Browser has been closed since last visit
            if(!_piwikc){
                //so we have to increase totalVisits
                totalVisits = _piwika[3]++;
            }else{
                //else don't change visit count
                totalVisits = _piwika[3];
            }

            
            //get old unique ID
            uniqueId = _piwika[0];
            //generate string
            _piwika = uniqueId+'.'+_piwika[1]+'.'+_piwika[2]+'.'+totalVisits;
        }
        
        setCookie('_piwika', _piwika, 730);    
        
        var timestampOfFirstViewAfterBrowserClosing = timestamp;
        
        //check if new Visitor or new session (>30min)
        if(!_piwikb){
            // new session => generate string
            _piwikb = uniqueId+'.1.'+timestamp;    
        }else{
            //old session
            _piwikb = _piwikb.split('.');
            
            //_piwikc is not set => browser has been close during last visit
            if(!_piwikc){
                //use new timestamp
                timestampOfFirstViewAfterBrowserClosing = timestamp;
            }else{
                //use old timestamp
                timestampOfFirstViewAfterBrowserClosing = _piwikb[2];
            }
            //join _piwikb parts and increase counter
            _piwikb = _piwikb[0]+'.'+(parseInt(_piwikb[1],10)+1)+'.'+timestampOfFirstViewAfterBrowserClosing;
            
        }
        
        setCookie('_piwikb', _piwikb, 1/48);

        //doesn't exist if browser was closed
        if(!_piwikc){
            setCookie('_piwikc', uniqueId);    
        }
        
        
        //add cookievalues to the request
        request += '&_piwika='+_piwika;
        request += '&_piwikb='+_piwikb;
    }
    
    // tracker plugin hook
    request += executePluginMethod(pluginMethod);
    
    return request;
}

Thats just my first attempt for a First Party Cookie solution. Code needs some improvement of course but just like to throw it an wait for some responses.

Thats just the client side approach, because I’m not really familiar with the server side code.


(Xaver) #5

found an interesting article with more details on GA cookies:

http:\Using the Google Analytics Cookies


(Xaver) #6

hmm, seems that nobody is interested in this topic style_emoticons/<#EMO_DIR#>/dry.gif


(vipsoft) #7

Thanks for the research. (Sorry… been too busy to reply earlier.) I don’t have anything to add at this time because I think the server-side cookie storage will guide/dictate the client implementation.


(Xaver) #8

Ok

Maybe someone is more familiar with the code and the cookie creation. An unique ID is created server side. Is it possible to use that one from the client side.

I really don’t know where to start, so maybe someone give me a little introduction.