Skip to content

The User-Centric Approach – Google Universal Analytics vs. Mixpanel

by on May 16th, 2013

About 6 months ago, Google released Universal Analytics, which – in a nutshell – offers 4 new key features:

  • Measurement Protocol – an API that can be used for multi-platform tracking
  • Improved Feature Configuration Management – through the GA admin screens, rather than the JavaScript agent
  • Custom Dimensions and Metrics – very similar to the GA Custom Variables feature, but with some admin control and improved support for the built-in GA features
  • Dimension Widening – which allows you to load dimensions via data upload, rather than having to do so one at a time via JavaScript (and exposing data you would prefer not to be exposed)

Since its release, I have often been asked about Universal Analytics. Now, after having had the chance to try it out for a while, my answer is as follows: From a market point of view, this release offers nothing new for the end customer. Google is now aligned with the rest of the industry, with their new features being old news for other existing tools.

What is unique about these new Google features, however, is that they are free, and definitely suffice for the needs of about 80% of website owners worldwide. Take for example the GA Real Time Reports that were released more than a year ago, which may be inferior to Mixpanel or ChartBeat, but are definitely good enough for most companies, and are free!

And what about Google Universal being more User-Centric?

At first, after reading the great reviews, I was sure that Google now recognizes the importance of placing the focus on the customer, rather than on the visit. I even believed, from what I had read, that this was the most significant change in this release. However, after working with Google Universal Analytics for the past 6 months, I can honestly say that GA has not changed its approach. (In Google’s defense, I don’t think they ever claimed to having made such a focus change and I don’t remember having seen an official document talking about such a change in approach.)

The funny thing is that looking back, despite there being such a buzz about this focus change (including how easy it is to link users across devices and use your own ID), no one ever explained the actual procedures. Even trying to figure out how their user identification actually works is almost impossible, without digging into the Measurement and Analytics Modules’ documentation.

In order to examine whether or not Google had in fact become more user-centric, I had to find the answer to the following 2 questions:

  1. Does Google Universal Analytics offer an equivalent to Mixpanel’s identify and alias methods?
  2. Which aspects of the updated Google Universal Reports make it user-centric?

60 Seconds on a User-Centric Approach

A user-centric / customer-centric tool (such as Mixpanel) needs to provide the following 2 aspects:

  1. A way to identify and link a user across devices and platforms
  2. The option for tying all activities together and providing data per visitor

Mixpanel provides two methods for identifying and linking activities across devices:

  • The “alias” Method, which allows you to add your own unique identifier to the current user
  • The “identify” Method, which can be used when replacing the current user on a device with an existing user in Mixpanel.

Here is a link to a great explanation about the differences between these 2 methods, but if you are looking for a way to remember it, try to memorize this:

  • “alias” should be used on the thank you page of your registration page
  • “identify” should be used on the thank you page of your login page

As for tying activities together and providing data per visitor, Mixpanel definitely does so, by providing reports that are focused on visitors (customer-centric) rather than on visits (visit-centric):

  • The Trends Report – The number of visitors/unique visitors who performed a specific event.
  • The Funnel Report – The number of unique visitors who performed a specific step in the funnel.
  • The Cohort Reports – The number of visitors who carried out specific events.
  • The People Report – What each and every user did within the tracked environment.

Maybe I shouldn’t say that Google has not changed its approach at all…

If the distance from visit-centric to user-centric is 1,000 miles, then Google has just made its first step. But truthfully, Google Universal is not even close to being a sufficient solution.

The only user-centric feature that Google added in Google Universal is a way to link events to a specific user both in the measurement protocol and the JavaScript agent (Analytics.js). You can see an example of how to do so in my previous post

This feature contains two changes: 1) It is easier to update the client ID (instead of updating the cookie) and 2) events sent via the measurement protocol will be part of the same session if they are all sent during the time the session exists (30 minutes as default but you can change that if you want).

Google Universal Analytics and Mixpanel Sample flow

So now let’s get back to comparing Google Universal to Mixpanel, with an emphasis on a user-centric approach. In this example, the site contains pages that are accessible for both guests and registered users, and is accessible by PC, tablet and mobile phone users.

Let’s take the following scenario:

  1. User1 comes to the site for the first time, from her PC, and visits pages A and B. At this stage she is a guest.
  2. She then registers on the site and visits page C
  3. A few days later, user1 visits the site again, this time from her mobile phone
  4. She then visits page A, logs in and visits page C

At this stage, we know that there was only one person (user1) who viewed 5 pages in 2 visits/sessions. However, Google Universal Analytics and Mixpanel will each give us different information.

Mixpanel

Due to the “alias” method that was called upon during the User1 registration (as explained above), Mixpanel will credit all 3 page views during the first visit to the same user: User1. And this is correct.

During the User1 second visit to the site, the first page view (page A) will be counted as a separate user in Mixpanel – which make sense, as at that point Mixpanel has no way of knowing who this user is, since she came through a different browser.

The “identify” method that was called upon during the User1 login will tell Mixpanel that this user is linked to User1 and will credit the next page view (page C) to User1. The problem is, however, that Mixpanel does not go back and credit the view of page A in the beginning of this 2nd visit (prior to logging in) to User1. I would expect the “identify” method to work like the “alias” method, which ties both past and future events to the user.

Therefore, the end result is Mixpanel will report 2 unique users: one (User1) with 4 page views (pages A, B, C and C), and one user with 1 page view (page A).

Mixpanel will report 2 unique users, one (User1) with 4 pageviews and one user with 1 pageview

There is some sense in the way Mixpanel works but I think it is not trivial and should be configurable by the customer. When I asked 5 people who are familiar with Mixpanel, 4 out of 5 of them thought the expected behavior should be 1 unique user with 5 page views.

One more thing I found with Mixpanel, which is kind of strange, is that calling “alias” after registration on a specific browser onto which a different user was previously logged into will cause Mixpanel to “hijack” the first user and tie all their events to the new registered user. I can understand the way in which Mixpanel implemented the “alias” method, but if my site is accessible from a public/shared PC or tablet, this could cause serious data collection issues. (I wonder if there is a solution for this, such as automatically clearing cookies upon logout or at the end of each session, perhaps?)

The main difference between Google Universal Analytics and Mixpanel, and probably the best example of how far Google is from services such as Mixpanel, is the fact that with Mixpanel you can actually see a list of all pages and activities that a specific user performed on your site – without any customization whatsoever. Google, on the other hand, requires massive customization and a lot of coding.

The People Report, offered by Mixpanel, shows the activities and attributes of each and every visitor on the site:

Mixpanel's people report shows the activities and attributes of each and every visitor on the site

Google Universal Analytics

Let’s go back now to the sample flow:

  1. User1 comes to the site for the first time, from her PC, and visits pages A and B.
  2. She then registers on the site and visits page C
  3. A few days later, user1 visits the site again, this time from her mobile phone
  4. She then visits page A, logs in and visits page C

Google will now report 3 unique users:

  1. User with 2 page views – page A and page B during the first visit.
  2. User with 1 page view – page A during the second visit.
  3. User with two page views – page C twice – once on the first visit after registration and once on the second visit after login.

3 unique visitors and the last page they saw according to Google Universal Analytics

Google Universal detects 3 unique users because unlike Mixpanel, it does not have an alias method. Mixpanel attaches the ID to the internal user ID, thereby ensuring that ALL events (past and future) will appear under the same user. Google Universal Analytics does not.

With Google, when you define a different client ID, you have to tell Google to override their internal user ID and start using yours. But once you do that, Google treats it as a different user. And this is the same behavior as the Mixpanel “identify” method, where only future events are sent on behalf of the user ID that you passed to the identify method.

There are rumors of a Google “User ID” field, which unfortunately is not yet supported. This field will probably provide the same functionality as the alias method and let you set your own ID without overriding Google’s unique ID.

So for now I recommend the following flow upon registration:

  1. Add a hidden field to the registration form which will hold the clientId:
    ga(function(tracker) {
      var clientId = tracker.get("clientId");
      $("#clientId").val(clientId);
    });
    
  2. Store the clientId sent with the registration form
  3. Instead of returning your own user ID upon login, reinstate the clientId from the registration stage and use that ID when creating the tracker.

This will give you exactly the same results for both Google Universal Analytics and Mixpanel.

But now that we found a way to keep the data accurate, let’s talk about the real problem with Google Universal Analytics: There is not even one report that provides data on a user level. There is no way to obtain the amount of visitors who completed a specific goal. There is no way to define a real funnel (as here too Google looks at visits rather than visitors), as you can only define funnels on a visit level, which in some cases is absolutely useless. There is no way to see what a specific user did. And the list goes on…

So if I go back to my original question, has Google really become more user-centric, my answer is no. Google Universal Analytics cannot, in my opinion, be considered an alternative to Mixpanel or other similar services.

  • http://marketingland.com/is-googles-universal-analytics-a-game-changer-48490 Is Google's Universal Analytics A Game Changer?

    [...] and measuring visitors as opposed to visits. (See an interesting comparison of approaches here.) What is new, however, is that UA is free — meaning that sophisticated analytics is now [...]

  • http://mailmarketinguk.com/is-googles-universal-analytics-a-game-changer/ Is Google’s Universal Analytics A Game Changer? | Uk Marketing

    [...] and measuring visitors as opposed to visits. (See an interesting comparison of approaches here.) What is new, however, is that UA is free — meaning that sophisticated analytics is now [...]

  • http://www.webacom.com/nanaimo-vancouver-news-blog/is-googles-universal-analytics-a-game-changer Is Google’s Universal Analytics A Game Changer? ,Vancouver Island, Canada

    [...] formation and measuring visitors as against to visits. (See an engaging comparison of approaches here.) What is new, however, is that UA is giveaway — definition that worldly analytics is now [...]

  • Chris

    Thanks for such a great overview! I was thinking about switching from Mixpanel to GA after the new features released at I/O, but this has convinced me to keep Mixpanel as the primary place to track.

    I do like the download attribution that’s new to GA – we’ll be using that to track conversions from web–>app download.

  • Ed

    Do you know if KISSMetrics has the same ‘flaw’ as Mixpanel in that it ‘does not go back and credit the view of page A in the beginning of this 2nd visit (prior to logging in) to User1′?

    I agree about Universal not being user centric though. Too many people are touting it as THE way to solve cross device tracking, without having used it and looked at HOW it does it. And yes, the fact that you can’t access the CID in interface is a drawback, and that all the reports still show visit level metrics.

  • pere rovira

    Very interesting article! I think that we must keep in mind that Universal Analytics is still a beta, hence it could be improved / changed in the coming months, before being a stable new functionality. The fact that the underlying data infrastructure allows for user tracking is in fact a big change, we now must wait and see what will be possible, in other words, if we’ll get new reports at all to reflect and analyse all this new user data.

    About going back and stitching all session-level data into a specific, identified user, how does it work from a privacy perspective? You’re in fact de-annonimizing data, I guess we should inform the user somehow that this could be possible.

  • Anonymous

    As far as I know, KISSMetrics’ identify method works exactly the same as the one in Mixpanel. According to their documentation, the identify method “Sets the identity of the current person, so we know to attribute FUTURE EVENTS to.”

  • Anonymous

    I agree that the new version is a huge step further but it is still far from being close to what Mixpanel or KISSMetrics have to offer on the user level.

    Regarding the privacy perspective, in general, I think it will be a good idea to inform the user that you track his/her activity (and as far as I know, it is the law in some countries). Keep in mind that we can identify the user across devices only if he identifies first (by signing in or providing email or any other personal identifier) so there is nothing we do “behind his/her back”…

  • Guillaume Dupuy

    I agree that in its current form, without UserId, UA has actually done little to become more user-centric. I disagree, though, that Google has never made a public claim to have taken such a focus: the Google I/O video at https://www.youtube.com/watch?v=CzmNbmwDMUs indicates just the opposite (jump to minute 12).

    Web Analytics are a complicated subject. For me, things got a little clearer when I tried to specifically understand the meaning of what is commonly referred to as “visitors”. Looking at the history of web analytics through that lens helped me tremendously in gaining a better understanding of the recent changes to Google Analytics. Quick a lengthy comment, but worth the read I think.

    Cookie-based analytics: visitor = clientID
    —————————————————–

    First, there was cookie-based analytics: with cookie-based analytics, “visitors” are in reality clients (as in client-side), i.e. any place where a cookie can be left (typically, a browser). A client hits your site: if it has your cookie, the tracker gets its ID and sends a hit with this ID. If it doesn’t, it creates a ID for you, stores it in a cookie and sends a hit with this ID. All that with the commonly accepted caveat that resetting cookies on a client means they’ll be recognized as an entirely new client for the future. Simple enough.

    Mixpanel: visitor = client, or user, or both + clientIDs and userIDs are unique
    ———————————————————————————————

    Then people started to ask for report consolidation based on users, instead of devices, and we got mixpanel and KISSMetrics. Mixpanel and KISSMetrics have a mixed approach: at the core, they track visitors the good old way, using cookies and client IDs. Each client has an ID, stored in a cookie, and whenever it hits your site the tracker sends a hit with the ID. What mixpanel (and KISSMetrics, but I’ll stick with the former since I know their API better) adds, though, is an additional layer that lets you identify visitors differently. Through cookie-based identification mixpanel has a way to say ‘I know this client’ ; through you auth system though, you also have a way to say ‘I know this user’: why not use this info as well? The problem is in the way they mix these two sources of identification.

    With the alias method, Mixpanel essentially defines visitors as “a clientID, or a userID, or both”. If a visitor is only defined by a clientID, it’ll be identified by mixpanel using standard cookie-based tracking. If it’s only defined by a userID, it’ll have to be identified by you through the mixpanel.identify(userID) method. If it has both IDs, it’ll be identified in both situations, i.e. when mixpanel recognizes the corresponding client or when you recognize the corresponding user. Of course, the mixpanel.alias(userID) method is here to associate a client and a user: mixpanel will read the clientID from its cookie, get the corresponding visitor and add the userID to it.

    This is where hell begins. For one, there can’t be 2 [clientID, userID] couples with the same userID: with the way mixpanel does things, this is essentially a technically impossible scenario (using mixpanel.alias(userID) with the same userID on several clients isn’t supported, as indicated in their docs at https://mixpanel.com/docs/integration-libraries/using-mixpanel-alias). And yet one user can access your site through different clients, and this is where you get to your example with mobile browsing, where you end up overestimating the number of users hitting your site.

    Secondly, and this is also by design, there can’t be 2 [clientID, userID] with the same clientID: if you call mixpanel.alias(userID) on the same client with 2 different userIDs, it’ll simply update the object with clientID = the ID in the mixpanel cookie twice, each time with the submitted userID. Which leads us to the problem you mentioned with public-access, or more generally shared clients.

    Maybe more of minor problem (but still a problem), clients are identified using a cookie-stored ID. What if a user empties their cookies on the client they created their account from? Said client gets a new clientID that’s not aliased, and since I can’t ever alias it (see my first point), that device is off my radar forever.

    Not so great, huh? Enter Universal Analytics.

    Google UA: visitor = there’s no such thing as a measurable visitor
    ————————————————————————————-

    The conclusion seems to be that there’s no such thing as a “measurable visitor”. Or at least, not in the sense we usually interpret it, which really is : “a distinct user, logged or not, visiting our site”. If you use userID to measure visitors, you ignore all the logged-out hits. If you use clientID, you ignore the fact that users use different clients to access your site and fail to consolidate cross-device data. If you try to use both, you fall in the mixpanel traps. So where does that leave us?

    From the Google I/O video, it seems that Google’s answer is, for any hit they record, to store the originating [clientID, userID] couple and interpret the data later. Note that in this case clientID will always be set (since it is calculated by analytics.js), while the userID will be set whenever you choose to set it (likely, when users are logged in).

    This approach gives Google many advantages: they can now see things how many different devices each user utilizes on average to access your site (or “device overlap”). Or how many users access your site from shared computers (shared by, on average, how many users). And while we have yet to define what can be defined as a visitor in that circumstance, Google can always reconstruct this notion at will from the data: for starters and rather obviously, all [clientID, userID] couples with the same userID represent the same visitor. Interpreting [clientID, null] is a little bit harder, but you can use several strategies (and refine them as you go, since you’ve got the underlying data anyway):
    - all [clientID, null] couples with the same clientID could represent the same visitor, which is the standard, Classic Analytics status quo
    - [clientID, null] couples that share their session with a single [clientID, userID] couple can be attached to the corresponding userID

    - etc.

    We of course still have to wait to see what final implementation will come out. As far as I’m concerned, I think the result has good chances of actually beating mixpanel and KISSMetrics’ current offering, simply because it is more accurate and flexible. You are right to report the lack of user-level reports though, and we’ll see what Google does there as well.

    Very impatient, have to say ;-)

  • Anonymous

    Thank you for your interesting and comprehensive comment+post. I’ve read your post but there was no option to comment so I will do it here. Hope you will have the chance to read it.

    The main issue here, as far as I see it, is to find the right business rule of determining a visitor. Mixpanel (and KISSmetrics) provide the identify and the alias methods which are equivalent to the clientId and userId as presented by Google Universal.

    I did not understand from the video that Google are going to interpret the data retroactive as you say but let’s assume this will be the case. Even if they will be able to do that, how they will decide if to credit one user for all the past activities or not. I am not sure I understand how will they avoid the “mixpanel traps” as you presented it.

    As I wrote in my original post, the decision if to credit past activity to the user who just signed in should be the decision of the site owner. Something that Mixpanel do not offer right now and I hope that Google Universal will provide in the future.

    Thanks again,

    Shay

  • http://wirelesscashflow.com/19-key-insights-into-universal-analytics/ 19 Key Insights into Universal Analytics – Wireless Cashflow | Wireless Cashflow

    [...] In The User-Centric Approach – Google Universal Analytics vs. Mixpanel Shay Sharon compares these two tracking tools, examining the new UA features and asking whether [...]

  • http://knowbymobile.com/disney/19-key-insights-into-universal-analytics/ 19 Key Insights into Universal Analytics – Disneyland Vacations | Disneyland Vacations

    [...] In The User-Centric Approach – Google Universal Analytics vs. Mixpanel Shay Sharon compares these two tracking tools, examining the new UA features and asking whether [...]

  • http://knowbymobile.com/creditrepair/19-key-insights-into-universal-analytics/ 19 Key Insights into Universal Analytics – Credit Repair | Credit Repair

    [...] In The User-Centric Approach – Google Universal Analytics vs. Mixpanel Shay Sharon compares these two tracking tools, examining the new UA features and asking whether [...]

  • http://cabiznews.com/19-key-insights-into-universal-analytics/ 19 Key Insights into Universal Analytics | CABizNews.com

    [...] In The User-Centric Approach – Google Universal Analytics vs. Mixpanel Shay Sharon compares these two tracking tools, examining the new UA features and asking whether [...]

  • http://www.goatseo.com/ simone righini

    i wish i had read this post BEFORE implementing the massive Analytics customization that took us 2 weeks :|

  • Gregory Talon

    You’re clearly better off not using the id rewriting if you’re on GA. At least you would track 2 unique users.

  • http://blog.intlock.com/full-customer-journey-part/ The Full Customer Journey – Google Universal, Mixpanel and KISSmetrics | Web Analytics

    [...] and KISSmetrics by shay.sharon on February 24th, 2014 TweetA few months ago I wrote about the Google Universal User-Centric Approach, where I mentioned that in order for a web analytics tool to be considered a user-centric analytics [...]

  • http://advalue.mobi/2014/04/12/from-ga-to-ua-from-cid-to-uid-%e7%84%a1cookies-%e7%9a%84%e6%97%a5%e5%ad%90%e6%8c%87%e6%97%a5%e5%8f%af%e5%be%85/ From GA to UA ; From CID to UID ; 無Cookies 的日子指日可待 | AdValue

    [...] and measuring visitors as opposed to visits. (See an interesting comparison of approaches here.) What is new, however, is that UA is free — meaning that sophisticated analytics is now [...]

  • Anonymous

    Hi Ed,

    I don’t know if this one is still relevant for you but just wanted to tell you that KISSmetrics actually have a way to overcome this problem. If you want to read more about it, please go to my latest post – http://blog.intlock.com/full-customer-journey-part-iimanaging-user-identities-google-universal-mixpanel-kissmetrics/

    Thanks,
    Shay

  • https://www.webmatros.com/mixpanel-wordpress-integration/ Mixpanel WordPress integration in three easy steps

    [...] Then there’s Google’s Universal Analytics. But they’re apparently still beaten by Mixpanel. [...]

  • http://news.seowebmarket.com/2013/06/is-googles-universal-analytics-a-game-changer/ Is Google’s Universal Analytics A Game Changer? | SEO Web Marketing News

    [...] and measuring visitors as opposed to visits. (See an interesting comparison of approaches here.) What is new, however, is that UA is free — meaning that sophisticated analytics is now [...]

  • Anonymous