Analyst vs. Implementer in SiteCatalyst

Adam Greco of Web Analytics Demystified recently published a list of “pet-peeves” when it comes to Omniture implementations. Now Adam is not just one of the smartest people in our industry, he knows as much about Omniture as it’s possible to know. Still, the list contains a couple of points that I think illustrate the difference between the perspectives of an implementer versus those of an analyst and highlight why so many Omniture implementations go wrong when it comes to an analysis perspective.

Most of Adam’s pet peeves identify obvious bad practices in SiteCatalyst tagging. There are a couple, however, that raise more difficult questions. Among these I would number Adam’s first and biggest pet-peeve:

Tracking Every eVar as an sProp
I would say that my biggest pet peeve is when clients have an sProp for every eVar they have set (or vice versa). When I see this, it is an early warning sign that the client doesn’t fully understand the fundamentals of SiteCatalyst. While there are definitely cases where you would capture the same data in both an eVar and an sProp, they are usually few and far between. As a rule of thumb, I only set an sProp if:
• There is a need to see Unique Visitor counts for the values stored in the sProp
• There is a need for Pathing
• You have run out of eVar Subrelations and need to break one variable down by another through the use of a Correlation (which will go away in SiteCatalyst v15)
• There will be many values (exceeding the unique limits and you just want data stored so I can get to it in DataWarehouse or Adobe Insight

For the most part, that is it… Beyond that, I tend to use eVars and Success Events for most of my implementation items.

I think this is far too hard on the much-maligned sProp. Setting every eVar as an sProp is undeniably sloppy and almost always does reflect some lack of understanding about the functions of each. But when an implementer isn’t deeply certain of the roles for each, I’d much rather have them set every eVar as an sProp than follow Adam’s advice and neglect their sProps.

From my perspective, the really problematic sentence here is this one:

“While there are definitely cases where you would capture the same data in both an eVar and an sProp, they are usually few and far between.”

It’s a problem because the cases where you would capture the same data in an eVar and sProp are all too common in SiteCatalyst – even given Adam’s list; and there are several common cases not on Adam’s list at all.

One of the most common cases where an sProp extends an eVar is the association of a variable with the page name. Props do this automatically (on page calls), eVars don’t. For implementations that neglect to move pageName into an eVar, this is a critical difference. There is almost no variable in Web analytics that you don’t want to cross-tabulate with the page on which it occurred. Props do this out-of-the-box. So copying an eVar to a Prop automatically gives you at least one significant and new capability.

Props can also be setup as lists. List Props have a bunch of unfortunate limitations in SiteCatalyst, but they also have some specific uses. If you’re trying to track impressions on a module or dynamic site, you’ll often find that ListProps are the most convenient mechanism you have. This isn’t a duplication of an eVar (because you can’t do this in an eVar), but it is an important sProp function that often gets ignored in standard implementations.

sProps also work differently when it comes to some aspects of segmentation (particularly Page Containers) – and they often provide a mechanism for picking out specific pages in a segment in a way that simply can’t be duplicated with an eVar. This isn’t the kind of problem that an implementer has to deal with – but segmentation is at the heart of analytics and I’m always reluctant to give up analytics capability.

The truth is that the prop variable is like the classical concept of the atom (I’m indebted to our own Paul Legutko for this analogy) – indivisible and unique, for reporting and analysis purposes. What you see is what you get in unique image-requests – even if this means lots of “unspecifieds” in the interface or blank values in DW. The eVar is more like modern quantum theory, where eVar values can create “spooky action at a distance” (Einstein) within Data-Warehouse, Discover, and the interface sub-relations. It’s often the case that you can run a DW request asking for two different variables (say, Internal Search Term and Site Content) in both a set of props and a set of eVars, and get 80% “unspecified” in the prop variables but 20% unspecified in the eVars. In that situation, the prop is usually what you want and to get similar clarity from the eVars would take significantly more work.

You don’t need to really add functions to Adam’s list to see why duplication of many eVars makes sense. Let’s consider some of the key points:

• You have run out of eVar Subrelations and need to break one variable down by another through the use of a Correlation (which will go away in SiteCatalyst v15)

Most standard Omniture contracts severely limit your ability to do eVar Subrelations (in V14 – which is, after all, the basis for every implementation we’ve all seen). Most sites live with only three full subrelations. That means you can only fully cross-tabulate three of your eVars unless you want to pay Omniture extra.

And with eVars, the story is even more complicated. eVar subrelations have hidden “gotcha’s” – an eVar with a week expiration and last attribution (like a marketing eVar) will not subrelate successfully to another eVar with a visit expiration and linear attribution (like a content eVar). Props, being discrete, don’t have this issue.

sProps provide a much more generous set of cross-tabulations (Correlations in Omniture speak) without additional payment. Unless you’ve setup a very skinny implementation, you’ll run out of eVar subrelations and still have about 30 variables to go (maybe more).

There is nothing more maddening than finding that you can’t cross-tabulate a variable like Search Term with Search Results Count because they’ve been setup as eVars and not with corresponding sProps. Cross-tabulation is a basic function of analysis and the idea that you wouldn’t want to cross-tabulate almost every variable is foreign to an analyst.

• There is a need to see Unique Visitor counts for the values stored in the sProp

I know there is a deep uncertainty in the Web analytics community over the value of visitor-level statistics. I don’t share it. Visitor is the fundamental level of all analysis. There are variables for which you don’t need to understand visitor counts and visitor distributions (would that I could always get visitor distributions – and you can use eVar counters to custom create these), but I’ll paraphrase and say that they are few and far between.

• There is a need for Pathing

This bullet point actually leads to another “pet-peeve” that I think illustrative of the difference between an implementer and analyst viewpoint. Adam rightly complains that setting up Pathing on a variable that doesn’t change in a session is a mistake. That’s pretty hard to argue with!

But here’s my “analystics” pet peeve with pathing – people don’t use it nearly enough. In a previous post I listed a whole range of typical meta-data items that should be captured in an implementation:

1. Functional Taxonomy: Describing what the pages is supposed to be doing in the broader site-structure
2. Site Taxomony: The hierarchical levels that the page occupies (e.g. Products/Detail)
3. Product Taxonomy: The product/family the page concerns (e.g. TVs/LCD/ModelX)
4. Topic Taxonomy: A topic coding of the copntent (e.g. International Affairs/Middle East/Egypt/Revolution)
5. Audience: The visitor segments the page is designed for (e.g. All, engineers, consumers, health-care providers, professionals, etc.)
6. Sales-Stage: The place in the sales stage the content is direct to (e.g. Early, Middle, Late)
7. Page Components: The modules the page contains (e.g. videos, images, reviews, etc.)
8. Component Classification: The value or status of the page or component (e.g. Overall Review Rating is High or Low, Price is Discounted or List, Availability is Out-of-Stock)
9. Content Cardinality: The amount of line-item content on a page (e.g. Number of Search Results Returned, Number of Products Listed, Number of Reviews)
10. Page Length: The number of words or screens of text on the page (e.g. 800 word description, 200 word article, article in 3 pages, article in 1 page)
11. Content Source: The publisher, source, author of the content (e.g. Columnist X, Database Y, blogger Z)
12. Publish Date & Days since Changed: The recency and freshness of the content

Every single one of these variables should be cross-tabulated. Every one. About half of these variables would benefit from pathing.

Pathing is also powerful in the analysis of dynamic applications, systems like faceted search (especially!), internal link tracking and even internal search term tracking. Unfortunately, the vast majority of Omniture implementations I review don’t enable pathing on any meta-data variables and don’t track faceting or dynamic applications at all.This is a much, much more widespread and serious problem than mistakenly pathing a static variable.

I’d rather have an implementer randomly enable pathing on props than not use it at all – at least there’s some chance they are providing me useful information! From an analyst perspective, I’m far more annoyed when people don’t try to use functionality they paid for than when they use it poorly.

There’s no doubt that eVars and sProps duplicate functionality. There’s also little doubt that Omniture is increasingly making eVars a more general purpose type of variable – one that may eventually replace sProps. Version 15 solves two of the biggest issues with eVars in all previous versions of SiteCatalyst: uniques counting and cross-tabulation (to some extent). Make pathing of eVar instances an option, and you might well be getting close to a single variable-type view of the world.

I’d welcome that. The two variable types in SiteCatalyst are needlessly confusing and have little in common with other BI or analysis systems. But that’s the future – not a version that any implementation we’ve ever seen was built for. In V14 and every extant implementation of Omniture, it’s sound practice to duplicate many of your sProps and eVars and it’s even better practice not to ignore the sProp and its additional capabilities. I’ll take a confused but robust implementation over a clear but crippled one any day. It’s a little painful to see an eVar duplicated as a prop (or vice-versa) when it makes no sense, but it’s much less painful than having to pay for a cross-tabulation when you happen to need one or being simply unable to see a visitor count or setup a path analysis.

As an analyst, it’s just hard for me to understand how the biggest pet peeve around Omniture implementations should be something that doesn’t cost a penny, doesn’t sacrifice any functionality, and adds at least some analytic richness.

I have similar issues with Adam’s take on Vista Rules:

VISTA Rule Chaos
The final pet peeve I will mention is related to VISTA Rules. Let me start by saying that VISTA and DB Vista rules are not bad. They can be very powerful, but it is also true that they can be easily misused and wreak havoc on a SiteCatalyst implementation. When using VISTA rules, it is critical that you and your entire team understand WHEN the rules are being used and WHAT they do in terms of setting variables. I have seen many cases where a developer will change a variable not knowing that there are VISTA rules impacting it. You need to make sure VISTA rules are heavily documented and as you change your site or implementation, they need to be factored into the equation. One suggestion I have is to add the phrase (SET VIA VISTA) in the name of any variable that is set via a VISTA rule in your documentation so there is no missing it!

The other pet peeve I have related to VISTA rules is when they are used as a “band-aid” to avoid doing real tagging. In the long-run, this always comes back to haunt you. I see many clients creating band-aids on top of band-aids until things fall apart. I am ok with companies using Vista rules to get things done quickly, but I recommend that, over time, you phase out as many VISTA rules as you can and move their logic to your regular tagging so you have all of your logic in one place.

There’s good stuff in here and, in fact, I agree with almost everything Adam says. I really like his idea of naming variables explicitly as Vista – that’s great practice. It can quite confusing for developers to understand how a Vista-encoded variable is getting set (I’ve had that experience). And who could argue with extensive documentation?

It’s what’s missing more than what’s here that bugs me. I see many Omniture implementers who’s attitude seems to be that whatever can be put in the tag should be. I take the opposite viewpoint. Our job isn’t just analysis – it’s optimization – making the Website better. Every single thing you add to a tag adds page weight and execution time.

The best organizations these days are incredibly sensitive to page load times; I think they’re right. If I can collect or classify information without impacting page weight, I’m usually in favor of it. Omniture tags (and some of the Omniture plug-ins) are quite weighty. If you can replace them with a Processing Rule (V15) or a Vista Rule, you ought to consider it. Any discussion of Vista Rules which neglects the benefits of back-office processing as opposed to client-side processing is missing a vital dimension.

It’s often worth the price tag (and the governance pains) that VISTA rules bring to move code out of the tag into the server. With Processing Rules (in V15), this will get even better. In general, I think anything that can be done with Processing Rules should be done with Processing Rules so that you can minimize the tag load and execution time. Because VISTA rules are harder to maintain and cost money, I’m less aggressive about recommending them. Nevertheless, anytime you’re looking at any substantial block of code or any moderately complex javascript execution, you should strongly consider a VISTA rule as an alternative to tagging.

Omniture is a big and complicated system and there is no one right approach to an implementation. Even very skilled practitioners will likely disagree in certain cases. But that’s also why I keep hammering away at the theme that when you’re planning an Omniture implementation, you can’t afford to ignore analytics knowledge no matter how skilled or knowledgeable your technical implementers are.

It’s important to know the ins-and-outs of every technical feature of Omniture. It’s even more important to know exactly how and why you’re going to use those features. Omniture implementations place a significant premium on pre-planning and careful thinking about the problems you’re trying to solve when you create an implementation – and nowhere is that more relevant than in the types of variables you’ll capture and the nature of the variables you’ll use.