Frederick Vallaeys is a Product Evangelist for Google AdWords. In this role, he helps advertisers learn which Google products can best solve their marketing needs. He also represents the needs of advertisers with the engineering and product management teams. His main product focus is on ads quality and bulk tools like the AdWords Editor and the AdWords API.
Prior to Google, Frederick was an engineer at Sapient and a part-time wedding photographer who found new customers through AdWords. He joined Google in 2002 to help bring AdWords to the Dutch and Belgian markets. He earned his B.S. degree in electrical engineering from Stanford University in 2000.
Key Points
Hoo boy! I went through this interview to try and extract the most important points made, and I will do the best I can here. However, if you are a serious AdWords professional, I’d suggest you read the entire interview from end to end.
The main thing you will get from this interview is that the Quality Score you see in your Google AdWords account differs significantly from the Real Time Quality Score that Google uses to determine how your ad ranks. There is definitely a strong correlation, so Quality Score is a useful metric, but an understanding of Real Time Quality Score can give you an extra edge in understanding what it is you need to do to make your optimization efforts as successful as possible.
Quality Score is the number you see in your Google AdWords account. It is a number between 1 and 10, where 1 is a horrible score, and 10 is an awesome score. Some key points about Quality Score are:
- It is mostly based on historical clickthrough rates of the keyword and ad text.
- Additional factors include landing page quality and load time of the page, but these are secondary factors.
- Quality Score (QS) is based on data from exact match only. Even if you bid on a broad match keyword, such as “cruises”, only exact matches with the keyword are used to determine the QS.
- The published number is the aggregate for all instances of that keyword in your account.
- When you first add keywords into an new account, Google will show the system wide average for that keyword as your Quality Score.
- If you have an existing account, and you add a new keyword, than the account history is a factor in the default Quality Score.
Real Time Quality Score is the number used by Google to help determine your ad rank. It has a lot in common with QS, but is calculated in real time and takes into account many additional factors. Some key points about Real Time Quality Score (RTQS) include:
- Specific query performance is taking in to account. For example, if you bid on “tennis shoes” and someone searches on “discount tennis shoes”, but you sell only expensive tennis shoes, chances are that the resulting user interactions will end up in a low RTQS for this particular query.
- RTQS is personalized to the user based on query history. For example, a recent search on “Rome” followed by a search on “hotels” is more likely to show adds for hotels in Rome.
- RTQS personalization is session based. Once the session cookie is deleted the query history used for personalization is lost.
- Other personalization factors include location and time of day.
- The +1 button does not factor into RTQS … yet. However, it can impact QS and RTQS by increasing Clickthrough rate.
- +1 is associated with the URL, regardless of whether or not it is clicked on in the ad, organic results, or on the web page.
- Site links drive CTR increases ranking from 17% to 30% and can also result in more qualified customers (higher conversion).
- CTR expectations are normalized by position. So if the number 1 position usually gets a 30% CTR and you are getting 20% that is a negative.
- RTQS is determined at the keyword-ad level. There are no ad group or campaign components to RTQS.
That’s it for the summary points. However, in the body of the interview there is much more, including Frederick’s recommended process for optimizing your QS and RTQS, lots of examples, and why bidding your keywords high when you first launch them is a smart thing to do.
Full Interview Transcript
Eric Enge: Can you tell me how Quality Score is used?
Frederick Vallaeys: The Quality Score is Google’s way of ensuring that we show the most relevant ads to our users, and we deliver high quality leads to advertisers buying the clicks from us. The Quality Score obviously factors into the ad rank together with the advertiser’s bid.
It helps determine which advertiser has the highest position on that page. The Quality Score that you see in the account is determined by a number of factors and is mostly based on the historical click through rates of the keyword and the ad text.
The Quality Score is only based on data from results on exact match.
The Quality Score is only based on data from results on exact match. That means the keyword the user types in has to be exactly the same as the keyword chosen by the advertiser. There has to be an exact match between those two regardless of which match type the advertiser selected. Also, we only use data from google.com, not display network traffic or traffic from our search partners.
That’s the data that builds up the Quality Score. We also have additional factors such as landing page quality and load time of the page, but those are secondary factors. The biggest thing we look at is the historical click through rates of the ad text with the keywords inside of the account.
Eric Enge: That’s specific to what we see published in AdWords, is that correct?
Frederick Vallaeys: Exactly. What you see published in AdWords is going to be a number between one and ten. A Quality Score of one out of ten is a terrible Quality Score, and a score of ten is a fantastic Quality Score. What you have to keep in mind is that the number we publish is the aggregate for that specific keyword. It reflects all the data we have on that keyword for your account.
The key point here is that this is an average, and an average is never great which is why we also calculate a Real Time Quality Score internally.
The key point here is that this is an average, and an average is never great which is why we also calculate a Real Time Quality Score internally. The average you see in the accounts is good for figuring out where you have an issue.
As an advertiser, if I have to prioritize which keywords to optimize, this is a good indication. Any Quality Score below a seven is a place where you might want to start looking. The lower that number the bigger an issue you have.
Eric Enge: When you open up a new account, and there isn’t any click through rate history, I’ve seen situations where the Quality Score is quite low but the numbers come up as the account ages.
Frederick Vallaeys: Right. What typically happens when you start up a new account, or you put a new keyword for the first time into an existing account, is we take a system-wide average based on advertisers who have run on that keyword in the past. What often happens is that the keyword may be fairly broad and may not be the best performing keyword system-wide.
As your account ages and you start getting impressions and clicks on that keyword, we can build a specific picture of how you, an advertiser with those specific ad texts, will do on that keyword. If you are a good advertiser that knows how to write a compelling ad text for all the keywords, your Quality Score will certainly increase at that point and become much better. It also becomes your own Quality Score as opposed to that starting point system-wide average.
Eric Enge: Can keywords with a bad history have a negative impact on another keyword’s quality score?
If an account has a set of keywords that in aggregate have a low QS, this can have a negative impact. Zero impression keywords do NOT matter because those contribute no CTR data.
Frederick Vallaeys: In the absence of specific data about how a keyword performs with a specific ad, we rely on system wide data and account-level data. If an account has a set of keywords that in aggregate have a low QS, this can have a negative impact. Zero impression keywords do NOT matter because those contribute no CTR data.
Keywords with few impressions and few clicks could in aggregate have a large number of impressions with a low CTR and this could hurt the account. Keep in mind though that even if there is a negative impact on the account, this won’t matter as soon as we have enough data about how a keyword performs with a specific ad because we’d use that specific data for QS rather than the less specific account level data.
Real Time Quality Score
Eric Enge: Let’s say we have a keyword such as “tennis shoes.” How is Real Time Quality Score, both displayed and calculated?
Frederick Vallaeys: Many people will type in “tennis shoes” but others may type in variations of that keyword such as “discount tennis shoes” or “Nike tennis shoes.” If you had that keyword in the broad match then your ad would have been eligible to show on these different variations.
For the Real Time Quality Score we calculate at the exact moment a user did the search and take into account what these variations are. If you sell expensive tennis shoes, and someone did a query for discount tennis shoes, we would show your ad and maybe that ad had an eight out of ten Quality Score. It’s a mismatch to what that specific user was looking for because they weren’t looking for expensive tennis shoes. In that case it would not be the best ad to show.
The real time system allows us, based on the additional data for this specific situation, to know this ad is not the best ad for that case, and to give preference to some of the other ads.
We think it’s a real positive for advertisers, because in the past we would aggregate and you would get clicks that maybe weren’t from the most qualified potential customers because we were looking at averages. Now we can look at how they formulate the query and how that impacts their likeliness of being interested in this advertiser’s ads.
Instead of a eight out of ten, the Real Time Quality Score might be a five out of ten telling us this ad is not a great ad for this query. This will affect the ad rank and, in some cases, the ad doesn’t show.
In the “tennis shoes” situation, when someone types in “discount tennis shoes” we are looking beyond exact match and you have a separate Real Time Quality Score calculated for the performance of the query “discount tennis shoes” against that keyword, that ad and that landing page. We could look at some interesting cases that would match really ambiguous keywords which are difficult to bid on.
If you as an advertiser pick that relatively generic keyword, we can find a subset of queries that do well for what it is you are selling.
For another example, consider the keyword “jobs.” You could be looking for Steve Jobs or you could be looking for jobs in San Francisco. How do we know? If you as an advertiser pick that relatively generic keyword, we can find a subset of queries that do well for what it is you are selling, whether it’s a blog about Steve Jobs’ company or whether it’s a blog or website for finding a job in San Francisco. Many years back, the AdWords system wasn’t quite as specific with its Quality Score. What it would do in these ambiguous cases is not run the advertiser’s ads because we would say, “okay, on average this is a pretty bad keyword, it doesn’t perform that well.” We would lose sight of the specific queries in which it actually did do well.
With the more sophisticated system we have today, if there is a small subset of queries that work well for you, we can find those and often show you in quite a high position even though all the other queries for that same keyword might not have done well for you.
Another example I like to use is “discount cruises.” If someone looks for discount cruises, it’s not ambiguous in terms of what they are looking for, but it could be ambiguous in terms of the destination they are looking for.
There are companies that offer Alaskan cruises and companies that offer Caribbean cruises. Generally, people are more interested in the Caribbean or warm weather cruises. With that generic keyword “discount cruises” you might do well on most queries because most people want to buy your Caribbean cruise.
In those few instances where someone is looking for an Alaskan cruise, it would be a poor decision to show your ad because you don’t sell that cruise. If we had gone on the average, we would have shown the ad because most people look for Caribbean cruises.
This provides a better user experience because users aren’t seeing an ad for Caribbean cruises just because it happens to have a high overall Quality Score.
With the real time system we see that the user typed in the word “Alaskan” in addition to “discount cruises.” This is probably not the best time to show the ad, and it prevents the advertiser from showing an ad that’s unlikely to lead to a sale. This provides a better user experience because users aren’t seeing an ad for Caribbean cruises just because it happens to have a high overall Quality Score.
Personalization and selection of Ads
Eric Enge: In the scenario above, where the user provides more information based on adding a qualifying word to the query. For discount Alaskan cruises you don’t show the Florida or Caribbean cruises ad. Could you look at the user’s past query history and see that they recently read blogs about Alaska or things of that kind? Is there anything like that in play at this point?
There is a personalization factor in place. This works by looking at previous queries the user has done … when we talk about personalization it’s actually on an anonymous basis.
Frederick Vallaeys: Yes. There is a personalization factor in place. This works by looking at previous queries the user has done. A good example of this is a user came to Google, did a search for Rome, and the next search they did was for hotels. What Google knows is that they probably were thinking about hotels in Rome as opposed to hotels anywhere. Rather than show generic ads for hotels, we can look back at that session data and show more relevant ads based on that. That’s the extent of what we can do at this point.
I would like to note that when we talk about personalization it’s actually on a anonymous basis. It means we know what a certain cookie is doing, but we don’t know what a certain person is doing. We know that cookie ID 1234 searched for Rome before they searched for hotels, but we don’t know that the cookie is Frederick Vallaeys.
Eric Enge: You obviously have to avoid the privacy concerns. Does the cookie that allowed you to do this survive across its sessions?
Frederick Vallaeys: No. We found that’s usually not a great thing to do because the correlations you start seeing actually go down quite a bit. Also, we don’t always combine the previous searches to the current searches because if there is a clear shift in topic that the user is searching for then it doesn’t make sense to look at that previous data.
Eric Enge: This personalization that we spoke about is a factor in Real Time Quality Score?
Frederick Vallaeys: The other mechanics we look at are the location of the searcher and the time and day. There are a number of other factors we don’t disclose, but we do evaluate many factors that could potentially have some impact. We look at CTR, and if there a strong correlation between this factor and CTR, that’s a factor we could continue to use. Location and time are two good examples that do matter.
Eric Enge: If it’s November and somebody in Massachusetts typed in “discount cruises” are you more likely to show a Florida cruises ad than an Alaska cruises ad?
Frederick Vallaeys: Exactly. We might give preference to an ad on Florida and Caribbean cruises for people from a cold location.
Eric Enge: Correspondingly, if you have someone in California typing that, you might actually show a Hawaii cruise ad rather than a Caribbean cruise ad.
Frederick Vallaeys: Exactly.
Eric Enge: What are some of the correlations for time of day?
there have been a number of studies in the travel industry that show in the morning people tend to research hotels they may stay at. At lunch they talk to their spouses to get approval to book a certain hotel. In the afternoon they may be more likely to book that hotel.
Frederick Vallaeys: You can think about differences in behavior even if they were searching for the same thing at different times of day. For example, there have been a number of studies in the travel industry that show in the morning people tend to research hotels they may stay at. At lunch they talk to their spouses to get approval to book a certain hotel. In the afternoon they may be more likely to book that hotel.
So, if we find a query in the morning for a certain type of item, we might give preference to more research-oriented ads, and in the afternoon we may focus on more transaction oriented ads. That’s difficult so the system depends on having enough statistically significant data to make those decisions.
Eric Enge: Right, because you don’t know if they went and talked to their spouse, but you do know they tended to click on review-oriented ads as opposed to book-it-now oriented ads.
Frederick Vallaeys: Exactly.
The role that the +1 button plays into the Quality Score
Eric Enge: What about the +1 button that you now see appearing on ads. Is that something you factor into a Quality Score at this point?
Frederick Vallaeys: It doesn’t factor into the ranking yet. However, what we typically see whenever a new ad format or a new feature of an ad is introduced, such as the +1 button, is that it sometimes increases click through rates. If the click through rate increases, that leads to a better Quality Score so there is definitely an indirect factor by having strong +1 recommendations and endorsements that more people could click on your ad.
+1 is essentially bringing social to the moment of relevance.
+1 is essentially bringing social to the moment of relevance. If a user sees that five of his buddies have booked the same vacation or done business with the same cruise line that’s a pretty strong endorsement and that user is more likely to also click on the ad, check it out, and buy from them. If you as an advertiser can build that following of +1 clicks and get people to endorse you that should be positive for you. If that seems to be a useful thing to use in terms of Quality Score, we absolutely could start thinking about integrating that.
Eric Enge: Are people clicking on those +1 buttons in the ads in any volume? I could see +1′ing a great article, but I’m not sure what the proclivity would be of people to +1 an ad.
Frederick Vallaeys: That raises another good point which is if you are using +1 as a publisher, an advertiser, or a business the +1 actually is associated to a certain URL. So, even if you don’t have a +1 next to your ad, but you get people to +1 your website, that all feeds into the same pool of data.
Later on when somebody searches and sees your ad, those recommendations will show up even if those +1′s were done from your website or the organic results. It’s a whole ecosystem that persists across all the different touch points you might have with that customer, whether it be through Google or through your own website.
As far as the volume of how many people have done this, I can’t talk about that. It’s still early stages for this, but we are pleased with the way people are using it at this point.
The power of using the new ad extensions
Eric Enge: One of our clients is using the seller rating ad extensions. That’s kind of a corollary, this whole business of including reviews and ratings into the whole process.
Frederick Vallaeys: Exactly. I think it fits into the bigger picture of new ad formats you see on Google, and it stems from the fact that we realize that sometimes the picture is worth a thousand words, and the ad doesn’t have to be purely text.
You can also answer with a map. If it’s a local search you can enhance with product prices and images if it was a product search. If it was a search for a new movie then it might make sense to show the trailer right there. Positive seller ratings and reviews are a good thing to surface because it helps build trust and brings in those clicks that an advertiser was looking for.
We’ve seen site links drive increases in CTR anywhere between 17% to 30%.
A specific example to look at is site links, which is probably the easiest of the new ad formats to implement because it’s literally going into your campaign and putting in up to ten links associated to each of your campaigns. We’ve seen these drive increases in CTR anywhere between 17% (search) to 30% (mobile). These are fantastic increases in CTR simply by showing more information that’s useful to users.
Eric Enge: Similar to the +1 button, it’s something the eye notices and attracts a little bit of mind space.
Frederick Vallaeys: Exactly. We want to be careful because people are drawn to new things, but we need to make sure that those new things are not just drawing clicks because they are different, but because they are actually useful. We are careful in terms of launching these new features and testing them and making sure there is actual user benefit in them.
On the flipside, when the user sees more it typically also means they are better qualified by the time they make the click and come to you as an advertiser, so you are more likely to convert that customer. A great example of this is again in the travel space.
Let’s say someone is looking for a destination and you have a travel site with car rentals, hotels, flights, and vacation packages. In the past you would have taken that user to your generic page where they could have done all four of those things. But, if you now show four site links to each of those different areas of your site, you’ve done two things.
You’ve told that user “hey, by the way you might not have realized it, but we also do car rentals.” The second thing is the user goes directly to that page for the thing they were looking for. Now you can take them to a page where, instead of cluttering it with the things they weren’t looking for, you actually put special offers and pitch the product they were looking for.
In the case of car rentals you show them what discounts are available in the space that you might have otherwise had to use to say “hey, you can also book flights here” which they weren’t looking to do at the time. It’s a positive thing for both the user and the advertiser.
Eric Enge: I saw what Apple did with site links. They show their current hot offers. It’s a very, very smart way to use that feature.
Quality Score and Position Normalization
Eric Enge: Coming back to Quality Score and the click through rates. I assume you have some way of adjusting expectations based on positions, because obviously one would expect the first ad to get the most clicks. To put a strawman concept out there, if we thought the first ad was going to get 30% of the page search clicks, and the second was going to get 15% and so forth then if the first ad gets 25% and the second ad gets 20% then that starts to be a sign that the second ad is the better ad. Am I interpreting that correctly?
Position normalization says that we have different expectations for CTR for the different ad positions.
Frederick Vallaeys: Yes, you are spot on with that. We call it Position Normalization, and it’s exactly as you described. Having a certain CTR, say 25%, could be a really good thing if we were expecting you to get 15% in the position that you were in. Your Quality Score could go up. Many advertisers look at the CTR in their accounts and try to judge everything on that. However, it’s important to look at both the CTR number as well as the Quality Score number in your account.
Eric Enge: You want to look at them together as it’s a relative thing.
Frederick Vallaeys: Exactly. You look at them in combination, and the more important thing to look at in your account is the return on investments you’ve received from those ads. The Quality Score is a number we put in there to help you figure out where it is you could perform better and possibly decrease your cost and increase your position by having more relevance. If that is driving ROI, then that’s the only thing that matters to advertisers.
Eric Enge: You don’t want to lose sight of the end goal. The Quality Score is basically a tool to help you better get to that goal. The point you just made about the Position Normalization, is that you get to look at all the things together. I need to look at it in a holistic fashion so it can tell me where the opportunities are.
Frederick Vallaeys: Exactly, and a simple technique is to look at which of your keywords have a sub-bar Quality Score; and that could be any number. That could be the lowest ones in your accounts or it could be literally at a one level or a two level. Then you can look at your search query report.
From that you start seeing these different variations, and now you can start figuring out why is it that it wasn’t performing well at the aggregate level, and then how can I make my account more specific by building out new ad groups for these different search queries that we are also triggering.
Typically, when you do that, you increase your relevance because you are now taking more specific keywords and building ad text specifically for those which help you boost up your click through rate.
Tips for optimizing your AdWords account
Eric Enge: If you are a publisher that wants to do optimization on your account, what are the steps you recommend publishers should go through?
Frederick Vallaeys: I recommend that you look holistically at your accounts. Sort it on a keyword basis from lowest to highest Quality Score and apply some filter so you are not looking at anything that doesn’t have a lot of impressions yet.
Look at which ones have the highest volume and not a great Quality Score.
I would say a thousand impressions and up. That’s the baseline where you would start looking at it, and then do a secondary sort on that. Look at which ones have the highest volume and not a great Quality Score. Go after the high volume first even if it’s not necessarily the absolute lowest Quality Score, but it’s still in that bucket where the Quality Score is not quite where you want it to be, and start optimizing on those.
Then try to figure out if you could write better ad text for that keyword as it stands now or do you need to break that keyword into more specific variations, build new ad groups around that to create ad text that’s more compelling and maybe lead it to a landing page that’s also more specific.
Eric Enge: Is there an ad group or campaign level component to Quality Score?
Frederick Vallaeys: The QS is at a keyword-ad level. So the way you structure ad groups plays a large role in determining QS. However there is no ad group or campaign QS component. I.e. if you took the same keyword and ad and moved it to a different ad group or campaign, the quality score would remain the same.
Eric Enge: We did talk about Position Normalization earlier, but is there an argument in some situations for bidding higher? To drive history faster, or do things to try to help the Quality Score go up?
I think you hit the nail on the head with the statement that it (bidding higher) helps you build history faster in some cases.
Frederick Vallaeys: I think you hit the nail on the head with the statement that it helps you build history faster in some cases. Keep in mind when you bid higher it usually means you are going to get a higher position on the page.
In those higher positions if you go from being on page two to page one, that’s going to have a huge impact on how quickly you accrue impressions. It’s those impressions that will give Google the confidence to make a Quality Score judgment that’s specific to your account as opposed to the system-wide averages.
If you, as an advertiser, are doing much better than the system-wide average then it would benefit you to prove to us as quickly as you can because that will then decrease your costs in the long run.
It’s about building that volume, but not about anything else because there is Position Normalization. Bidding up to a higher position and getting that higher CTR isn’t a guarantee of getting a better Quality Score in the long run.
Eric Enge: Right, because presumably the Position Normalization is adjusted on a keyword basis. Position normalization for market expectations on one keyword might be different than the expectations on another keyword.
Frederick Vallaeys: Right.
Eric Enge: That eliminates any possibility that you could fool the Position Normalization algorithm with the bids. The only thing you gain is that you can accelerate the development of your own history.
Frederick Vallaeys: Exactly.
How the real time math helps advertisers
Eric Enge: In summary, the Quality Score we see in AdWords is actually a very valuable proxy basically for the real numbers because you can’t possibly handle the data for the real numbers as a human.
Frederick Vallaeys: Exactly. That brings up another good point. One thing I like to harp on is that Google has a lot of data, and we are very good at using that data to give the best results to advertisers. Conversion optimizer is actually a good example of this.
To the point that you just made, we at Google collect data on a query-by-query basis, can have an expectation of how that’s going to perform. The problem is that even if you had that as an advertiser, there would be no way for you to bid in real time based on those factors.
That’s where Google can actually do a good job for those advertisers, and that’s where conversion optimizer comes into play. That’s using all of Google’s power of crunching numbers to make sure that you are meeting your ROI targets, and let us handle all the heavy lifting of determining the right CPC.
Eric Enge: Thanks Fred!
