Bid Management Algorithms Demystified

During every bid management software sales pitch, the PowerPoint deck inevitably gets to the section on bid management technology. The sales reps proudly proclaim that their brilliant engineers have developed an incredible technology that will not only increase your ROI by 40% in days, but also cure cancer and solve world peace. As proof of the sophistication of this technology, they’ll wow you with complex words like “algorithm”, “portfolio management,” “patent-pending”, and “proprietary code.” Most search marketers at this point in the meeting either nod their head and feign understanding, or start to daydream about their weekend plans. Indeed, if you actually deigned to ask a question about the technology to the sales rep, he probably would have no idea how to answer your question anyway. In sum, most discussions of bid management technology are lost on all parties to the conversation – both marketers and salespeople.

It doesn’t have to be this way! My goal in this article is to outline the basics of bid management science, so that you can a) know what different terms mean; and b) ask the right questions to get the right answers to help you choose the right bid management software for your business.

At a high level, almost every bid management technology falls into one three basic approaches: portfolio management, rules-based optimization, and positional optimization.

Portfolio Management

Portfolio management is perhaps the oldest and most-well known approach. A portfolio management bidding program optimizes your entire search program against a specific business objective. For example, let’s say your goal is to maximize your company revenue but with a constraint of at least a 20% profit margin. A portfolio bid management system might set bids to achieve 40% margin on some keywords, 10% on others, and 5% on others, with the goal of giving you a blended average of 20%.

You might be asking “if I want a 20% margin, when would it ever make sense to bid for a 5% margin?” Consider a keyword that might only get 10 clicks and $10 of revenue at a 20% margin, but 500 clicks and $5000 of revenue at a 5% margin. In the first case, you would only end up with $2 of profit dollars but in the second (due to the fact that you are bidding higher) you would get $250 of profit dollars. If you could then offset this keyword by another keyword that gets 40% margin, you would end up with more revenue and still hit your profit dollar goals.

In my experience, portfolio management technology works best when you have a specific, non-profit based goal for your search marketing. This would include goals like “maximize revenue within a given budget”, “maximize leads within a budget”, or “maximize revenue with a specific margin constraint.”

Rules-Based Optimization

The next type of bid management technology is rules-based optimization. Unlike a portfolio-based approach, a rules-based optimization strategy is centered on keyword-level bidding. As with portfolio management, you start by establishing your business objectives for your search campaign. A rules-based algorithm (note: algorithm is basically a fancy word for the overall program that determines bids) then applies your objectives to every keyword in your account. Initially, it starts with keywords that have lots of clicks – enough to make them “statistically significant” (i.e., there is enough data to be reasonable confident that the keyword’s past performance is indicative of its future performance). If you have an objective of 20% margin, and you have a keyword with 1000 clicks at $1 each and overall revenue from these clicks of $2500, the system would perform a calculation like this:

Total revenue from clicks = $2500; Revenue per click (RPC) = $2.50; Margin Goal (MG) = 20%; Bid = RPC(1-MG). Bid = $2.50(1-.2). Bid =$2.5(.8). Bid = $2.00

The system then might move on to groups of keywords that , taken alone don’t have statistically significant data, when grouped together can be bid collectively. This is known as “clustering”, and most clustering occurs either as result of “semantic clustering” (the keywords are similar words) or “behavioral clustering” (user respond to these keywords similarly).

Rules-based bid management is probably the most common technology found in bid management systems today. This is probably because it is relatively easier to develop and generally works well across many different business objectives.

Positional Optimization

The final bid management approach is called positional optimization. Positional optimization (as the name implies) attempts to find the ideal position in the search results to achieve your business objective. This is far and away the most complex bidding system and deserves a little extra explanation.

If you think about all major paid search programs today (Google, Yahoo, MSN, Ask, etc), they all determine position on their search result page (SERP) based on three variables: Maximum cost per click (CPC), click through rate (CTR) and Quality Score (QS). If you remove QS from the equation, you are left with CPC X CTR, which actually results in a cost per thousand impressions (CPM) system. In other words, the advertiser that shows up in the first position on the page is the advertiser that generates the most revenue per impression for the search engine, not the most revenue per click.

Not surprisingly then, if you looked at the amount an individual advertiser needed to pay for positions 1 through 10 on a SERP, you would see that the effective CPM (eCPM) drops almost exponentially by position. For example, you might have to pay an eCPM of $500 for position #1, $300 for position #2, but at position #5 the eCPM may only be $20. That’s because both the Max CPC and CTR are much higher in the higher positions.

Positional optimization algorithms combine an understanding of eCPM by position with the amount of revenue the advertiser makes by position. Revenue by position is determined CTR multiplied by conversion rate multiplied by revenue per conversion. When you multiple these three factors together you get your revenue per thousand impressions (RPM). For example, if in position #1 you have a CTR of 10%, a conversion rate of 10%, and a revenue per conversion of $50, you would end up with an RPM of $500 (1000X.10 /10 X $50). If the same numbers in position #2 were 5%, 5%, and $40, your RPM at position #2 would be $100 (1000X.05 /5 X $40).

When you subtract your eCPM from your RPM, you are left with your earnings per thousand impressions, or EPM. For example, if the eCPM for position #1 was $90 and the RPM was $500, your EPM for position #1 would be $410. If in position #2 your eCPM was $30 and your RPM was $460, your EPM would be $430. In such a case, it would be smarter to set a bid to achieve second position on the SERP, as the overall profit is higher.

This sort of optimization is very difficult to perfect for many reasons. First, it is hard to predict the right bid to achieve a specific position. Second, the system must account for competitors changing their bids. Third, the system must understand the difference in volume by position. Fourth, most keywords do not have data at every position (or even most positions) which requires the system to make predictions based on limited data.

At the end of the day, however, I believe that this is the most powerful algorithm of the three discussed, simply because it makes bid adjustments based on how the search engines themselves optimize results. In a traditional rules-based platform, you would assume that reducing your bid would increase the difference between your revenue and cost, thus increasing profit. But it is very possible that a bid reduction might move you into a lower position where the RPM dropped at a faster rate than the eCPM, resulting in a decline in EPM. If you don’t optimize the same way the search engines optimize, this scenario is bound to happen.

9 Questions to Ask A Vendor About Bid Management Technology

To complicate matters further, the truth is that most bid management systems today combine elements of more than one of the above theories into their bid equations. As a result, if you ask a vendor “are you rules-based, portfolio-based, or position-based”, they may very well explain to you that they are a hybrid of all three (though again, if you are talking to a sales rep, you are more likely to just get a blank stare anyway!).

Nonetheless, I think you need to ask the question and see what comes back. Here’s my list of nine questions you should ask to determine whether a specific bid management technology is right for you:

1. What type of bid management algorithm is this (portfolio, rules-based, positional, hybrid)?

2. How do you deal with keywords that don’t have enough statistical data?

3. Do you take into account day-parting and geo-targeting?

4. Do you factor in Quality Score? How?

5. Can I set objectives based on revenue? On profit? On leads? On budget? On a combination of the above?

6. Can your system accept historical data (prior to implementation) and use this to “learn” so that it can make smart bid adjustments from day one?

7. Can I manually override your technology for specific keywords/accounts/days?

8. If I wanted you to optimize for profit, how would your system do it?

9. Can you have one of your engineers walk me through the algorithm?

 

10 Tips When Choosing Bid Management Software

For those of you who missed this post on SearchMarketingStandard and on Sphinn . . .

During the Q&A section of the panel I participated in this week at PubCon, we got a lot of questions about bid management software. In particular, it seems that everyone in the audience acknowledged the need for it, but no one knew which company to select, nor how to select a bid management software.

So without further ado, my ten tips for choosing bid management software:

1. Don’t even think about building it yourself. I speak from experience here. Building bid management software requires a full-time team, ongoing maintenance, and a lot of trial and error. It will take you at least a year to build a basic version, and at least two to three engineers to maintain and iterate it after that. And it won’t be as good as the software currently available on the market.
2. Assess your expertise and what you really need. Assuming you listened to my first tip, you next step is to understand how you are going to use the software. First, let’s talk about your level of expertise. If you are an expert, you may want to let the bid management software run your tail terms (the 98% of keywords that make up 2% of your revenue) and focus on optimizing the head yourself. If you aren’t an expert, you probably need software that can manage everything for you, with a very simple interface, and possibly the option of full-service bid management combined with the software. Either way, you need to know exactly what you want before you start talking to software providers. Otherwise, you might end up paying for a Ferrari when all you really needed was a station wagon.
3. Understand implementation and de-implementation effort and impact. A lot of bid management software only works if you install a snippet of code on your Web site and if you allow the bid management company to change your URLs on the search engines. This can require significant effort by your internal tech team and changing your URLs in your search campaigns can result in a loss of keyword history (i.e., you will need to pay more to get the same position). Moreover, you need to understand what happens if you end your relationship with the company – will they change your URLs back, or are you stuck with their tracking for the rest of your life?
4. Always do a trial first. I’ve seen some really great PowerPoint presentations from bid management companies. It turns out its easier to make a good PowerPoint than it is to make a good bid management software. Never sign up for anything until you have taken it for a test drive for at least one month and if possible three or four months.
5. Set benchmarks for initial and ongoing success. Before you start any trial, understand the status quo of your campaigns. What’s your current revenue? Profit? Margin? Tell the bid management company your actual metrics and tell them what you expect them to hit for them to win your business. Make sure you factor in the cost of their services. For example, if a bid management company wants to charge you 5% of your spend, and you currently have a 10% margin on your spend, you should demand that they at least bring you 15% margin (and probably higher). By the way, most bid management companies will thank you for this – it gives them something tangible to shoot for!
6. Look for hidden fees. Does the contract include API costs, or do you have to pay these? Is there a charge for consulting and implementation? Is there a minimum monthly bill? Read your contract carefully and ask a lawyer for help if you are at all confused.
7. Ask for performance pricing. I know my co-panelist Kevin Lee is going to kill me for saying this, but don’t be afraid to ask your bid management company to put some skin in the game. If a company’s bid management software is a good as they say it is, offer them 50% of the incremental profit they make you to prove it! More realistically, perhaps ask them to take a slightly lower percentage of spend in return for a performance bonus if they achieve certain goals (see Kevin, I’m not as unreasonable as I first seem!)
8. Get a short contract. If possible, try to get a month-to-month contract (though this will be hard to do). If you can’t make this happen, a six month contract is usual very doable.
9. Be hesitant about handing over your head keywords. For the 50 to 100 keywords that drive most of your revenue, I usually recommend good old human management. Why? Well I believe that a good search analyst just gets an almost intuitive feel for how to grow top keywords, something that computers just can’t do. And managing your top keywords in-house can save you a lot on bid management fees, especially if less than 50 keywords make up 20-30% of your ad spend.
10. Keep testing new competitors. The bid management world is ever-changing. I see new and exciting bid management companies popping up regularly. Always keep a campaign or two available for the next great thing.