This question invariably gets asked in every panel or interview revolving around header bidding. “Hack” is a term dripping with bile and contempt – it symbolizes a ragged workaround versus some mythical notion of organic development.
But such development rarely occurs in the digital advertising space – its sordid history is filled with hacks, whether it was hard-coding display ads or the early days of real-time bidding (i.e., pre-OpenRTB).
In both of those cases, the hack was merely a bridge across the muck into a greener pasture. So the best answer to that oft-repeated question is, “Yes, but a lot of great developments in advertising technology have been hacks that lead to better systems.” Header bidding is actually a great hack, and also an attempt to hijack Google’s walled garden.
Even so, can we picture the pasture past the bridge – a world beyond the hack and even the walled garden? Yes, but first we need to understand how header bidding has forever altered the transactional landscape, and then reconsider how the ad server itself operates.
Rise of Header Bidding, Fall of the Waterfall
To understand the rise of header bidding, we have to examine its larger purpose beyond bumping up publisher CPMs and giving buyers holistic views of publisher inventory. Header bidding’s ultimate goal is to make the waterfall defunct.
In the past, there was a reason for the waterfall, but these were days when the line between direct- and indirect sold inventory was solid and sharp, not blurry like today. Once upon a time, publishers would try to sell all available inventory on an upfront, guaranteed basis through their direct sales teams. Unsold inventory – sometimes given the filthy label of “remnant” – was the province of ad networks.
Publishers needed the separation to create a caste of “premium” inventory. They would cordon off inventory so it could only be purchased through direct sales, creating scarcity. They would block their prized advertisers from getting on their sites through indirect channels and potentially devaluing their inventory.
When a user arrived on a page, the ad server would check if there was a direct-sold campaign ready to ship. If none were (or if current campaigns had hit roadblocks like a frequency caps) and the placement wasn’t considered direct-sold only territory, the call would trickle down to the ad networks. These would have price tags loaded lower in the ad server. The call would keep working its way down until the impression was filled… Or there was no further south to go.
Programmatic channels – notably real-time-bidding-based auctions – changed the game entirely. Now advertisers could leverage data (via cookies) to target creative at choice users/audiences on an impression-by-impression basis in real time. No longer would they have to use publishers as a proxy for the audiences they wanted to hit.
However, at first the pricing wasn’t nearly as appealing for publishers as upfront guaranteed deals, and many advertisers attempted to use the platforms to “cherry-pick” target users on premium sites for ludicrously low CPMs. No surprise that publishers relegated RTB channels below direct-sold inventory within the waterfall, despite buyer and vendor protestations that programmatic didn’t equal remnant.
After a lot of painful bumping of shoulders between direct and indirect channels, mass advertiser adoption of RTB and developments like private exchanges bumped up programmatic CPMs to the point they could realistically compete with direct-sold campaigns. More and more advertisers wanted to take advantage of programmatic perks like data targeting in procuring premium inventory, and were willing to pay more to get at the so-called premium inventory.
Suddenly the waterfall was an obstacle – within an ad server, SSPs and other partners were only inserting a tag with the average price a bidder in their auction would pay for publisher inventory. This average could actually be far lower than the winning bid, but because the call had to trickle down to this tag, publishers could not know if they were leaving money on the table. Now that all demand partners could insert bids in near real time, the waterfall was preventing publishers from acutely comparing pricing between channels and partners.
But this limitation was advantageous for certain ad servers that also operated exchanges and/or demand-side platforms. All right, you know we’re mainly talking about Doubleclick for Publishers (DFP) by Google, the most widely used publisher-side ad server in the US. Google made the display landscape less competitive by launching Dynamic Allocation in 2014, which enabled its exchange AdX to insert a real-time bid into DFP for every impression.
Thus AdX could enter accurate pricing while other partners were stuck with their average tags, even though their bidders could potentially cite a higher price. Theoretically, Dynamic Allocation could enable AdX bidders to pay less for impressions than other partners would be willing to, therefore starving the publisher of revenue. This seemingly unfair setup spurred the adoption of header bidding.
Buy-side retargeters first utilized the concept of header bidding – installing code in the header of a browser that signals a demand partner, possibly holds an auction and then sends back a relevant bid through the header to the ad server. Within the ad server, the relevant bid is matched with its correct price tag out of hundreds installed in small increments.
SSPs saw an opportunity to get around Dynamic Allocation and developed their own header technology. The buy-side won the advantage of getting access to all the inventory publishers had previously cordoned off, allowing them to better evaluate what they were willing to pay for impressions. Publishers were lined up with buyers willing to fork over more cash for their inventory and display CPMs started trending upward.
Word traveled fast and publishers were integrating so many header partners that SSPs and exchanges introduced container tags known as wrappers that housed numerous partners’ code and made implementing new sources far easier.
Although ad servers are now flooded with partner price tags, header bidding has enabled the creation of virtual meta-auctions where all of a publisher’s demand sources can compete – direct or indirect sold. The playing field is finally level and they all lived happily ever after….
Or not.
Convolutions and Contortions
If the whole header bidding setup sounds pretty convoluted to you, join the party. This is why the tech is a hack – at its core, it’s a workaround the waterfall. But as you can imagine managing so many partners through so many integrations with so many corresponding tags is a burden… Oh, did we talk about reporting and evaluating partner performance? The display landscape may be a more level playing field, but at the cost of being woefully complicated and hard to oversee.
At sometime you have to consider how you’re going replace the stop-gap with a feasible, long-term solution. There’s no time like the present, because the downsides of header bidding – latency, data drain and the potential for endless arbitrage of publisher inventory – are becoming painfully obvious.
The tech puts a lot of weight on the user’s browser – the speed of the auction itself is limited by a user’s connection. For a desktop browser hooked up to wifi, this isn’t terrible, though it still can be lacking and slow down page load in an era of increasingly impatient Internet users.
On mobile – where traffic is fast migrating – connections tend to be slower and data is pricier. The latter causes concern because header bidding machinations are based in the browser, and could suck up data over time. The mobile web experience is already drawing a great deal of consumer ire in terms of latency and data drain (see: mobile browser ad blocker adoption), so it’s a risk to further irritate the audience.
(Because there is no browser and hence no header, in-app “header bidding” is done through an SDK or API connected to or within the mobile ad server. The technology is still in its early days.)
So the header talk has moved onto server-to-server (S2S) connections, which are lightning fast and can transmit far more data without weighing on the browser – your partner’s server does the heavy lifting before sending back the final valuation.
Ideally, you’d have an S2S connection within the ad server – which is exactly what Google’s Dynamic Allocation is supposed to be. In response to header bidding, Google has announced Exchange Bidding in Dynamic Allocation (EBDA), opening up its S2S connection to demand partners The Rubicon Project and Index Exchange.
However, this experiment is still filled with questions including whether Dynamic Allocation is serving as the mediation platform (i.e., picking the bid that goes through to the ad server rather than letting the server play decision-maker), how much data is shared, what chunk of change Google is grabbing from both partner and publisher, etc. It’s a bit of a black box.
Unfortunately, because of its nature Google is not going to open up DFP to each and every partner (the risk is high to hurt Google media revenue as true competition would force all intermediaries to significantly lower their margin), which may rob some publishers of prized demand. When DFP does enable demand source integration, the company may leverage an amount of control unacceptable to partners and publishers. Ultimately, this power grab will ruin the notion of a true meta-auction and unhampered demand between channels and sources.
As an alternative, some providers have introduced header-based S2S integrations to take the weight off the browser. Some have even built wrapper-like solutions that enable other demand sources to use their S2S tubes and potentially hold auctions on their servers, but similar to EBDA, questions over control and pricing are making the technology companies reluctant to team up. Some header partners claim that being on the page is a necessity.
Unique header-based S2S connections seems the path forward, but since publishers have shown a huge appetite for demand sources – an average of five or more partners, according to a recent AOL survey – so many integrations might require a bevy of resources… So many that it’s not actually worth the uptick in revenue.
However, there is another option.
The Case for an Open Ad Server
When I casually asked what she saw as the future of header integrations, an industry colleague suggested, “The death of the ad server.”
Has our old buddy the third-party ad server outlived its usefulness? Sure, every SSP can serve ads, so theoretically you could turn control to a header-based integration that you were also using as a mediation platform for your other demand sources.
But in effect that becomes an ad server – you can’t deliver creatives to their proper placements (and provide reporting on these actions) without one. So instead of going gently into that good night, the ad server – or at least our notion of the ad server – needs to change. It needs to open up.
The buy-side is increasingly pushing for direct connections into ad servers for several reasons. First off, it’s a way around the dreaded the “ad tech tax” – also known as death by a thousand intermediary cuts – and offers the transparency advertisers have long desired in the digital space.
Especially in the wake of Association of National Advertisers reports lamenting kickbacks and other unsavory practices by agencies, getting away from hints of arbitrage (in particular, agency programmatic platforms buying or agreeing to buy large swaths of publisher inventory for a low cost only to mark it up when delivered to clients) and demonstrating the mechanics of the transaction (a.k.a., “how the sausage is made”) is a priority.
Second is the push toward programmatic guaranteed – agencies and publishers agree to a guaranteed amount of spend and/or inventory, but the transactions are executed through programmatic channels such as private marketplaces. It’s the best of both worlds – sellers get upfront spend and buyers can select impressions on a real-time basis.
While header integrations offer the machinations to make these kinds of deals work – notably by offering buyers holistic views of pub inventory – the complexity of current setups means limitations on scale. DSPs and agency trading platforms have shown interest in header integrations, but truth be told it would be simpler and easier for all parties to deal with an S2S connection from the buyer right into the publisher ad server. No intermediaries, fewer auctions – the whole display (and eventually video) ecosystem benefits.
The closed model relying on tags in the waterfall is incompatible with programmatic guaranteed. Even DFP-managed S2S integrations are not going to cut it as noted above in regards to EBDA. Transparency is still in short supply – hidden fees, the potential to cut off access and partner rejection all inhibit programmatic guaranteed.
This is not to say that the next-generation ad server needs to be agnostic (i.e., not have a demand business), but that certainly would prop up its credentials as being strictly about the transactional tech. Publishers need the ability to manage S2S integrations and partners within the ad server to offer true meta-auctions and better serve their top advertisers.
What the buy side wants, it typically gets – there’s something to be said for appealing to the people holding the purse strings. While an open ad server would be greatly beneficial to publishers by offering them far more control, don’t be surprised if buy-side demands end up driving drive the push.
Beyond the Hack
There’s some worthy ad-tech style irony in the fact that in killing the waterfall, header bidding has made buying and selling digital media more complicated. As is often the case in digital advertising (think vendor consolidation), the path forward requires simplification.
We can all agree that header bidding is a hack, but moreover it is a hijack of the walled garden. An open ad server doesn’t require this hijack. It’s time to shift our focus from the workaround to the long-term solution. And that involves reconsidering how the ad server operates.