Some premium publishers are likely reading the latest batch of hysterical articles describing the scourge of botnet traffic on digital advertising and thinking, “Tough cookies.”
Which isn’t a bad way to put it – media buyers’ obsession with chasing cookies and buying the cheapest inventory available through RTB-powered exchanges has created a great deal of turbulence across the digital mediascape. Certainly the endless cookie hunt has driven innovation (e.g., private exchanges, data management), but the havoc wrought is becoming increasingly visible. Botnets are gobbling up high-value cookies, then luring exchange and network buyers to propped-up sites to purchase impressions seen by no one.
The slew of recent high-profile articles have focused mainly on how advertisers are getting screwed and the coffers of scammers are filled a fraction of a penny at a time. However, the real victims of botnet fraud are the companies at both ends of the digital media spectrum: that wasted brand spend should have gone towards “premium publishers,” which can be now be taken to mean “sites that real people actually visit.” Yes, publishers with living and breathing audiences that advertisers would like reach are losing out on revenue to scammers… And the intermediaries that enable them.
For premium publishers, the fraudulent traffic problem doesn’t end at lost revenue. Botnets can present serious problems for many premium publishers – especially those with high-value audiences – but also opportunity that can be capitalized on simultaneously.
Problem in Perspective
Short for “network of robots,” botnets are hordes of zombie browsers, typically infected through some type of malware, ordered by a controller to crawl a variety of sites – typically for nefarious purposes. Sources suggest the percentage of botnet traffic out there fluctuates between 15% and 20%. Scammers using botnets may sell this fake traffic to publishers looking to boost their numbers to impress advertisers. Or the scammers might be setting up scab sites (sometimes vast networks) with names like “happymomtime.com,” crap or scraped content and a slew of impressions per page. Get these onto ad networks or public exchanges and watch the money pour in; even when these junk sites are discovered and blacklisted, hucksters speedily set up new ones and the process starts again.
These botnet fraud techniques are not new – those who have been in the industry long enough will recall them as central to the click fraud plague as well as search engine manipulation. But botnet programmers keep getting smarter, even as media folk wise up to their tricks. Now botnets can register video views, fool ad viewability protocols and seriously muddle attribution funnels.
The biggest issue of the moment revolves around botnets scouring the web for high-value cookies and fooling platforms into believing they’re targeting in-market consumers. There’s a lot of money to snatch there as media buyers at squeezed agencies forgo context and content in the hunt for those precious, precious cookies.
Networks and exchanges have a nasty conflict of interest when it comes to botnet-riddled sites – in the short term, they’re making money off tainted transactions, but in the long term they will shed disillusioned buyers. Still, clearing thousands of sites of riffraff is easier said than done, especially on a continual basis (though solutions are fast appearing).
Premium Pub Dilemmas
So, sorry to say, premium publisher, your site probably has at least some botnet traffic running about, and you best do something about it because the consequences can get ugly.
First, you need to figure out whether that traffic is welcome or intruding. I won’t go into the controversies around the practice, but know that buying traffic typically opens the door to botnet traffic. If you’re going to buy traffic, know the risks, and realize that the cheaper traffic is, the more likely it’s bot gridlock.
Buying low-quality traffic might be the quickest path onto verification and fraud prevention company blacklists. Blunt instrument it may be, blacklisting URLs is the primary tool in fighting bot traffic. Getting on a blacklist could spell serious trouble for a mid-tier, niche or up-and-coming publisher relying on a programmatic boost for unsold inventory.
Those of you working for well-known, heavily trafficked properties are not in the clear either. If they traversed only scab sites, it would be easy to isolate bots; instead they travel to popular legitimate sites to give off the appearance of a real person. The referring URL for this Mommyfuntime.com impression is from NYTimes.com? Suddenly weeding out bots requires deeper analysis of traffic patterns.
Also, as mentioned earlier, botnets go searching for sites and pages with high-value audience cookies – where’s a better place to find them than a comScore 100 property? Of course, this isn’t just a problem for the big guys – a niche site with an attractive audience for a small but heavy-spending advertising pool could find itself swimming in bots. Publishers actually perpetuate the bot problem by dropping cookies on whatever comes their way.
Whether you’re a big or small legitimate publisher, bot fallout could be ugly: in addition to the specter of being blacklisted, advertisers buying directly may lessen or cut off their spends. Who can blame them if it seems a heavy percentage of their ads are being served to zombies?
The savvy publisher will argue that such wastage is built into pricing – a similar argument to the one used against viewability metrics. (On a related note, viewability is not the answer to dealing with bots either – bots are increasingly programmed to fool those metrics.) But in an era where squeezed agencies will fight tooth and nail over every digital dime, publishers shouldn’t give them that kind of leverage for negotiating price.
Especially not when you can stop bots from receiving ads and cookies.
Preventive Measures
The first step in tackling the bot menace is knowing what you’re dealing with. Verification and anti-fraud services can drop a pixel on your site, analyze your traffic on a continual basis and give you an idea of how much suspect traffic is running. This can be especially useful to those publishers buying traffic and data – if one resource in particular is sending in fishy prospects, best cut it off. However, rarely is one stream 100% fraud – a stream with a bit of suspect traffic might also sending in gems.
That’s where the next-level solutions come in. When suspect traffic arrives, service providers can prevent the ad server from dealing impressions or cookies in real-time. However, for most companies the decision to block the ad serving is based on whether the referring URL is on a blacklist. While blacklists are constantly updated, they’re suspect themselves – perfectly valid sites may end up on them for reasons detailed in the last section. Also mentioned earlier, scammers set up new scab sites as soon as they realize they’ve been blacklisted – being “up to date” is a relative term in this warp speed environment.
An advanced technique to fending off bots, detailed to me by Integral Ad Science CEO Scott Knoll, is to analyze the browser itself in real-time. It’s something akin to the “Voight-Kampff” Empathy test in “Blade Runner.” (Yes, I’m going full-blown nerd on you.) To tell the difference between a human and a replicant (basically an android with organic parts), a psychological test is administered using tools that measure bodily functions (in particular, eye movement). Replicants have a tell-tale reaction – an eye twitch.
In a similar fashion, the service provider rapidly fires a series of questions at the browser when it arrives on site and analyzes the answers in real-time. Bot-powered browsers will always respond with some detail that gives away their non-human status. Through bypassing unreliable blacklists, theoretically this method would seem to better ensure only real eyeballs are shown ads.
Always keep in mind that scammers are constantly upping their botnet game, building smarter programs that can fool detection. To reference “Blade Runner” again, the advanced-model replicant Rachael requires more than 100 questions before giving herself away; the average is 20-30.
Opportunity Knocks
Within the preventive measures lies the opportunity. While the slew of bot-related articles in the trades contain a great deal of finger-pointing (It’s the networks’ fault! It’s the exchanges’ problem!), the real lesson for advertisers here should be that the the days of wild cookie-chasing are coming to a close.
The best way for advertisers to avoid massive amounts of fraudulent traffic is to work with proven publishers – if agencies want to buy programmatically to use first and third-party data, they’re better off employing the private exchanges of publishers they already trust (e.g., buy directly from) than braving the fraud-riddled open exchanges.
For publishers, then, a certification of bot-free traffic is an incentive in getting advertisers to increasingly work with you directly and programmatically – it should be treated as leverage in negotiating as it shows you’re concerned about ads only being served to humans. Just as forward-thinkers like USA TODAY, Gawker and the Washington Post are designing ad products that make viewability tools redundant, pro-active publishers will use this “fraudulent traffic crisis” to their advantage in reeling in more revenue.