Search engines by Philippe Barbe 18 Nov 2020

Part 2 of 20 in a series examining the interplay of Data Science, AI, the media and advertising.

Which search engine do you use? Google? Bing? Duckduckgo?

Regardless of your answer, it will at best be only partially true!

Yes, you may predominantly use Google, Bing, Duckduckgo for general searches, but if you want to buy something, you may go to Amazon or another etailer to search for your item.

If you are looking for a new job you will likely visit multiple job boards to filter through the listings for positions matching your skills.

Looking for a scientific research paper? Head to arxiv.org or medrxiv.org. An app? Search the App Store or Google Play. Verifying a fact or researching a company? Look for that on Wikipedia. Your company has a wiki… type a keyword and search for a project.

It is not a single player like Google but the single concept of the search engine and its multiple implementations and iterations that has transformed media and advertising in at least two fundamental ways.

The End of the Classifieds

The impact of the search engine on the newspaper industry has been massive.

From the beginning classified ads were the lifeblood of the newspaper business with editors and publishers focused first on selling the space, then categorizing and sorting them easy to find for job seekers, used car buyers, etc. by a single keyword, alphabetically ordered.

Search engines on the internet offer a much better user experience allowing multiple search terms and advanced filtering to narrow down to what you are looking for.

The advantage of this better ergonomics is compounded by the scale that is offered by an online platform. While newspapers can print a few hundred job listings, an online platform can store millions. This profoundly changed the economics of the newspaper industry.

Where a classified may have cost $30 to sell an item in the narrow audience of a local newspaper, a 10% commission for a listing with a global audience is available on a typical e-commerce platform. At the same time an expensive editing and printing process, was replaced with an inexpensive posting mechanism.

Consequently the bulk of the classified revenue stream evaporated seemingly overnight. From its peak of $19.6B in 2000, by 2012 the Newspaper Association of America reported classified advertising had fallen to $4.6B and is likely much smaller than that today.

This rapid revenue decline considerably accelerated a trend that started with cable television… the disappearance of hundreds of daily newspapers in the US.

Beyond the economic fallout, there are profound social and political ramifications that result from the reduction of public information with less funding for investigative journalism and reduction of newsroom staff at both at the local and national level, and the degradation of journalistic standards in the chase for clicks.

Disclosing the Search

The second major impact of search engine technology is that to search, you must disclose either generally or quite specifically what it is that you are searching for!

This has several far reaching consequences (even though some are not that new), but here are three big impacts on media and advertising.

On demand advertising. Because it captures what you are searching, the search engine can show you ads for what you are searching… instantaneously and on demand.

This has changed the advertising industry, and by ricochet the media industry, in a fundamental way.

Just like its impact on classified ads, the combination of better user experience combined with the platform effect allowing for scale made online display advertising an product extraordinarily attractive product. Soon a vast amount of corporate advertising revenue shifted from traditional media to new online platforms where the search for goods and services was taking place… Google, Facebook, Amazon, LinkedIn, specialized niche sites… anywhere people went to look for things they wanted to buy or learn about.

Data science played a key role in enabling on demand advertising. From natural language processing tools to the statistical inference of conditional probabilities, to AI for identifying objects in pictures, it is possible to display items related to the search, though they may not match the exact keywords.

Profiling users. Through cookies, IP addresses and other browser and internet technologies, a company’s search engine can compile what you are searching over time allowing them to construct a digital you… a profile of who you are, what you like, what you bought, what you didn’t, where you were when you bought it etc. With a flawless memory the digital you knows more about you than you remember about you.

Searching for size 1 diaper? You very likely have a one or two month old child, from which one can deduce a likely birth month, store it, and keep proposing you items as your child grows, from toys, to cloths, to student loans, to cars, to mortgages.

Mail order catalog companies began doing this type of profiling very effectively by leveraging the power of mainframe computers in the 1960s and 1970s. While current technology allows for a larger scale, both in terms of audience and product, and over a longer period yielding deeper knowledge of the consumers, it is not fundamentally new.

In constructing and using user profiles, Data Scientists are able to infer the customer journey, propose timely products, and products distinct than those searched for, yet popular among users of these products which is why if you search for a bread machine, you will get ads for flour and baking pans.

Digital native information. Because all the information collected and inferred lives in computers, it can be exchanged, sold, and merged with additional structured and unstructured information gathered from disparate sources. Although the scale and uncertainty of the information make merging it across databases difficult, it is easier than ever and becoming more so.

The digital nature of all this information makes it easy to use, to query, to exchange, to transact.

Data Science plays a key role in these aspects, and opened up the opportunity for new companies to appear that specialize in merging information for specific purposes or industries, from multichannel advertisement platforms, to data products delivered through API. We will discuss some specific aspects of this in further articles.

Conclusion

All in all, surveying the digital transformation that has taken place in the media and advertising ecosystems, it is the search engine and underlying massive hardware and software infrastructure that enabled Google, Amazon, and Facebook and more specialized companies to disrupt and reshape these industries.

Although a key driver, search engines are not the only cause for the revolution in media and advertising. Another very significant one will be explained in the next article: the law!