GUEST ESSAY
Can’t help bot wonder — is this the end of the Google Search monopoly?

(Image: Leonardo.ai, prompted by author)

By Steven Boykey Sidley
Follow
26 Feb 2024 

Here are the advantages of AI chatbot output versus that of a good old ranked search.

A change in my behaviour has crept up on me, pretty much unnoticed. I realised recently that I almost never use Google Search anymore, despite doing a substantial amount of exploring on the internet every day as part of my job. What happened?


Google Search, the company’s original product (well before Gmail and Google Drive and Google Maps and their other products), has been not only one of the most successful products in history, but has also contributed in a major way to the internet, taking it from a moderately useful tool into public ubiquity.


There must be more to blockchains than just Bitcoin.
There is. And it's coming to a future near you soon.

It's Mine is an entertaining and accessible look at how Bitcoin made its mark, how it all works and how it challenges our long-held beliefs, from renowned expert and frequent Daily Maverick contributor Steven Boykey Sidley.Buy Now

The story of its genesis is now internet lore.

During the late Nineties, two young Stanford students, Serge Brin and Larry Page, frustrated at the creaky torpidity of current search engines, came up with a better idea. Instead of crawling around the global web for an answer to a user question, they decided to download the entire web every night onto a bunch of hard drives and then index everything so that it could be searched blazingly fast, unlike the prevailing technique of visiting and parsing the content of every website server on the planet.

It was not a problem to do this; the web was still small and manageable. When it got larger, they simply added more storage and more CPUs. Which is why they now own massive data centres on every continent, which continually update and index their store of information on an ongoing basis.

Their second innovation was to rank their search results according to how many other people had accessed the same website. Rather like a popularity rating, with the more popular websites placed at the top of the page. It was called the PageRank algorithm and, while it has been fine-tuned many times over the past few decades, it spawned an entirely new industry dedicated to rank-bumping. The difference between being on the first page or the second page (or beyond) has meant life or death to many commercial enterprises.

How successful is Google Search? It owns 90% of the market. It generated $175-billion for Google in 2023, which was 57% of its total revenue and 73% of its advertising revenue.

There have been some competitors along the way, like DuckDuckGo, which keeps your searches private and does not resell your data, and which has a small market share. There is Microsoft’s Bing, a perennial also-ran in the mighty Microsoft stable, and then there were a couple of so-called Google-killer startups like Neeva, which died quickly and quietly.

So why am I not using Google Search anymore?

Because of AI. When I need information I use ChatGPT, Gemini, Perplexity, Claude, Groq, Jasper or Pi. There are more, speciality AI search engines which I will get to one day, such as BloombergGPT for financial data.

So, what is the advantage of AI chatbot output versus that of a good old ranked search?

Possibly the most important advantage is that chatbots are trained on a dramatically larger corpus of information than is available to traditional search engines.

The first is the language of query. In traditional search engines we have learnt to query using keywords, usually no more than a few in a short sentence. In the chatbots we can simply frame our query in our vernacular (originally the chatbots were English-speaking, but they are a fast growing Tower of Babel – speaking French, Arabic, Chinese, Spanish and others).

This makes a startling difference to user experience. The chatbot queries can be massively detailed (up to one million words in the latest version of Gemini), which means that searches can be either very general or insanely focused or anywhere in between. The user is not presented with a ranked list of websites to choose from but receives a much purer response – text, images and audio of any user-defined length, untarnished by ranking tricks and untainted by advertising. These responses can then be refined further by additional queries and rephrasings, just as in real life.

The second advantage is speed. Anyone who has tried Groq (not to be confused with Elon Musk’s Grok) will have witnessed the astonishing appearance of the answer almost before their fingers have left the keyboard.

The third and possibly most important advantage is that chatbots are trained on a dramatically larger corpus of information than is available to traditional search engines. It is estimated that by 2027 or thereabouts, the chatbot training data will come close to having consumed all of recorded human information (yes, I know, copyright issues might intrude a tad here; the jury is quite literally still out on this matter). In any event, this means much richer responses to questions than the traditional search engines can offer because only websites and some other public repositories are accessible to them.


It is not unreasonable to conclude that Google’s 26-year-old search monopoly will be under threat.

There are a couple of well-publicised downsides too. The occasional and gleefully reported “hallucinations”, the inability to reference the source of the information and the fact that information is not necessarily current (due to the time lag between training and public accessibility). But all of these issues are in the process of being addressed – the incentives for solving them are immeasurable. For instance, Perplexity claims to have solved both the source-referencing problem and the presentation-of-real-time-information problem by mixing chatbot technology with older search techniques.

Read more in Daily Maverick: Stop Googling and Bing it, already

The sharp-eyed reader will know that Gemini is being integrated into Google Search, so Google’s competitive moat may well be protected. But the Microsoft-owned Bing has integrated the much better known ChatGPT, and so the search field is already being reshaped. Add to this the other new Chatbot aspirants I mentioned and it is not unreasonable to conclude that Google’s 26-year-old search monopoly will be under threat.

Is this a good thing? Indeed. The crumbling of monopolies is a happy noise. However, I will remember Google Search fondly. After all, it made that hallowed hall of fame wherein products become verbs, right?

Just Google it.

 DM

Steven Boykey Sidley is a professor of practice at JBS, University of Johannesburg. His new book It’s Mine: How the Crypto Industry is Redefining Ownership is published by Maverick451 in South Africa and Legend Times Group in the UK/EU, available