Machine Learning and Semantic Search What is Google's Ultimate Weapon?

The past few years have been exciting (or scary, if you’re not so open to change) for search engine marketers. Google has introduced a host of new technologies and systems to improve its already excellent search algorithm, and competitors like Bing, Apple, Microsoft, and even Facebook have introduced or upgraded corresponding systems to improve their users’ search experience.

The core goal of these optimizations is to solve one of the most difficult problems in the history of computers and artificial intelligence: the acquisition and understanding of natural language. Although machines are very good at procedural and logical tasks (such as solving mathematical equations or identifying features of object data), they have difficulty distinguishing the subtle differences that only native speakers can intuitively feel. For example, to us, the sentences "Where is the nearest burger shop" and "I want to eat a burger" convey similar intentions, but to basic machines, the two are very different: the first sentence is a retrieval of specific information, regardless of the speaker's thoughts; the second sentence conveys a clear desire, regardless of information retrieval. In the past few years, Google has been committed to breaking down the barriers between different languages ​​and solving the problems faced by its machine algorithms. But what is the ultimate goal of this effort: Does Google want to fully understand the intentions and subtle semantics conveyed by human thoughts? Or is this just an exploration in the development of its technology?

The roots of semantic search and the Hummingbird algorithm

In 2013, with the introduction of the Hummingbird update, the concept of semantic search first came into people's view. Previously, Google retrieved the corresponding web pages by determining the keywords of the search query. In the example of the hamburger restaurant mentioned above, Google will get the keyword "hamburger" and return the web pages that frequently mention the keyword - but this method cannot really capture the user's intention. It cannot distinguish whether the user wants to search for "hamburger restaurant" or "how to make a homemade hamburger." Although it still needs to be improved, the Hummingbird algorithm based on user intent analysis can capture the key factors of user intent and "understand" the content of the page, rather than sticking to the keywords contained in the page.

Personal digital assistants and voice search

Personal digital assistants, including Google Now and Siri, require a new level of semantic understanding. New obstacles have emerged, including translating spoken sentences into typed text, finding the right search terms, and responding in an understandable way. Although search results are still essentially Google's standard search algorithms, continuous improvements in artificial intelligence algorithms have advanced these technologies to the level we see today. The primary motivation is to make search easier to use, but the second motivation is to change user habits: voice search requires conversational input and contextual prompts, which requires more powerful and relevant results than traditional keyword-based input.

The emergence of RankBrain

The biggest news in the semantic space last year was the emergence of RankBrain, a machine learning algorithm combined with Hummingbird. I’ll briefly explain it by outlining its purpose:

Its role is to help Google understand the various lengthy, complex or ambiguous verbal query statements entered by users. It can be understood in this way: it translates those bad input statements that do not conform to grammatical rules into a form that can be interpreted by algorithmic logic. And because it uses machine learning, it can update itself, thus no longer relying on manual adjustments and upgrades by developers.

The rise of “related questions”

In order to make the user's search experience more fulfilling, Google is gradually promoting "rich answers" (concise versions of "answers" that sometimes appear above traditional search results). Recently, Google launched "related questions", which encourage users to further explore their search topics. What is particularly interesting is that the answers to "related questions" are currently different from their corresponding answers as rich answers, which means that the two are based on different parts of Google's search algorithm (for example, we can see Google Knowledge Graph and RankBrain). In any case, it seems that Google is committed to not only understanding user queries, but also answering them concisely and clearly.

Three predictions

To deduce what these observations mean, I want to offer three predictions about how Google might choose to proceed over the next decade:

1. Machine learning will become a new target. Currently, machine learning only exists in the more optimized part of Google's search algorithm. I guess its application will extend to other areas, including content quality analysis and backlink context evaluation.

2. Google will work on turning every search query into an answerable question. With the accumulation of rich answers and related questions, it is clear that Google wants to provide users with the most direct information. I think we will see more efforts in this direction, including turning basic query statements into complex, answerable questions.

3. New technologies will drive more people to voice search, which naturally means more complex questions and more direct answers to these questions. The more people use voice search, the more data Google has, and the better search results it has for users - in general, we get a more sophisticated search system. Therefore, I guess Google will push more people to use voice search in the next few years.

Apply these three predictions however you see fit: you could start adopting new content marketing strategies to respond to user questions; or use new techniques for SEO to avoid the frequent changes brought about by machine learning; or, you can just wait for an easier and more intuitive way to search for yourself. We can't say for sure about the future, but looking back at the great progress we have made in the past few years, we can foresee that the next decade will be a very groundbreaking decade.

As a winner of Toutiao's Qingyun Plan and Baijiahao's Bai+ Plan, the 2019 Baidu Digital Author of the Year, the Baijiahao's Most Popular Author in the Technology Field, the 2019 Sogou Technology and Culture Author, and the 2021 Baijiahao Quarterly Influential Creator, he has won many awards, including the 2013 Sohu Best Industry Media Person, the 2015 China New Media Entrepreneurship Competition Beijing Third Place, the 2015 Guangmang Experience Award, the 2015 China New Media Entrepreneurship Competition Finals Third Place, and the 2018 Baidu Dynamic Annual Powerful Celebrity.