How does Google and SEO work
What does Google do?
Search engines like Google follow links. Google follows links from one web page to another web page. Google’s crawlers spider more than a billion pages every day.
A search engine like Google consists of:
- a crawler
- an index
- an algorithm
We’ll explore these terms in further detail below.
Crawlers, spiders or bots
A crawler follows the links on the web. A crawler is also called a robot, a bot, or a spider. It goes around the internet 24/7. Once it comes to a website, it saves the HTML version of a page in a gigantic database, called the index. This index is updated every time the crawler comes around your website and finds a new or revised version of it. Depending on how important Google deems your site and the amount of changes you make on your website, the crawler comes around more or less often.
How does Google find your site?
For Google to know of the existence of your website, there first has to be a link from another site – one Google already knows – to your site. Following that link will lead to the first crawler-session and the first save in the index. In the old days you could submit your website to a search engine. Today, that isn’t possible anymore. Search engines nowadays follow all links on the web.
Google’s secret algorithm
After indexing your website, Google can show your website in the search results. Google has a specific algorithm that decides which pages it will show in which order. How this algorithm works is a secret. Nobody knows exactly which factors decide how Google determines search results. Moreover, factors and their importance change very often. Testing and experimenting gives us at Yoast a good feel for the important factors and the changes in these factors
The value of links for search engines
It’s very important to have a basic understanding of how Google and most other search engines use links: they use the number of links pointing to a page to determine how important that page is. Both internal links (from the own website) as well as external links (from other websites) can help a website to rank high in Google.
Some links are more important than others: links from websites that have a lot of links themselves are generally more important than links from small websites with few external links
Universal search
In addition to the organic and the paid results, Google also embeds news items, pictures and videos in its search results. This embedment is called universal search
Crawlability
Depending on how you maintain your website, it can be easy or difficult for Google to crawl your website. If you have good crawlability, Google will be able to index your site without problems. There are a few ways in which a crawler can be blocked from your website. If the website or a page on your website is blocked, you’re telling Google’s crawler: “do not come here, this area is forbidden”. You won’t turn up in the search results in most of these cases
There are several ways you can prevent Google from accessing certain pages. We’ll explore them in the Technical SEO module of this course.
Google’s mission
Google’s mission is “to organize the world’s information and make it universally accessible and useful”. In other words: to build the perfect search engine that helps people find what they are looking for. Google always wants to show you the best result for your search query. Google has updated its algorithm numerous times over the years, but their goal remains the same: Google tries to get you the best result.
Google gives you the best results by ranking the most relevant and usable websites, and combating spam. Sites that are only built to make money, or otherwise created purely to rank to make money, should not be on top of the search results. Sites that give the user what he or she searched for should always be on top. Google also rewards sites that provide good user experience (including for instance sites that load fast).
RankBrain
RankBrain is a Google algorithm, but a very advanced one at that. It is a machine learning system that helps Google better decipher the meaning behind the terms people search for. It serves the best-matching search results related to those queries. When RankBrain was first announced, Google called it the third most important ranking factor. Presumably, RankBrain can somehow summarize what a page is about, evaluate the relevance of search results, and teach itself to get even better at it with time. The common understanding is that RankBrain, in part, relies on the traditional SEO factors (links, content, keywords, etc.), but also looks at other factors that are specific to the search term. Then, it identifies the most relevant pages in the index and arranges the respective results in SERPs (search engine result pages).
Important Google updates
Throughout the years, Google introduced several major updates. We’ll discuss the most important ones of the last seven years and the implications of these updates.
Panda (2011)
In 2011 Google released its first Panda update This Panda update tried to diminish those websites which were purely created to rank in the search engines. Panda mostly focused on on-page factors. In other words, it determined whether the site offered information about the search term visitors used. Two types of sites were hit especially hard by the Panda update:
- affiliate sites (sites which mainly exist to link to other pages)
- sites with very thin content
Google has periodically re-run the Panda algorithm since its first release.
Penguin (2012)<h/3>
A year later, Google rolled out the first Penguin update. Penguin particularly looked at the links websites got from other sites. It judged whether the sites linking to your website like and admire your products or content. If the links were artificial, Google no longer assigned link value. In the past, lots of people tried to boost their ranking by buying links. Penguin tried to diminish the effect of buying, exchanging or otherwise artificially creating links. Websites with a lot of these artificial links got hit hard by this update. They lost their place in Google's ranking. This update has also run several times since its first inception and is now even said to be run continuously.
Hummingbird (2013)
In August 2013, Google released Hummingbird. In this update, Google laid down the groundwork for voice-search. Hummingbird pays more attention to each individual word in a query, ensuring that the whole search phrase is taken into account, rather than just particular words. This should lead to results matching the entire query better. The results were not immediately clear, but over time, Google started showing more answer boxes in the search results (see Image 1), that gave the answer directly instead of enticing people to click on to a web page. Voice-search has become more and more important as more devices (Google Home, Alexa) have started using voice search.
Mobilegeddon (2015)
In 2015, Google introduced the mobile update, dubbed “Mobilegeddon” by the industry. As more and more people use Google on mobile devices, Google used this update to boost sites that have mobile friendly pages in its mobile search results. Around the same time, Google showed that mobile devices accounted for over 50% of all search queries.
Possum (2016)
In September 2016, the Possum update applied several recent changes to Google's local ranking filter. After Possum, Google has shown more varied results depending on the physical location of the searcher (the closer you are to a business physically, the more likely it is that you'll see it among local results) and the phrasing of the query (even very similar variations now produce different results).
Mobile indexing first algorithm (2018)
Last but not least, Google is currently rolling out their mobile indexing first algorithm. This means Google will create and rank its search listings based on the mobile version of a site, even for listings that are shown to desktop users. As more and more searches happen on mobile, Google wants its index and results to represent the majority of their users, who are mobile searchers. This means that you need to make sure the content and links on the mobile site are similar enough to the desktop version so that Google can consume the proper content and rank your site as well as it did by crawling your desktop site.
Google’s search results page
Almost everybody knows what the Google Search Engine Results Page (SERP) looks like. We’ve all been there. We encounter that page with every search we do. However, it’s not always clear what elements search results consist of exactly. Which of those results are paid for and which are not? And did you know you actively need to provide Google with extra information to create some of them? We’ll take you through the Google SERP and make sure you’re up to speed on the difference between paid results and organic results, snippets, featured results, Knowledge Graphs, and answer boxes. We’ll start with how the page looks as a whole and then zoom into the different types of search results you can encounter.
Different looks of the SERP
The default page of Google’s SERP is a page on which different results appear. Google decides which results fit your search query best. That could be “normal” results – a blue title with a green link and a black description below – but also news results, shopping results, images, or a Knowledge Graph.
What the SERP looks like largely depends on what you’re searching for. If you’re searching for a product you can buy, Google will show shop results on the SERP. For example, if you’re searching for an electric guitar for kids, Google shows a page that starts with shopping results, as shown in Image 2. For a site to show up there, it’ll have to pay Google – note the word “sponsored” in the upper right corner.
However, if you’re searching for information about the planet Mars, you’ll encounter a totally different looking SERP. In Image 3 you can see that when you’re searching for “Mars”, you’ll get a SERP with news articles and a Knowledge Graph (the block on the right) with lots of information about the planet Mars.
If you want to, you can apply some filters to the search results yourself. Below the search bar in which you enter your queries is a menu which gives you the option of filtering results. You can filter images, videos, maps, shopping, books, flights, and finance (see Image 4)
However, the “All” option is far and away the most important one, so in this lesson, we’ll focus on that. Let’s take a closer look at the different elements a search results page can contain.
Sponsored results and ads
Google shows both paid and non-paid results. We call the latter organic results. It can be pretty hard to notice the difference between the two. The first couple of results are often taken by advertisements. Companies pay Google to show up as one of the first results for a certain search term. Sometimes it’s only one ad, but Google can show more ads as well. This depends on how many people search for a search term and who wants to pay for it. The cost of advertising is also related to the popularity of the search term.
You’ll recognize a paid result by a little green box with the word “Ad” shown on the left of the link to the website. The shopping results in Google, as shown in Image 2, are also paid results. If you want to advertise on Google, you should check out Google AdWords.
Organic results
The organic results are all of the results that are not paid for. The organic results that are shown first are the results that fit the search query of the user best, according to Google’s algorithm. Search Engine Optimization (SEO) aims to improve the chances to rank in the organic search results.
Research shows that as much as a third of the total traffic comes from the number one result, and 17% from the second result. That means that about half of the searchers click one of the first two results. Moreover, 90% of the searchers doesn’t get beyond the first page of the search results. The numbers steadily decline for each result. So being at the top of the search results is hugely important.
Snippets
Now let’s zoom into the individual search results. We call every separate search result a snippet. So, the organic result shown in Image 6 is a snippet. A standard snippet consists of three elements
- a title (in blue)
- a URL or slug (in green)
- a meta description (in black)
In the meta description, you should give a clear description of what your page is about. Your meta description should entice users to click the result, so it’s hugely important.
Sometimes, there’s more to these snippets than just the three elements we discussed above. A snippet could show extra information between the URL and the description.
We call these snippets with extra information “rich snippets”. A rich snippet of a pizza recipe could contain a picture of the pizza, the rating and the number of votes, the preparation time and the number of calories (see Image 7).
Featured results
Sometimes, a result is set apart from the regular search results, usually at the top of the page. This is called a featured result.
A featured snippet is a highlighted search box that answers the question you type into the Google search bar. Featured snippets often appear as a paragraph or a bulleted list, accompanied by an image. Since this featured snippet box is situated above the regular organic search results, everybody is bound to notice this. So, you can imagine the effect that might have. Having your content as a featured snippet not only brings in a lot of traffic, but it also proves your authority on the subject – Google picked you, right?
Knowledge Graph
Google’s Knowledge Graph box is that big block of information that appears on the right-hand side of your desktop screen after entering a search term.
This block contains relevant, context-specific information regarding your search. According to Google, this information is retrieved from many sources, including the CIA World Factbook and Wikipedia
Answer boxes
An answer box appears somewhere between the organic search results. It’ll give suggestions for questions that relate to the search query you typed in.
In this Article we’ve explained how Google works. We’ve seen that SEO is the practice of optimizing websites to make them reach a high position in Google’s – or another search engine’s – search results. Moreover, we’ve discussed all of the different types of search results that could appear on Google’s SERP.
