A summary of Google recommendations for Web developers.
In the U.S. and many other regions, Google is the dominant search provider. Moz claim that in 2018, 62.6% of searches in the U.S. are through Google, a further 22.6% are Google Images, and 4.3% are on YouTube. These Google properties are the top three search providers in the U.S. region.
In addition, voice search is a growing trend. Google are a leading provider of voice search services through Android phones, and other devices. The answers and results that Google voice search systems provide are based on the main search engine.
Today, about half of searches now result in no further clicks. This is partly because of the information that Google now shows directly on the results page. In these cases, Google is the last page that the user visits, but most of the content has been retrieved from third-party Websites by Google systems.
Statistics are from the presentation the SEO trends for 2019 you need to know about, by Sarah Bird, CEO of Moz.
Google prioritise sites by many factors, but pages must be fast, mobile-friendly and provide relevant information.
How Google Indexing Works
The system is collectively known as Googlebot, but has several components:
- The scheduling sends URLs to crawlers
- Crawlers follow URLs and get content
- Renderers process the content, running it as a Web browser does
- Indexers extract information from the rendered content
Googlebot performs a crawl and an initial index, then renders the pages later (perhaps days later).
- Discoverability: Help Google find the content
- Evaluable: Help Google understand the content
- Speed is a ranking factor
Google research indicates that users consider speed to be the most important factor in how they feel about a site. The break-point is five seconds: In 2018, DoubleClick analytics show a 53% lower bounce rate for mobile sites that have load times of less than five seconds.
The process of optimization should start with the URL.
- Single URL per piece of content: for efficiency, Google systems try to discard duplicate URLs
- Use traditional URLs (don’t point to fragments)
- The element of “link” with type “canonical” in HEAD sections in treated as a strong hint about preferred URL, but not necessarily obeyed
- If necessary, use Search Console to specify the part of URLs that should be excluded when checking for uniqueness
Providing the Content of a Search Listing
Google use the URL and these other items to populate the listing for your in search results:
- Page title: Use a “title” element in the HEAD section
- Description: Use a element of “meta” with name set as “description”
- JSON-LD structured data embedded in the page
Embedding JSON-LD data in the source of a page enables Google to extract meaning from the page and show rich results.
- Individual sitemaps are limited to 50,000 URLs
- Use a sitemap index file to submit a list of multiple sitemaps
- Consider that not every page on your site should be indexed
Google crawlers honor robots.txt files. The expected format for these files is described in the Robots.txt Specifications.
Page Design and Rendering for Search
The search renderers use the latest stable release of Chromium. The version of Chromium is updated every six weeks, when a new stable version of Google Chrome is released. By default, the renderers emulate Chromium on mobile.
For images, either use standard img tags in your markup, or list the images in JSON-LD markup for the page. Googlebot will not see lazy loaded images. References to images in CSS are not followed.
Your code can spot “Googlebot” by looking for that string in the User Agent: it may be claiming to be either desktop or mobile.
Ensure that all of the assets that are needed to render a page are accessible to the crawlers. If assets are slow to load, then they can timeout when Googlebot fetches the page. This will result in some of the page not being indexed. Avoid timed interstitials, because these are effectively timeouts.
Google Search Console
The Google Search Console is your main interface for understanding how Google interacts with your Website.
- The Search Console includes a Fetch as Google feature, so that you see what Googlebot receives from your site, before rendering
- Google provide search data with the Search Console API
- Google now also provide an API for fast link submission
The Chrome User Experience Report
The Chrome User Experience Report (CrUX) measures user experiences on popular Websites.
The CrUX dataset is updated monthly. Aggregate data for origins (Websites) is published on Google BigQuery, so that you can query that data yourself with Google Data Studio or other tools. PageSpeed Insights uses URL-level data.
Website Testing Tools
- Google “mobile-friendly” test can also show the source code of a page, as rendered by Googlebot.
- Google “rich results” test shows how Googlebot sees your page when rendering as a desktop browser
The details of the page ranking systems are secret. To help content creators understand the criteria that are applied, the Google Search Quality Guidelines are published. This document is the handbook for Google staff who check the results of the search engine.
The Webmaster Guidelines are a brief summary of good practice.
- Build a successful web presence with Google Search (Google I/O ‘18)
- Use Lighthouse and Chrome UX Report to optimize web app performance (Google I/O ‘18)
- SEO Snippets series
- The Beginner’s Guide to SEO, from Moz.
- SEO Capability Maturity Model, from Moz.
- the SEO trends for 2019 you need to know about
Web Page Performance
See the separate article for recommendations for Web page performance.