Googlebot and WRS : Things you need to know

Googlebot and WRS : Things you need to know

Google crew is working constantly to keep the documentation about the search engine updated and helpful for webmasters and SEOs. Google published today a new documentation about its web rendering service (WRS) explaining some interesting things.

WRS in a nutshell :

Let me first explain you what is a web rendering service : It is simply a web browser that is able to access, get and render your website (or web application) source code. So Googlebot is using a web rendering service in order to crawl & render websites and then qualify, index and suggest them as results for users. You are surely thinking now about “Fetch as Google” option in Search Console.. Yes that’s solicit the WRS. Web rendering goes through several steps from establishing connection, getting/ executing code and scripts, rendering layout, applying style to painting..etc (some useful informations here). Every step is critical and may modify the way the WRS (then Google) sees our website. You understood, it is a serious topic.

Things you need to know about Google WRS :

Non supported features by chrome 41 wrs

  • “Googlebot and WRS only speaks HTTP/1.x and FTP, with and without TLS” and i found this confusing so i asked Gary Ilyes about HTTP/2 detection and support

Gary gave this answer : Measuring HTTP/2 website performance is not the WRS job, it’s rather another service (see below the Twitter conversation)

Twitter conversation with Gary Ilyes

  • Google is disabling some capabilities and urges us to use the feature detection to ensure that ou content or feature will be ok for the customer (in our case, Googlebot WRS) : “Feature detection involves working out whether a browser supports a certain block of code, and running different code dependant on whether it does” Documentation here
  • Google also advise to use Polyfills ( A code to enable some not supported feature by an old browser, see the documentation here : https://remysharp.com/2010/10/08/what-is-a-polyfill)
  • Service workers are disabled
  • IndexedDB and WebSQL interfaces are also disabled
  • WebGL interface is also disabled so VR or 3D contents are not currently indexed
  • Googlebot WRS clears all session, cookies, local storage data so it is stateless through page loads
  • Special interfaces that require user consent are auto-declined by Googlebot such as geolocation, notifications, push, camera, microphone, accelerometer..etc
  • Google recommends two things to debug rendering issues:
  • And i recommend you to use the targeted Chrome version (M41) and its console

Final Notes :

It is so important to debug websites with the targeted Web Rendering Service in order to detect problems upstream and ensure a good Google Crawl and understanding. I will update this post with useful informations and tweets, so if you have feedback let me know 😉 Thank you for reading.

Leave a Comment