Google’s Martin Splitt explains Google’s indexing process from discovery to ranking and where rendering fits into that queue
How a website renders is a part of the indexing queue and like the other parts has an impact on SEO and ranking. Google’s Martin Splitt explained in a webinar hosted by Duda how rendering impacts SEO and what the crawl process looks like from initial discover to ranking.
The webinar began with Bartosz Goralewicz relating his observations that rendering and layout started surfacing as a trend at Google around 2011.
His personal theory is that it played a role in quality updates.
Perhaps more so today than in the past, rendering and page layout play an important role because of the increased interest in Core Web Vitals.
It could be said that much of the search community is more aware of things like the technical aspects of layout and rendering than in the past.
Rendering influences how fast a web page loads and consequently impacts sales and in some cases rankings.
Does Rendering SEO Help Rankings?
Bartosz started out by asking Martin Splitt a provocative question:
“Can rendering SEO help me rank better?”
Martin paused a moment to think then answered:
“I usually don’t answer ranking questions. I’ll make an exception here.
Generally speaking, no.
But specifically speaking, if there is a problem where something breaks your render and the content doesn’t show up, then if the content doesn’t show up and Google doesn’t see the content properly, then, that might actually hurt you in the sense of, we don’t see the content.
So we might not index the page or we might index the page, but not ranking for the content that you care about.
So yes, in the end it can make a difference and it can have an impact. Yeah, of course.”
Where Rendering Fits From Crawling to Indexing
Bartosz followed up with a question about where rendering fits into the crawling and indexing process.
He asked:
“Where does rendering sit in the whole scenario? …My understanding of that was always that Google creates a queue then crawls, renders and obviously optional, indexes the page. Would that be an oversimplification?”
Martin answered:
“That is slightly oversimplifying, but it’s fundamentally true.”
Bandwidth Affects Website Crawling
This is something that maybe doesn’t get discussed enough and that’s the ability of the server to handle not just Google but all the other bots and site visitors hitting the website.
Google will adjust the crawling if it seems like the server can’t handle the crawl.
This is called, Crawl Capacity Limit. Read more: Why GoogleBot Doesn’t Crawl Enough Pages on Some Sites
Martin Splitt discussed the issue of bandwidth and how it affects crawling.
Martin continued:
“We get lots of your URLs and we get so many URLs that we can’t crawl them all at the same time for the obvious reasons.
We can’t crawl all the URLs all the time at the same time for reasons of bandwidth. So there’s only so much internet bandwidth that you can use.
If you have an online shop and you come online tomorrow with the new online shop website and you have a million product URLs, your server might crash if we crawl all these million URLs at the same time.
So we have to spread this out through time, so there’s a queue in between us discovering a URL and the URL actually being crawled.”
From Crawling to Rendering
Martin next explains the next steps on the road to indexing content.
Martin:
“What happens then is once we have crawled, we can already look into the HTML …we can see …into the HTTP status.
If it’s a 404 HTTP status, then pretty much the processing ends here. If there’s a robots meta tag that says not to index, then our work ends here as well.
But if we get a bunch of HTML content and we can go forward with processing that in the rest of the pipeline, we also then queue the page for JavaScript execution, which is what you would call it “rendering.”
The second queue is very opaque in the sense that you don’t really see how long it takes us to render.
So like in the intake, there’s the URL that we discover and the output of that is either an indexed document or a non indexed document.
That’s pretty much what can happen here.”
What Indexing Queue Looks Like
Martin next provided a simplified outline of what crawling and indexing looks like.
Martin:
“So there is an additional queue that you skipped over, and there are a few more complications where the simplified model might not necessarily apply, but you can assume that the flow normally is discovering, crawling, queuing, rendering, indexing, and then potentially ranking later.”
From Discovery to Ranking
Martin Splitt provided a useful overview of how Google goes from discovering a web page to ranking that page (potentially). Martin seems to know quite a bit about how Google works internally so it’s always enlightening when he shares some of what he knows.
