Welcome to WebmasterWorld Guest from 54.209.227.199

Forum Moderators: phranque

Streaming content to a page

How does googlebot like that

     
7:11 pm on May 17, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2548
votes: 719


I have begun using asynchronous web frameworks. In my case I'm using Python Quart which is an async wrapper for Flask.

Typically the web server receives a request from a client. It processes the request, waits for data from the DB, generates the entire page, and then sends all the data back to the client for rendering by the browser. As such the client needs to wait for the entire page to be processed before the client can get the first byte.

By streaming the page's content, the above the fold content can be prioritized, generated, sent to the client while the below the fold content can be left to wait for i/o blocking assets such a data from the DB. Then as this data becomes available it can be sent to the client. The benefit of the streamed content is that the client can get the first byte as well as the above the fold content in milliseconds.

The upside is that the user sees the content much sooner but the down side is that browser can end up with a really long page load time.

The question is how does Googlebot handle the stream? Will Googlebot wait around for a long time? What impact will it have on Google's page speed measure?

The other advantage for my use case is that I can reduce the number of request to the server, because with streaming I will be able to generate images and send them with page directly as opposed to sending a page with an image url, that will then result in another request to the server, and then to the DB etc. This will reduce the load on the server and likely increase speed.

Has anybody tried this?
9:07 pm on May 20, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2548
votes: 719


I ran a small very informal test. I created a very basic page that prints a string roughly every 5 seconds 8 times. I ran the page on localhost and tested it using Google Chrome Lighthouse (audit tab in dev tools).

The outcome: First Contentful Paint, Time to Interactive, First Meaningful Paint and First CPU Idle all score a 0.6ms. This is fast and will likely be slower in the real world. The only metric that is slow is Speed Index which is about 17 seconds, which shows up in red, this ultimately hurts the performance ranking at 73%. The total request takes 40 seconds (5s x 8).

So which of these metrics is Google using as ranking factor? Is there a benefit to delivering ATF content faster and the cost of slowing down the completion of the page?
10:06 pm on May 20, 2019 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:9714
votes: 925


What will the user see? In the real world that is all that counts.
1:26 am on May 21, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2548
votes: 719


What will the user see? In the real world that is all that counts.

Without ranking in Google there will be no real world users that will ever see the website.

That said, by streaming the content and prioritizing ATF content the user will see content much faster.
2:07 pm on May 21, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2548
votes: 719


I take it from the lack of interest in this thread that there are not a lot of people (if any) doing this.

I should mention that traditionally the issue with this approach was that having a long response time eg: 20 or 40 secs would block the thread for that time. Thus for any reasonable amount of traffic, one would need to have a very powerful machine able to support many threads concurrently. But now with the asynchronous frameworks, threads are not blocked while waiting for I/O, so a single thread can handle many requests nearly concurrently.
12:55 pm on May 22, 2019 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 15, 2001
posts:1798
votes: 53


HI NickMNS I am not doing it and would not know how to. However surely doing it this way the page will only completely have downloaded when the slowest elements has loaded, the same as by traditional page load means?
2:15 pm on May 22, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2548
votes: 719


However surely doing it this way the page will only completely have downloaded when the slowest elements has loaded, the same as by traditional page load means?

Yes and no, it depends.

Yes:
the page will only completely have downloaded when the slowest elements has loaded

But traditionally one needs to wait for the request's response before beginning to render the content, so the user sees nothing until the long lead element has completed. In the streaming model, one sends partial responses with the elements that have already completed. So the user sees some of the content while waiting.

No:
The async frameworks do not block the thread while waiting for I/O. So assume that the processing of your long lead element get started first, say with a long look-up to the DB. In the traditional model this would block the remaining elements from being processed, so one waits for the long item, then completes the quicker items and the sends the response. Now in the async model the long lookup to the db will not block the thread so the quicker elements can processed while waiting. The net result, that the entire response will be delivered faster.


It depends:
In practice the benefits will depend on the content:
- if one has content where there is little I/O then the benefit will be less
- if the processing is CPU intensive, then it will block the thread and there will be no benefit.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members