Welcome to WebmasterWorld Guest from 54.196.244.186

Message Too Old, No Replies

Making Infinite Scroll Search Crawler Friendly

     
3:00 pm on Feb 14, 2014 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:22306
votes: 239


Your site’s news feed or pinboard might use infinite scroll—much to your users’ delight! When it comes to delighting Googlebot, however, that can be another story. With infinite scroll, crawlers cannot always emulate manual user behavior--like scrolling or clicking a button to load more items--so they don't always access all individual items in the feed or gallery. If crawlers can’t access your content, it’s unlikely to surface in search results.

To make sure that search engines can crawl individual items linked from an infinite scroll page, make sure that you or your content management system produces a paginated series (component pages) to go along with your infinite scroll. Making Infinite Scroll Search Crawler Friendly [googlewebmastercentral.blogspot.com]


You think infinite scroll is cool? Search-friendly infinite scroll is even cooler! Crawling the infinite scroll [googledevelopers.blogspot.com]
With some implementations of infinite scroll, crawlers aren’t great at scrolling down or clicking “load more”, so they may not crawl items displayed after the initial page load. To help the crawler see all the content, we recommend converting the infinite scroll page to paginated series by using the HTML5 History API. (Of course, the pagination is seamless to the user.)
11:09 am on Feb 18, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3080
votes: 67


In other words be careful when using the latest technology because Googlebot might have issues. Keep it as simple as possible to avoid Googlebot problems.
1:45 pm on Feb 18, 2014 (gmt 0)

Senior Member from LK 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 16, 2005
posts:2417
votes: 17


Now, what about making infinite scroll user friendly?
2:46 pm on Feb 18, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator goodroi is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 21, 2004
posts:3080
votes: 67


User friendly is a very big issue because there are so many different types of users. Mobile vs desktop users, high-speed vs low speed, older users with vision problems, users speaking different languages, older browsers vs. newer browsers, etc.

When implementing infinite scroll into a website, I would say it is more user friendly to ensure there is an alternative to infinite scrolling. Once there is an alternative for each major segment of your users, you can then figure out how best to address each segment.

If you have many users on slower connections you probably want to be more careful on how much additional information you load when they scroll down.

If you have older visitors with poor vision, then you probably want to display larger thumbnails if your infinite scroll is a photo gallery.

If you have a massive archive, then you might want to add shortcuts so a user does not have to scroll down for 10 minutes to see your April 2012 content.

Infinite scrolling can be a good thing for users if the webmaster thinks through the many different issues that it can possibly introduce.
2:54 pm on Feb 18, 2014 (gmt 0)

Junior Member

10+ Year Member

joined:May 8, 2005
posts: 158
votes: 0


So, my advice from a usability perspective is to **REALLY** think about how your users use your website before you implement infinite scroll.

I recently spent some time on a B2B site that thought it would be a good idea to implement infinite scroll in their search results. What they didn't consider was that their search results can be quite long and their users might want to spend DAYS scanning the results, in detail.

Infinite scroll, in conjunction with long results lists and extended browsing sessions is a recipe for an extremely frustrated and angry user base. If result sets are fairly static (e.g. sorted alphabetically), then the user might want to the ability to quickly jump back to a previous position. (say the 'W') section).

In my opinion, infinite scroll is cool for mobile devices, but, just because it works well with handheld computers, doesn't mean its appropriate for all audiences. Don't make the Windows 8 mistake!
2:35 am on Feb 19, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Jan 25, 2005
posts:14624
votes: 88


In other words be careful when using the latest technology because Googlebot might have issues. Keep it as simple as possible to avoid Googlebot problems.


That's one theory while Google is promoting Angular JS which is barely crawlable at all!

Some of their demos show a page full of empty <div> tags that get replaced with pages and content on the fly, much like infinite scroll, so if Google can promote such technologies you know they're figuring out how to index them as well.

Most importantly, experiment first before leaving you or a big customer out to dry. Find out what Google's limitations are before investing heavily in tech that the SE's can't crawl. Also, make sure they can ALL crawl your site because if Bing and others can't crawl the site and you get a penalty in Google then >POOF!< you are completely gone.

Also try investigating alternative methods of exposing the same data to search engines.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members