homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Do spiders crawl popups & drop down menus?

5+ Year Member

Msg#: 3700080 posted 6:27 pm on Jul 16, 2008 (gmt 0)

Hi guys :)

I was under the impression that spiders can't crawl JavaScript popups and drop down menus.

But when I recently looked in WMT, we have Duplicate Title Tag issues from popup and drop down URLs.

For example, on our site, you can choose how many results to view (10, 20, 30, etc.) via a drop-down menu. The URLs of these pages vary slightly but all have the same title tag. The only way of accessing the links is through the drop down menu.

Should I <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> these pages to get rid of the duplicate title tag issue?

Any insight is really appreciated.

Thanks a ton!


Receptional Andy

Msg#: 3700080 posted 6:32 pm on Jul 16, 2008 (gmt 0)

Generally speaking, spiders can extract URLs and similar from javascript, which means that links in drop-down menus and similar can be spidered and indexed.

In addition, Googlebot has somewhat 'experimental' crawling of HTML forms [webmasterworld.com].

So, realistically, you have to consider content accessible via forms (javascript-based or not) to be spiderable, and if you don't want such content indexed, robots exclusion is the way to go. You could do this via a robots.txt file (typically utilising the major search engines' support for wildcard entries) or a meta tag is fine. If you think there are links from the pages that are worth spidering, you might consider noindex,follow.


5+ Year Member

Msg#: 3700080 posted 1:22 pm on Jul 17, 2008 (gmt 0)

Perfect, thank you so much!

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved