homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / WebmasterWorld / New To Web Development
Forum Library, Charter, Moderators: brotherhood of lan & mack

New To Web Development Forum

    
Search Engines and Databases
Can spiders find my forum pages?
paul_sully




msg:965654
 12:20 pm on Mar 12, 2003 (gmt 0)

Hi,
Thanks for the info,

I have a php forum on my website (phpbb2), can google search page contents even though they are in a forum database?

Paul

 

Giacomo




msg:965655
 1:15 pm on Mar 12, 2003 (gmt 0)

I have a php forum on my website (phpbb2), can google search page contents even though they are in a forum database?

Googlebot does follow dynamic URLs (i.e., URLs with a querystring such as ?forumid=1&threadid=2 in them). However, there are a few limitations: first of all, Googlebot limits the number of dynamic pages it will crawl; secondly, as repeatedly stated by GoogleGuy, crawling URLs with a PHP session ID in them can be a real pain in the a$$ for Googlebot, so you'd better strip those off your forum URLs if the UserAgent is Googlebot/2.1 (+http://www.googlebot.com/bot.html).

Since this has all been discussed before, I suggest you do a site search of google dynamic pages or google dynamic urls or google querystring on WebmasterWorld. Alternatively, you may use Google to search this site specifying site:webmasterworld.com along with your keywords.

Then, once again, make sure you read and understand Google's Information for Webmasters [google.com]: the answers to most recurring questions are right there, in the guidelines [google.com] or in the FAQs [google.com].

About your specific question, yes, there is a way to do that (i.e., make your forum URLs static-looking and strip off the PHP session ID if UserAgent == Googlebot) with phpBB. However, to make phpBB spider-friendly is not a simple task and requires hand-modifying the phpBB code following the instructions in a couple of phpBB "hacks". Unfortunately, AFAIK those hacks have not been packed together and released as a single MOD ("official" phpBB modification) yet, so you will have to go through several posts and "merge" the instructions you will find in there.

A few starting points from the phpBB support forum:

[Tutorial] Google & phpBB [phpbb.com]
Search Engine Friendly and Customer Header / Footer [phpbb.com]
[Reason] How Google spiders PHPBB! [phpbb.com]
Unusual Request [phpbb.com]

Following the instructions from the above posts I have been able to obtain a modified phpBB which is totally spider-friendly. So it is not impossible to do. It only takes some patience.

Good Luck! ;)

[edited by: WebGuerrilla at 7:50 pm (utc) on Mar. 12, 2003]
[edit reason] forum move [/edit]

paul_sully




msg:965656
 3:10 pm on Mar 12, 2003 (gmt 0)

Is it right that googlebot can follow src links? like framesets?

Can it follow javascript menu's that dont use href?

PAul

Giacomo




msg:965657
 3:18 pm on Mar 12, 2003 (gmt 0)

Paul,

Before posting a new question, please take some time to scan through previous posts: your question might have been already answered.

Also, please start a new thread whenever posting a new, unrelated question.

> Is it right that googlebot can follow src links? like framesets?

Yes it is.

> Can it follow javascript menu's that dont use href?

Currently, no.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / New To Web Development
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved