How can a crawler search inside a database?

search in a database web site.

[erdsah88]

web a search site called <domain removed> on google with site:

it says it has found 1.7 million mathes but infact there is only two pages and the rest is a database driven (turkish-english)dictionary application.

Does anyone know how do we make the crawler understand that it is a database driven site?

Google Specifics

<Sorry, no personal urls.
See Forum Charter [webmasterworld.com]>

[edited by: tedster at 12:02 am (utc) on Sep. 19, 2006]

[soulful house]

A crawler can't look directly in a database.

You need to implement a sitemap structure, and you need to link all sort of pages resulting from relevant database queries. Do it right or it might be hard to change it afterwards....... URL rewriting (canonicalization) might be very important, do it from the very start!

[selomelo]

Since it is a dictionary, you may need some kind of "pagination class" to create browseable (and crawlable) links for the entries. Especially, if very long SIDs are generated, perhaps you would also need to remove such SIDs, and implement a rewrite to help search engines crawl your site without problem, as said above.

If you can sticky me your URL, I would have a look. Although I am not a coder, I too have a dictionary site.

BTW, welcome to webmasterworld, and greetings from Turkey.