Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Cannot get into search engine listings

Trying for almost 6 months



10:19 am on Nov 7, 2001 (gmt 0)

10+ Year Member

Hi Everyone,

I'm quite new here,
can anyone tell me why I got log entries like this for the majority of search engines that tried to index my site:

2001-11-07 09:16:48 - W3SVC141 W1 80 GET /robots.txt - 200 0 299 183 94 HTTP/1.0 www.rooflift.com Mozilla/3.0+(Slurp/si;+slurp@inktomi.com;+http://www.inktomi.com/slurp.html) - -
2001-11-07 09:16:48 - W3SVC141 W1 80 GET /Default.asp - 200 0 0 173 47 HTTP/1.0 www.rooflift.com Mozilla/3.0+(Slurp/si;+slurp@inktomi.com;+http://www.inktomi.com/slurp.html) - -
They stop at default.asp page. And even this page is not listed in any of the search engines.
A couple of days ago Google was able to visit the default.asp page and other linked pages. I am waiting for them to appear at google.com
What can I do with the rest of the search engines?

Thanks, Andrew


11:52 am on Nov 7, 2001 (gmt 0)

10+ Year Member

Hi Andrewb and wolcome in the forum

One thin you could do is to make sure that you have a robots.txt for the robots to read.

Work on the title tag, you have no keywords in your title.

Run your pages in the simspider at [searchengineworld.com...] or the HTML validator.

Read up on some of the post in this forum


11:59 am on Nov 7, 2001 (gmt 0)

10+ Year Member

Hi, is it your site in your profile? If so I don't believe the robots will follow your javascript navigation. Try adding a site map or a footer on each page without javascript navigation.


12:15 pm on Nov 7, 2001 (gmt 0)

10+ Year Member

This is my robots.txt file
User-agent: *
Disallow: /html/
Disallow: /images/
Disallow: /photos/
it allows access to webroot folder.
These tree folders contain info that is not needed in the search engines.

I will add some keywords to the title, but is this a criteria to get to the search engine?

I am doing searches like
or just "rooflift", "rooflift.com"

All I want is to get listed even by my domain name :)

I used tools at searchengineworld to find that my meta keywords tag was bad and the page had some html errors,
can this make a problem for spiders?

Talking about links: I have no javascript navigation on the site.
It goes like this:
<a href="CaseHistories.asp" onMouseOut="MM_swapImgRestore()" onMouseOver="MM_swapImage('i51','','images/_CaseHistories.gif',1)">

javascript is only used to make onmouseover picture


Featured Threads

Hot Threads This Week

Hot Threads This Month