Forum Moderators: open

Message Too Old, No Replies

Gbot deep crawl pointer domain

Has not bee back since?

         

derekwit

9:00 pm on Jan 5, 2003 (gmt 0)

10+ Year Member



Three months ago I had a pointer domain pointing to my main domain. I noticed that Gbot had spidered that domain. After I saw what had happen I immedately to that pointer out of my domain. The next month Gbot tried to deep crawl that site again. But unforunately I had removed it and it was unable to find it. Now for the last (2) months Gbot fresh crawls my site but does not deep crawl my domain.

Key_Master posted this thread:

HTTP/1.1 400 Bad Request
Server: Microsoft-IIS/5.0
Date: Sun, 05 Jan 2003 03:50:23 GMT
Content-Type: text/html
Content-Length: 87
(This server violates the HTTP standards by
returning content after the header in a HEAD request:)
<html><head><title>Error</title></head><body>The parameter is incorrect. </body></html>

That's the error I got. Your server isn't returning a proper head request (it's giving both the server headers and the page contents). Oddly enough, on every tool I ran, the last line of HTML was missing from your home page. The only time I've seen this happen is when an improper content-length was given by the server. Viewing from a normal browser doesn't cause the error to happen. Some very weird stuff is going on there.

"ALL" is equivalent to "follow,index" so your robots meta tag is not your problem.

What tools did you use. What can I do to fix?

Thanks in advance for the help...

Key_Master

9:36 pm on Jan 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Check your domain here to see the problem:

[mdb.ku.dk...]

SamSpade shows the same problem as well. What's really strange is that some tools don't seem to see the problem. It's almost as if your server changes the rules depending on what part of the internet the head request came from. Now all of this doesn't mean it's the reason why Googlebot won't crawl your site but it would definately raise alarm bells for me.

Yes, you should check your error logs and let your host know about this problem.

derekwit

11:00 pm on Jan 5, 2003 (gmt 0)

10+ Year Member



No quite sure that links is working correctly. I have tested the top rated sites for my Keyword I get the same response for all pages as I do for mine. Any other suggestions.