'Blank' records for my site

FAST has spidered some of my pages but refused to do anything with it.

[ses4j]

My site has been up for ~6 months now. FAST-Webcrawler came about 10 times in the first 4 months, got my robots.txt (which did NOT exclude it, for sure) and went away without getting anything else.

The whole time it had a 'blank' record for my homepage - it didn't come up in any searches, but if I put in the URL as my search term, it would come up as:

www.example.com
Description: my title from my home page in lowercase
http://www.example.com

and the 'FAST facts' were:
Last changed: 1970-01-01, 00:00 GMT
Document size: 42 bytes

If I clicked 'who links to this page' I'd get from 40-60 inbounds listed.

If you'd like to see it in person, sticky me and I'll be happy to send you my URL and gripe in private. :)

I had some very tentative hits, finally, last month, it touched a few more pages and did the same thing with those - 'blank' records with no last changed and 42b size (untrue of course).

Why won't it just get my site? Or at least the pages it pretends to have? Any ideas?

Do I care? What % of traffic do you typically get from alltheweb.com? Any?

Thanks!
Scott

[edited by: engine at 11:52 am (utc) on Nov. 14, 2003]
[edit reason] examplified & de-linked [/edit]

[heini]

>Last changed: 1970-01-01, 00:00 GMT
Document size: 42 bytes

Sounds like a problem on the spiders end to me. Was it FirstPage?

[ses4j]

FirstPage? Do you mean FrontPage? Nah. It's a very straightforward site - mostly static .HTML, very simple CSS, normal text nav bar, a few .php pages and just a touch of validating JavaScript here and there. There is NOTHING weird about it, I can't imagine the spider's screwing it up.
Whatever... to hell with 'em, I'll just have to stick to riding the Google wave for ~70% of my referral traffic. I hope Florida treats me well after the storm blows over, right now I'm in the dumps.

Scott

[Rumbas]

Scott, no not Front Page, but FirstPage - the name of one of the FAST spiders:
[webmasterworld.com...]

>I'll just have to stick to riding the Google wave

I wouldn't do that. The old story about the eggs and only one basket. As you mention regarding the florida update - what if Google drops you?

Give the FAST spider some time, but don't stress about it, it will eventually come by and eat all you put in front of it :)

[takagi]

Have seen those several times at AlltheWeb; usually the description is the link-text (with periods replaced by spaces). Just means the page is known to FAST, but not yet spidered. If the robots.txt is OK, just wait some longer and the problem will go away.

Google would in such a situation only show the URL as a link in the SERPs and a 'Similar pages' link. No snippet, no size, no 'Cached'.

See also: ATW reads robots.txt then leaves [webmasterworld.com]

[ses4j]

YES! I'm IN! I'm so excited. Now I can't wait to see the 2 ATW hits a week fill the gaping void left by Google dropping me in half my important keywords from first page to gone in the Florida maelstrom..

btw: I didn't change anything, it just took 'em 6 months or so to decide I'd win the "will he pay?" battle-of-wills. Ironic how I was on my way to check one last time before I broke down and paid up for PFI... HA!

Scott