homepage Welcome to WebmasterWorld Guest from 54.235.61.62
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Strange listing issues
Can someone spare 2 mins to help me?
badass101




msg:743387
 11:12 am on Apr 22, 2005 (gmt 0)

Hi all,

I've got a problem with a fairly new site I've made.
It's got a few good backlinks, and there are more coming.
The 'bots have all been around to say hello, but they only seem to be grabbing / and robots.txt
Googlebot did crawl a few more pages the other day (5 or 6) but nothing else.

My listing hasn't appeared in MSN at all yet.
It's in Google, but it's currently URL only (it was URL + description last night! and URL only a few days before).

I was wondering if someone could spare 2 mins to have a look at my robots.txt or my page and let me know I'm not doing something wrong.

Stickymail me for the URL, or let me know some suggestions/thoughts if you have any.

Thanks a million

Craig

 

MHes




msg:743388
 12:45 pm on Apr 22, 2005 (gmt 0)

Hi

Try this:

[searchengineworld.com...]

jschmitz




msg:743389
 2:02 pm on Apr 22, 2005 (gmt 0)

You need to check your pages, not just your robots.txt... and sometimes they come, sniff, and then come back. Robots.txt doesnt really guarantee anything in particular....some sites that I have put up have done well right away without it, and some with it take quite a will to get crawled deeply. Still, you shoudl be showing some ranking if your internal links are good and your pages are coded well.

j

badass101




msg:743390
 2:27 pm on Apr 22, 2005 (gmt 0)

Hi guys,

Thanks for your feedback. The robots.txt validates ok - I'd checked that before posting.
The site is built with a popular CMS, so I'm assuming the internal linking is ok - the site works ok!
Everything is mod-rewritten to be search engine friendly (except a few sections which I can't adjust, but they aren't major.)

Anything else I should check?

Thanks

Craig

kpaul




msg:743391
 11:35 pm on Apr 22, 2005 (gmt 0)

how new is fairly new?

it seems like you've covered all the basic bases so i would concentrate on building new content each day and getting relevant links.

check out Brett's classic 26 steps to website success for other things you can be doing.

patience is a virtue ;)

shri




msg:743392
 12:37 am on Apr 23, 2005 (gmt 0)

It can take upto a month or more for the first deepcrawl to start.

badass101




msg:743393
 8:54 am on Apr 23, 2005 (gmt 0)

I should try and be more patient, but you all know what it's like! ;-)

Just seemed a bit odd, as most of the other sites I've done get deep crawled very quickly.
I guess it's just the law of the Google!

I noticed Teoma came by last night and took a page that wasn't the index, so some spidering is happening.
Does it usually only take the robots.txt and one page?
(I've never concentrated on it much before)

Thanks

Craig

badass101




msg:743394
 2:18 pm on May 5, 2005 (gmt 0)

Ok, here's the update for you all - with a few questions!

I'm getting hit every day without fail by Googlebot for my robots.txt and /
Last week Googlebot (a different one, with Mozilla 5 compatible agent in Urchin) came by and deep crawled. This was last Thurs.
In Google, I'm still just showing my homepage.

I'm also being hit by Slurp from Yahoo! - In fact, it's gobbling me up - It grabs a large chunk of pages daily and did a huge hit a few days ago (more pages than I have, so it must have gone a bit haywire)
In Yahoo! I have 2 entries if I search for #*$!.com and press 'More results from this site' and both are my homepage - one with my Meta description and one with my content snippet.

Oh, and same story from MSNBot - it loves spidering me but won't give me anything on the SERPS.

I'm also being slowly grabbed by Teoma, but I have no idea how to check if I'm in the SERPS there.

So, I'm still not appearing in search results - I know it's a finger in the air job to predict timescales, but does it sound like something isn't quite right?

Craig

jdhuk




msg:743395
 9:40 pm on May 7, 2005 (gmt 0)

Hey Craig,

Just a thought, is the domain brand new never used before?

The Combsinator




msg:743396
 11:31 pm on May 7, 2005 (gmt 0)

I went to the link that Mhes just posted and every file said Error..I'm thinking that isn't good :(.

badass101




msg:743397
 12:54 pm on May 8, 2005 (gmt 0)

Hey Craig,

Just a thought, is the domain brand new never used before?

As far as I know - what's the most accurate way to check?

badass101




msg:743398
 12:55 pm on May 8, 2005 (gmt 0)

I went to the link that Mhes just posted and every file said Error..I'm thinking that isn't good :(.

Huh?
Sorry, I don't understand what you mean... (me being stupid!)

jdhuk




msg:743399
 7:51 pm on May 8, 2005 (gmt 0)

As far as I know - what's the most accurate way to check?

You can check if your domain was previously owned here....

[archive.org ]

badass101




msg:743400
 10:26 pm on May 8, 2005 (gmt 0)

Thats the way I knew - no, nothing comes up on a search there.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved