homepage Welcome to WebmasterWorld Guest from 54.204.182.118
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Strange listing issues
Can someone spare 2 mins to help me?
badass101

5+ Year Member



 
Msg#: 29152 posted 11:12 am on Apr 22, 2005 (gmt 0)

Hi all,

I've got a problem with a fairly new site I've made.
It's got a few good backlinks, and there are more coming.
The 'bots have all been around to say hello, but they only seem to be grabbing / and robots.txt
Googlebot did crawl a few more pages the other day (5 or 6) but nothing else.

My listing hasn't appeared in MSN at all yet.
It's in Google, but it's currently URL only (it was URL + description last night! and URL only a few days before).

I was wondering if someone could spare 2 mins to have a look at my robots.txt or my page and let me know I'm not doing something wrong.

Stickymail me for the URL, or let me know some suggestions/thoughts if you have any.

Thanks a million

Craig

 

MHes

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 29152 posted 12:45 pm on Apr 22, 2005 (gmt 0)

Hi

Try this:

[searchengineworld.com...]

jschmitz

5+ Year Member



 
Msg#: 29152 posted 2:02 pm on Apr 22, 2005 (gmt 0)

You need to check your pages, not just your robots.txt... and sometimes they come, sniff, and then come back. Robots.txt doesnt really guarantee anything in particular....some sites that I have put up have done well right away without it, and some with it take quite a will to get crawled deeply. Still, you shoudl be showing some ranking if your internal links are good and your pages are coded well.

j

badass101

5+ Year Member



 
Msg#: 29152 posted 2:27 pm on Apr 22, 2005 (gmt 0)

Hi guys,

Thanks for your feedback. The robots.txt validates ok - I'd checked that before posting.
The site is built with a popular CMS, so I'm assuming the internal linking is ok - the site works ok!
Everything is mod-rewritten to be search engine friendly (except a few sections which I can't adjust, but they aren't major.)

Anything else I should check?

Thanks

Craig

kpaul

10+ Year Member



 
Msg#: 29152 posted 11:35 pm on Apr 22, 2005 (gmt 0)

how new is fairly new?

it seems like you've covered all the basic bases so i would concentrate on building new content each day and getting relevant links.

check out Brett's classic 26 steps to website success for other things you can be doing.

patience is a virtue ;)

shri

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 29152 posted 12:37 am on Apr 23, 2005 (gmt 0)

It can take upto a month or more for the first deepcrawl to start.

badass101

5+ Year Member



 
Msg#: 29152 posted 8:54 am on Apr 23, 2005 (gmt 0)

I should try and be more patient, but you all know what it's like! ;-)

Just seemed a bit odd, as most of the other sites I've done get deep crawled very quickly.
I guess it's just the law of the Google!

I noticed Teoma came by last night and took a page that wasn't the index, so some spidering is happening.
Does it usually only take the robots.txt and one page?
(I've never concentrated on it much before)

Thanks

Craig

badass101

5+ Year Member



 
Msg#: 29152 posted 2:18 pm on May 5, 2005 (gmt 0)

Ok, here's the update for you all - with a few questions!

I'm getting hit every day without fail by Googlebot for my robots.txt and /
Last week Googlebot (a different one, with Mozilla 5 compatible agent in Urchin) came by and deep crawled. This was last Thurs.
In Google, I'm still just showing my homepage.

I'm also being hit by Slurp from Yahoo! - In fact, it's gobbling me up - It grabs a large chunk of pages daily and did a huge hit a few days ago (more pages than I have, so it must have gone a bit haywire)
In Yahoo! I have 2 entries if I search for #*$!.com and press 'More results from this site' and both are my homepage - one with my Meta description and one with my content snippet.

Oh, and same story from MSNBot - it loves spidering me but won't give me anything on the SERPS.

I'm also being slowly grabbed by Teoma, but I have no idea how to check if I'm in the SERPS there.

So, I'm still not appearing in search results - I know it's a finger in the air job to predict timescales, but does it sound like something isn't quite right?

Craig

jdhuk

10+ Year Member



 
Msg#: 29152 posted 9:40 pm on May 7, 2005 (gmt 0)

Hey Craig,

Just a thought, is the domain brand new never used before?

The Combsinator

10+ Year Member



 
Msg#: 29152 posted 11:31 pm on May 7, 2005 (gmt 0)

I went to the link that Mhes just posted and every file said Error..I'm thinking that isn't good :(.

badass101

5+ Year Member



 
Msg#: 29152 posted 12:54 pm on May 8, 2005 (gmt 0)

Hey Craig,

Just a thought, is the domain brand new never used before?

As far as I know - what's the most accurate way to check?

badass101

5+ Year Member



 
Msg#: 29152 posted 12:55 pm on May 8, 2005 (gmt 0)

I went to the link that Mhes just posted and every file said Error..I'm thinking that isn't good :(.

Huh?
Sorry, I don't understand what you mean... (me being stupid!)

jdhuk

10+ Year Member



 
Msg#: 29152 posted 7:51 pm on May 8, 2005 (gmt 0)

As far as I know - what's the most accurate way to check?

You can check if your domain was previously owned here....

[archive.org ]

badass101

5+ Year Member



 
Msg#: 29152 posted 10:26 pm on May 8, 2005 (gmt 0)

Thats the way I knew - no, nothing comes up on a search there.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved