Google wont crawl my site

Google not crawling site

[ukee]

I have started a new website which I opened in february. Most of the other search engines have crawled my site well but google only seems to come to my site at the beginning of every month and maybe crawl one or two pages (the statistics record 5 hits) then does not come back until the following month.

Does anybody have any idea why it does this, the site doesn't appear in the google listings at all. I have a reasonable number of inbound links including some from pr5 pages. Other sites that I have built with a lot fewer inbound links have always been listed and generally within two to three weeks.

Thanks for your help with this, I'm just not sure if I'm doing something stupid that google doesn't like.

[edited by: mack at 4:00 pm (utc) on April 28, 2005]
[edit reason] Removed URL [/edit]

[txbakers]

patience my young padawan

[mack]

hi ukee,

I made a slight edit to you post, sorry but we can't allow personal url's.

What you have described is very common for a new site. What you will find is that each month Google will take more pages. Eventualy it will start to deep crawl your site.

As for new sites not appearing in the results. You may want to search the site for "sandbox"

What I find is sites in a competative industry seam to take a while to get listed properly. Not so competative term sites seam to get listed a bit faster. his is however not always the case.

Mack.

[MkSlim]

As for new sites not appearing in the results. You may want to search the site for "sandbox"

Hi Mack

I'm actually having the same problem as ukee. What do you mean by "search the site for sandbox"?

thanks,

MkSlim

[MichaelBluejay]

He means search *WebmasterWorld* for the word "sandbox". It's the term used to describe the difficulty of new sites to rank in Google.

[hp11]

What do you mean by "search the site for sandbox"?

The "sandbox" is not an official Google term, it's just a term that everyone else uses to explain why most new sites have problems with the big G. New sites in Google don't only have a tough time ranking, they can also not appear for their own company name &/or domain name.

[MkSlim]

Thanks everybody.

[peterdouglas]

I made a website almost a year ago, it shows up well in Yahoo, but still is not showing up in Google, even if I type in the exact company name. I manually submitted it and make updates to the site every month or so. It's very frustrating! Any ideas on this one?

[specter]

Maybe has your site been banned?

[peterdouglas]

Banned? I hope not! When I registered the URL, it was brand new. I even checked it in the Wayback Machine and there's never been a website attached to it that might cause it to be banned.

I can't see why it would be otherwise banned. It's a website for my friend's small business (a driving school), so there's nothing offensive, no keyword spamming or anything like that. Yahoo picked it up just fine. I manually submitted to several search engines as well as using an automated service, maybe that's what did it? Perhaps Google doesn't like that?

If so, is there still hope of ever getting it listed on Google?

[PumpkinHead]

If Google is visiting your site a few times a month then it's not banned. I had a similar situation with my site a while back but after a while google began to visit and now, after a couple of months...its starting to crawl deeper into my site.

Patience :)

[peterdouglas]

If ONLY Google would crawl that site!

The only way to find it on Google is to type in the exact URL. On Yahoo, it comes up on the first page of various keyword searches.

Maybe better meta tags would make a difference? (even though it works exceedingly well on Yahoo already).

[specter]

Banned? I hope not! When I registered the URL, it was brand new. I even checked it in the Wayback Machine and there's never been a website attached to it that might cause it to be banned.

If I'm not wrong,you claim to made your web site one year ago...

So,in order to be more accurate Is important to know when your site was submitted to big G:

If it's a year there is ceratinly a problem.
If it's only a month or so ,then,it's normal as reported also by PumpkinHead.
If you have the site as of one year but you've "renamed" or "moved",then,it'is normal,again.

Check your log and pay attention if Google recognize at least your home page,and if yes how many times it is "visited" ;otherwise what are the paths tracked.

Remember that Google recognize sites starting from outbound links:So if your site isn't linked to anyone it willn't be crawled.If you changed url but it isn't linked to anyone it is worth like zero.
Check also your internal links ;keep in mind that non textual links could be considered as defective links by big G...

Let us know

Specter

[peterdouglas]

I submitted that website to several search engines in June or July 2004, so that's almost a year ago. I've checked the logs and dont' see any Google bots that crawled it, but I may be missing it. There are quite a few hits from Yahoo and (some) MSN searches, plus hits from print advertising.

Today I submitted the website to some relevant driving school and driving related directories, so hopefully that will help.

To be honest, that was one of my first decent sites, and I made many errors, like not using meta tags and using images maps for links, which I'm sure don't help. But it's always #1 or at least on the first page for many Yahoo searches. Maybe I'm just lucky.

But, looking back on it now, I think the main reason might be that out of 14 pages, two are "under construction" although they're French pages. The 7 English pages and 5 of the French pages are 100% complete. I think if I redesign with CSS, make good use of meta tags and ensure all pages are complete, that should get me listed on Google, what do you think?

[specter]

There is a problem that doesn't depend on meta tag as Google ignores them.
And it doesn't depend also on the fact that your pages are not completed,altough big G loves content.
It could just depend on your image maps used as links;as already said Google has some difficult to recognize non textual links.

Or on the quality/missing of inbound links.

In my opinion,is not necessary to modify meta tags.
As you 're good ranked on Yahoo and MSN,I would avoid absolutely to touch them.

Try to add content at least to your home page and link it to ALL the other pages of your site via simple textual links being sure that they work well (you could use a tool such as Link Viewer to check them).Get at least two or three good inbound links.Pay attention:in order to give you a result,the links have to come from indexed pages,so be sure that pages want to link you are already ranked on Google.
This is required also for the directories you've submitted to:if the directory is not listed in Google SERPs,its link is invalid for you.

Once you satisfy these conditions you can submit your site to big G.To do it before is only a waste of time.

Let me know and feel free to sticky me if you want.

Hope to be helpful

Specter

[peterdouglas]

Thanks Specter, those are some good points. The site is almost a year old and in need of a redesign. I don't plan to change the content much (except to finish the two incomplete pages). The main purpose will to be to give the website a fresh, new look, add or replace some pictures, use CSS instead of tables and to use better graphics as my skills in all areas have greatly improved in the last year.

Oh, there is one change we were thinking of doing... having the home page be a basic page with a choice to enter the site in English or French. Now, that would certainly not be good for search engines, unless I added a paragraph of content to the index page. Or, I'll leave it as-is where it defaults to the English page and every page on the site has a link to switch from English to French or vice-versa (landing on the equivilant page).

There are actually NO meta tags or headings used! I guess I should be lucky for the hits from Yahoo and MSN. I manually submitted to quite a few search engines when I originally made the website.

There are currently no inbound links to that website. But, yesterday I submitted the site to a few relevant directories that ARE indexed on Google, so hopefully that will help. Once I redesign it, I'll add a link to it from my portfolio site, which is indexed on Google. Aside from this, I don't know where else I could find others to link to the site. There are very few outbound links, just one to my web design services site at the bottom of the home page and a couple to pages on a government website, and a link to download Adobe Acrobat reader since one of the outbound links is for a government PDF.

One last thing... how do I "sticky you" and quote others? (I'm new to this forum, but I'm so glad I found it)!

Thanks.

[specter]

To sticky someone you only click on the mail icon on the left of his post.
To quote you have to copy/paste the desired text and include it in the quote code.
A complete list of codes is available here:

[webmasterworld.com...]

If I'm not wrong it should be reachable from your control panel.

[jeffb]

Two things made a sudden difference in Google spidering for me. I had started a website a year ago and then pretty much abandoned it in favor of other priorities. Whenever I checked, there was absolutely no spider activity at all.

A couple of months ago I turned my attention back to that site, added a blog and got a single link from each of the two home pages of the sites I had been working on.

Almost instantly, all the search engines started crawling it like crazy. I wish now that I had done those two actions separately so I could have seen which had the greater effect. But I'm sure both had a lot to do with the sudden spider activity.

[peterdouglas]

It might have been a combination of both, or maybe the links. I submitted my site in question to several relevant directories, and now it comes up as #3 in a Google search when someone searches the name of the company.

I'd try a blog, but it's really not appropriate for this site, and my friend who owns the company, will not bother to keep up a blog.

[specter]

It doesn't depend on Blog.
Inbound links are the secret.

[pantheria]

okay , I am a beginner so please explain , what are inbound links?

[jeffb]

Inbound links are the links you get from other sites. The search engines treat links that you get from other sites as if those other sites were "voting" for you as a relevant site on the topic of the text that they put in their link to you. The more sites that "vote" for you, the more relevant the search engines consider your site to be.

That's the basic expanation. There's a lot more to link popularity than just the total number of links. Links from sites that the search engines suspect are owned by you (that have a very similar IP address, suggesting that they are hosted from the same server) are ignored. Links that appear on every page of a totally unrelated site (suggesting that you bought what's called "run-of-site" links) are ignored. Links from sites that are totally unrelated to your site's topic are given less weight in figuring the "vote" total than links from sites that are closely related in topic. Links from sites that are themselves considered highly relevant sites (such as major directories) are given more weight in figuring the "vote" total than those that few other sites link to.

But inbound links are an important part of how the search engines determine how relevant your site is for various keywords.