Welcome to WebmasterWorld Guest from 34.231.247.139

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Inclusion in the Google Index

E-Mail sent to me from Google.

     
6:46 pm on Feb 16, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 5, 2001
posts:2732
votes: 11


Greetings WebmasterWorld :)

Figured I'd make a post here since I haven't in a while. I recieved an email from someone at google telling me the benifits of allowing the google spider to index my site. The site in question has a robots.txt file disallowing all search engines.

The email includes the URL in question and states that atleast 50% of internet searches come from google and its important to allow google to spider the site.

First reaction was, automated responce? Did I send an inquiree to them? Nopes certainly not for this domain so I sent an email back asking why the email and I expected it to bounce. To my surprise this morning I got a reply back saying they want to include the site because it was identified as being highly ranked.

New sort of team at google trying to get people uninformed and disallowing google to switch back and allow google to spider the data?

10:48 pm on Feb 20, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 11, 2005
posts:143
votes: 0


I disagree. Even if scrugle develops its own content, there is are lot of webmaster who would be looking for revenue. MSN its just gonna pick those up and eventually kill scrugle. Its a damn huge web.
3:14 am on Feb 21, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 23, 2005
posts:57
votes: 0


Netscape was the king. Then IE came out of the closet :)

Same thing _could_ happen to google also.

3:17 am on Feb 21, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


I agree with xalex... adsense/adwords alone is a highly profitable business. Why would Google kill that revenue?

Regarding overture... well I am going to give them a try soon... Will divide the spending between overture and adwords and see which performs better in terms of ROI.

3:19 am on Feb 21, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


Hi great_9,
I really hope it will... because if it happens we'll be back in glory days!
Regards
3:47 am on Feb 21, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 21, 2005
posts:21
votes: 0


I wouldn't be surprised if MSN takes over.. The MSN bot has been crawling my sites (relatively small too) frequently (almost every day). Google I'm lucky if they update one page per month. I've also just begun to show up in the ranks ing Google again for my keywords, but MSN and Yahoo have both been listing me #1 for a while. It still boggles my mind how a search for my name didn't bring my site (myname.com) up anywhere in Google when it used to be #1. It's not like I changed my name..
4:01 am on Feb 21, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 19, 2005
posts:5
votes: 0


I agree with xalex... adsense/adwords alone is a highly profitable business. Why would Google kill that revenue?

Right, isn't adwords more than 90% of google's earnings or something like that?

4:02 am on Feb 21, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 19, 2005
posts:5
votes: 0


Also, you all seem to know exactly when the bots are crawling your sites, and the paths they take, etc... what tool or log viewer do most of you use?
4:16 am on Feb 21, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:139
votes: 0


I agree with xalex... adsense/adwords alone is a highly profitable business. Why would Google kill that revenue?

They will not kill Adwords, that’s for sure. But they may kill Adsense (in other words, get us publishers outside the loop). That's what i meant.

Hopefully a new player will be around to pick up the pieces (us publishers) when it doe's happen and start it all over again, hopefully.

That's my long term view on the Adsense program anyway. I see it coming and it wouldn't take long now. Watch out and be prepared!

10:50 pm on Feb 21, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2003
posts:730
votes: 2


From Andy:

"If a site is intentionally trying to influence the SERPs with black hat techniques, then they deserve to be caught and dealt with."

"By saying goodbye to Google, it no longer is a consideration, or a concern. I can maximize my site for the other SEs with more stable results."

I read these two parts of your message and found them to be inconsistent with each other. It all sounds like it's not about how your results fare in Google, but about how much you can earn.

Cheers

11:02 pm on Feb 21, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 11, 2005
posts:513
votes: 0


I read these two parts of your message and found them to be inconsistent with each other. It all sounds like it's not about how your results fare in Google, but about how much you can earn.

Inconsistent?

If a site is doing something it shouldn't be doing, it deserves to be caught and dealt with.

If your site is no longer in Google, due to a GBot ban, the ups and downs of the updates cease to be a consideration. I find MSN and Yahoo to be much more stable and consistent with their SERPs, without the huge swings that Google so frequently has. I think people are tiring of the Google drama.

Can't be much clearer than that. Neither of those statements mention earnings.

Andy

4:10 am on Feb 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 11, 2005
posts:143
votes: 0



But they may kill Adsense (in other words, get us publishers outside the loop). That's what i meant.

No. Thats suicidal. AdSense is huge revenue source for them. Plus its much more diversified business model. If google search fails, then how is google gonna make AdWords work?

6:30 am on Feb 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:139
votes: 0


If google search fails, then how is google gonna make AdWords work?

Over google's own content network and the large number of services they currently have and about to have (I listed just a few above).

This will automatically reduce the ad inventory for adsense publishers, consequently reducing adsense earning (for publishers) to the point were publishers will start pulling out the ads from pages due to very poor earnings (and will start looking for alternatives). This could signal the end of the adsense program (not the Adwords program).

7:07 am on Feb 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


<< Over google's own content network and the large number of services they currently have and about to have (I listed just a few above).

This will automatically reduce the ad inventory for adsense publishers, consequently reducing adsense earning (for publishers) to the point were publishers will start pulling out the ads from pages due to very poor earnings (and will start looking for alternatives). This could signal the end of the adsense program (not the Adwords program). >>

I don't understand why you are saying this? Let's say Google is earning 70 billion now.. mainly through it's publishers and adwords and google's own "existing" content.

Eventually let's say Google manages to creat a WHOLEEEE lot of content and earn 100 billion from just that..
Why would google want to leave the 70 billion [or say 30 billion] from the web publishers?
And this will definitely be picked up by MSN or Yahoo... Why would google want that? Isn't 170 billion more than 140 billion?

7:42 am on Feb 22, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 19, 2005
posts:5
votes: 0


Isn't 170 billion more than 140 billion?

Right... I don't see any drawback in keeping adsense, or a point in dropping it.

That takes me back to an earlier point on another topic. Why ban the googlebot just because you're not getting the high rankings that you want? Isn't 1.1 million page views better than 1 million?

If your only actual problem with Google is that the crawler eats bandwidth, and you can't monetize that with the search results it provides, than you have bigger problems.

8:02 am on Feb 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


I think the actual problem is that google has been so good to so many web masters before that the expectations are verrry high... it's kind of like a love marriage :-D you expect more than what you would with an arranged marraige.

Frankly, I have not received any real SERPS from google [still in the sandbox] and I really really think [and hope] that Google will change its newly formed strategy of monetizing everything. This on the long run will be highly degrading to it's brand name. I do like Google for its Adsense/adwords...

Sometimes I wonder if Larry Page and Sergey Brin are still on the drivers seat after the IPO!

ALSO:
All these threads about "low earnings"... so they all have legit websites? How many are actually referring to built for adsense sites? Is there really any way to judge anything from forum posts!?

11:38 am on Feb 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:139
votes: 0


Why would google want to leave the 70 billion [or say 30 billion] from the web publishers?
And this will definitely be picked up by MSN or Yahoo... Why would google want that? Isn't 170 billion more than 140 billion?

Asking such question is only if one assumes that there is an endless stream of new Adword advertisers with infinite budgets coming in (as well as infinite number of web surfers). I don't think there is and your figures are sky high.

It is all about supply and demand and market forces. Currently as it stands now, content (ad space on quality “clean” pages) is in short supply which drives or at least holds ECPM prices at +-20% $4-5. Once more content is introduced (by google) ECPMs to Adsense publishers will fall as supply outstrip demand. That simple.

Also don’t forget that many adwords (aff) advertisers are outside the loop now with the new rules recently introduced. This further decreases the demand for ad space. Sure, some of them will be back once they'll learn how to create landing pages on their own servers etc. but it is not going to be what it used to be before. Lots & lots (aff) advertisers have disappeared following the introduction of the new rules.

Check the Adsense board and you can see for yourself how it is already impacting/hearting Adsense publishers earnings. I think the switch was done approx 1st Feb which almost immediately translated into less earnings for many publishers.

Firing a few Adword campaign to promte aff products and earn commissions almost turned into a national sport. Aff advertisers are almost all gone now and so is a serious chunk of ad space demand.

My two cents...Party is about to be over. At the end of the day, it is all about supply and demand. Demand will increase with time but not as fast as supply (Google intrducing tons of new content, and less we forget gmail almost infinate ad space as mre and more avrage joe's sign up with this service).

The reasons above will Keep ECPM prices low for some time to come and may eventualy kill Adsense (it will no longer be needed, we have all the ad space we currently need, lets keep only a selcted few publishers - the big publishers, tell the rest we thank them very much for the PR work, they can take a rest now).

1:24 pm on Feb 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


Okay I do understand what you are saying but have taken into account the following:

1) Internet users are growing.

2) Yahoo has had affiliate impressions for a long time now but affiliate marketing is still alive and kicking.

3) ECPM depends on loads of factors including what happens AFTER the user clicks on the ad.. does he eventually convert for the advertiser... this has an effect on the ECPM.

4) Will scrogle book page show up on SERPS? Amazon and knowledgestorm also allows users to read the book but they do not show up on results [they have it blocked in some kind of frame] Will google be implementing such a thing.. Probably they have to so that NO ONE can copy the book's material.

As I see it scoogle, gmail... all these programs will have a very low conversion rate and thus lower ECPM. They'll have billions of impressions but they'll earn low.

Here's an example:
Say I have a site about digital cameras. Now suppose scrogle has a book on digital cameras. How will the user get to that book? Say through a search engine result... what did he search for? "how does a digital camera work?" and BANG! Google's book is there.. it's obvious that the user is NOT in a buying mode.. so even if the he sees an ad he may NOT convert.
Whereas my website has reviews of a digital camera... now unless there's a book on "Kodak 320 review" I am hoping #*$! will not show up. Not to mention that when this user comes to my site he IS in a buying mode.

They DO need us publishers for creating those conversions.

[This is just an example of course... I know nothing about digital cameras ;-)]

We saw the same thing with Google's blogger.com earnings... It was nothing to talk about really. But they sure did get a whole lot of impressions.

As far as I see it I think adsense and adwords will both grow.. these are novel programs which work! Though I seriously doubt if Google will see the end of this decade if things with the NATURAL SERPS don't improve.

Would really love to hear what the senior members here have to say.

2:46 pm on Feb 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:139
votes: 0


1) Internet users are growing.

Not as fast now as in previous years. There’s almost a computer connected to the internet in every house nowadays.

As I see it scoogle, gmail... all these programs will have a very low conversion rate and thus lower ECPM. They'll have billions of impressions but they'll earn low.

Disagree.

Example, Jim writs an email to his girlfriend talking about their planed trip to Europe. They are both researching accommodations. Girlfriend opens Jim’s email (in gmail) and see accommodation ads in Europe above Jim’s message, Girlfriend clicks and may book something or send a link back to Jim from the advertiser page...you get the point.

My point being, they are not necessarily junk impressions. They are as good if not better then web site content impressions. Via email you really get a pick at what people topic of interest is and what they are currently researching. They no longer need to look for the ad on the SERPs, The ad finds them and it is laser targeted to the content of the email they are currently reading or sending. They discuss it right there on the gmail page with their friends/family. They no longer need to look for the ad on the SERPs while researching the topic. The ad finds them as they write or read their email and their topic of interest is being analysed.

It is a much better targeted Ad match to my opinion and may yield better conversions for the advertiser then lets say content advertising. This also explains why RIO is better from search pages then content pages. Reason, when searching the user actually types his topic of interest into the search box…ads deliver accordingly and RIO is normaly better for the advertiser (then content ads).

3:34 pm on Feb 22, 2005 (gmt 0)

Full Member

10+ Year Member

joined:Jan 21, 2005
posts:298
votes: 2


Agreed: on your GMail example. But I really do not think that it will be an end to adsense buddy.

And yes... agreed on your search engine ads point of view too. But saying that addition of another service will kill all affiliates and publishers is wrong. The market will evolve yes... but there will always be adsense or the like.

If your search engine example is taken to be the cornerstone of Internet business... then yahoo / MSN / Lycos and thousands of other small services would not have survived.

We all know what kind of PR Google holds... Anyone using the internet for the first time will be taught how to search.. and naturally Google! But still.. here we are making a business out of adsense and affiliates.

The market is constantly evolving... and tomorrow I believe there will be another alternative to adsense that will pay higher. Google will be forced to pay higher percentage of the bid to keep publishers... and no I don’t think they’ll say.. hey! What the hell we have Gmail now.. let them go. They need the PR / the 30-40 % cut on the revenue. It’s symbiosis. Everything is still the same... just a few little additions.

The web is really tooo big for google's GMail or scrogle to rule alone. There'll be other kings [Google] and other kingdoms [yahoo] for us to choose from. We are still at the beginning of this Internet age... Things will change... New things will prosper... But there'll always be a way to work and get paid. After all everything is about people and what they choose... And part of the reason why people buy on the internet is because of the brilliant copy and content we publishers produce. Which GMail / Yahoo mail do not have and cannot have.

The girl friend may click on the ad on Gmail but if she has brains she’ll surely research to find the best place to stay.. Find out what the people say about that place she’s considering to book for her vacation. Even if we take a real dumb girl…. Well then I wonder if she knows how to use a credit card online.

Ruling out the human factor is not a wise decision and I am sure Google or any other big player knows that. Any business needs partners… and it’s because of these publishers that Google has gained it’s name and they have been wise to stick with them. If they leave… well the people leave… and who knows maybe a new player will emerge and that’s where all those left alone people will flock… that’s where the party will move… and as we know.. a party means business.

Tomorrow morning I can shift to adsonar and get paid more... I've heard that they have higher CPC here at WebmasterWorld. But I choose to stay with Google because of it's trustworthiness and it's name in the industry and Google knows that I as a publisher am an important link.

3:51 pm on Feb 22, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 27, 2003
posts:730
votes: 2


I read this thread and I see lots of people blaming Google for their misfortune. But what I want to know, is how much responsibility each of you has about the fate of your respective Web sites?

I read a bunch of recrimminations, but I never see anything about what "bad" things you guys might have done to get banned or whatever. And worse, I think the attitude of people saying Google owes them is wrong. If Google wants to show your results in the last pages of its search engine, it's their business. No one "deserves" to be in the top results.

Many of the criticisms against Google seem to hide other motives such as magic key words like "Florida" and "Allegra."

This is what I hinted at when I quoted Andy's inconsistent positions.

Finally, some of the critics here seems to be "professional" Google haters. I'm not an expert in viral marketing, but I'm not an idiot, either. It doesn't take a smart alec to sniff a well targeted and organized PR campaign. And I, one of the typical and desired targets, am not falling for it.

Clean up your acts, guys.

4:07 pm on Feb 22, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 9, 2004
posts:139
votes: 0


roycerus,

Some wishful thinking there but what can I say,

I can only hope (like many other webmasters who earn their leaving online) that you will prove correct and things will pan out the way you draw them.

After all this is what we all want!

Time will tell...

4:13 pm on Feb 22, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 11, 2005
posts:513
votes: 0


This is what I hinted at when I quoted Andy's inconsistent positions.

Finally, some of the critics here seems to be "professional" Google haters. I'm not an expert in viral marketing, but I'm not an idiot, either. It doesn't take a smart alec to sniff a well targeted and organized PR campaign. And I, one of the typical and desired targets, am not falling for it.

Clean up your acts, guys.


Aren't you assuming a lot here? Since you can't possibly be familiar with the sites of everyone who has posted in this thread, you seem to be lumping them all together as bad sites, deserving of the recent fall they've experienced in the Google SERPs.

I can tell you that my site is 100% clean. No cloaking, no doorway pages, no purchased links, no outgoing links that aren't on topic, no links to anything other than quality sites of interest to my site's visitors, no keyword stuffing, no hidden text, etc. I've done nothing that I know of to deserve sitting at #250 for my site name while other sites above mine all LINK TO ME! That is not relevancy.

There is nothing in my act to clean up. Google presents itself as a fair, relevant search engine with the most indexed pages of all SEs. I know for a fact that this is not necessarily the case in all situations, and by allowing my site to remain buried in their SERPs is acceptance on my part that I have a poor quality site, not deserving of a higher rank. And that is not correct.

I know it is impossible to judge this without seeing a site, but sometimes you have to take people at face value. My site is well liked, and considered an authority site in its field. It has several PR9 inbound links from highly respected sites. Based on where it sits now, Google's opinion of my site is that it's less worthy than the spammy directories, and copycat sites that offer no value whatsoever that rank above it. Based on principle, I will not allow my site to remain in Google's index as long as there's a black mark placed on it by Google. It deserves better. Google's searchers deserve better.

This isn't about money, it's about believing in the quality of your work, and standing behind it.

9:40 pm on Feb 22, 2005 (gmt 0)

New User

10+ Year Member

joined:Feb 19, 2005
posts:5
votes: 0


AndyA, you sure sold me on your site. Is this a Google only problem, or are you getting better results with MSN/Yahoo!?
12:52 pm on Feb 23, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 11, 2005
posts:513
votes: 0


Breadstick,

Yes this is a Google only problem. Google Search, that is. I'm doing very well in Yahoo, MSN, and I'm getting more traffic from Google Images than Google Search. Strange that the images are worthy, but the pages they're on aren't...

2:47 pm on Feb 28, 2005 (gmt 0)

Preferred Member

10+ Year Member

joined:Feb 17, 2004
posts:597
votes: 0


Well, I give Google the benefit of the doubt that the current results are a glitch - I will wait for the upcomming update, but if nothing changes - I WILL ban Google from visiting my website.
My website is not banned, but it fell from #10 to #110 - exactly 100 spots. And the thing is that I havent done anything out of the ordinary - it was a day like any other, and all of a sudden - BAM - 100 spots down, so a website with exactly 4 lines of text could get in my old spot :) I dont think so, google!

It's business, and of course everything comes to money - I give Google content so they can generate revenue, and I expect the same in return - throw me a bone here, ah?

The webmasters who post good things about Google in this post, are those with good current SE positions, but they will soon find how wishy-washy Google is, and will come back crying about their new position.

I have always been in the top 10 results on G for my very competitive keywords, and I have gotten over all kinds of spammy techniques and kept my position until the last update. But in the past few months the tendency was - same visitors, lower quality, which, to me, it means that the Google's fame is slowly fading away, and the only thing that keeps them right now is the fact that major ISPs (AOL, RR, Comcast, etc.) are using their search results, but this wont be for long either - I see visitors to my website from Yahoo, which have aol.com IPs :)

And lets not forget that Yahoo and Hotmail are still the top email providers, and for Gmail trying to create that "big thing" with the invitations only - aint working. Plus, I cant remember the last time my email reached 1MB - I dont save spam, and emails from frends at best reach 8KB/per.

And on another note - I dont understand Google facinating DMOZ - only 1 out of 20 links there is worth clicking.
Yes, I am sick of directories in the G's search results - I did not go to a search engine to be transferred to search in a directory!
Geeesh!

This 85 message thread spans 3 pages: 85
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members