Google Link points and Robots.txt

Google points algo and side discussion of robots.txt format

[tmw]

does google have any points fixed for particular type of linking ie wheather use any points for links which affects the ranking.
thank you

[Brett_Tabke]

Many people believe there are. There are some varying theories as to the value of each style of link. My opinion, based on nothing more than experience is that I'd rate them this way: (Yahoo link, 100 points, ODP link 50 points, major authority hub link like a topic directory 5points, inbound link 2 points, insite link .50 points). Others don't believe insite links count - I do...

[paynt]

I'm going to agree again with Brett here and will add that I think you can make this .50 weight figure higher with a strong internal link strategy. Particularly true for Google.

I have found so many sites ranked higher from what appears to be a very strong internal link strategy then sites with lots of incoming links yet no internal linking strategy evident.

Start from the bottom up. Make your own site strong first and then add those incoming links. This may make your site more appealing to external link partners.

Additionally I’ll suggest you look for links from university professors. These can go for as much as 25 points if I follow Brett’s weight theory. And I think hub/authorities on theme can often weight much higher then 5 points. Certain link partnerships the same and can weigh in higher than 2 points.

[tmw]

YAHOO - 100 POINTS
ODP - 50 POINTS
UNIVERSITY PROFESSORS - 25 POINTS
HUB/AUTHORITIES - 5 POINTS
PATNETSHIP LINKS - 2 POINTS
INTERNAL LINKS - 1 POINTS
EXTERNAL LINKS - .50 POINTS

is the abouve table is correct Brett_Tabke and paynt i do agree with your stratergy.

The spider(Google Bot) will crawl the links on your site, internal and external links. If the bot finds an external link back to you, you get one point.
A directory listing eg:
DMOZ
Yahoo
Looksmart
NBCI
Will give you about 100 points for each listing.

Now lets look at the links:
All your insite links should be a keyword. Do not use index or home unless you are targeting these keywords.
Eg:
<a href="mysite.com">Index</a> = wrong
<a href="mysite.com">keyword</a> = hits

Okay now take it one more step:
Inbound links must have your keywords in them (same as above).

Is the above explanation correct?.

dear brett give a few details about HUB/AUTHORITIES, UNIVERSITY PROFESSORS site.
how do i get links from these sites and where can i find these sites.

thank you
sathish

[paynt]

Hello again,

I'm going to suggest you also read the following discussions. They all have pieces of the same puzzle.

[webmasterworld.com...]
[webmasterworld.com...]
[webmasterworld.com...]

Hints: University links. Find them in your keyword research through Google. If you have an appropriate page to offer then write the professor direct and suggest the resource. Let them know you appreciate their resources and will be including a link to them at ... Thank them, etc. The key here is it must be something relevant and you need to find the right professor offering the right topic resource page. If you have the right products or services for this then it's a terrific link.

Special Note to anyone reading this: Please do not go out and spam professors. It’s worth the link to do it right and proper and only if you really have something to offer.

Debra

[startup]

I know where you got that post from, because I wrote it in response to a very general linkage question about Google.
Take note where the post states "Will give you about 100 points for each listing". This statement is designed to convey to the readers that directory listing carry more weight that any other inbound link. Do not assume all the directories have the same weight.
Keyword embedded links, whether they are insite or inbound do produce better results.

I will stand by the true meaning of that post, even if (tmw)has removed it from its original context.

[paynt]

<correction>

A directory listing eg:
DMOZ
Yahoo
<remove>Looksmart</remove>
<remove>NBCI</remove>
Will give you about 100 points for each listing.

Because we're talking about Google and I have never seen Google giving that weight to those directories. This is not to say those aren't teriffic links for other reasons.

Other directories that still rank with a hefty weight behind Yahoo and ODP are Webrawler, Excite, <missyou>Go</missyou>, and I see Study Web and similar directories behind that.

</correction>

[startup]

Excite and Webcrawler are Looksmart listings.

[Brett_Tabke]

I do not believe looksmart or snap count in the algo any longer.

[snap.com]
[looksmart.com]

aka:

User-agent: Googlebot
Disallow:

User-agent: *
Disallow: /

[startup]

Interesting, that would exclude the main LS data base, but not Excite and Webcrawler directories.

I concider NBCI to be the unknown player of all the major ditectories. To not have it on your to do list would be an mistake in my opinion. At this time they are using Ink for secondary results, but so was Yahoo at one time.

[Xoc]

Brett-

I think you misread how the a robot is supposed to parse the LookSmart robots.txt (or I misread what you said). The Disallow doesn't have anything after it. So that means that that particular spider is allowed to spider the web site. All non-identified spiders that weren't specifically listed get hit by the Disallow: *

So Googlebot should be allowed to spider LookSmart. The Snap robots.txt, though, disallows everything.

[skibum]

As far as ODP the 50 or 100 points that go with it, is that for ODP itself or will a site get more bonus points if Google spiders the category listings from ODP that are included on all the sites that use the data?

Would it make sense to create a page that links to all the places a site is listed around the web due to its ODP listing?

[tmw]

great! startup ur here
really ur explanation about linkpopul was good.thankz
only to provide info in this forum i pasted ur valuable post.
thankz again..
bye

[ashkenaz]

Please excuse the naivete, but how does Google (or anyone else) know that the link comes from a university professor? Are these links from .edu domains? This is of real interest to me, since I am a university professor and my website, though entirely an individual endeavor, is closely enough related to my teaching and research activities to merit a link from, e.g., an appropriate online syllabus. While I'd never compromise my academic integrity, does this thread mean that a link to my personal site from my university site would help my Google ranking? Or am I misunderstanding something? Many thanks in advance.

[chiyo]

that's an interesting point ashkanez.We have a lot of university sites like harvard and stanford pointing to us and Google gives us 75% of our SE referrals.

I have a feeling that many favour certain domains for referrals maybe not all edu domains or edu.au edu.sg for example.

If Google was using all edu domains they would end up with a lot of students pages which may or may not be "authoritative".

[paynt]

I suggest whether you are the professor or you're hoping a professor will list you on a link page that the link in question lead to an on topic-on theme site.

To answer your question ashkenaz it's a combination of things I suppose. First, professors are nicely linked into a huge site (the university) that with ease establishes itself as an authority of education, knowledge, etc. Then, so often professors and students (that doesn't matter) create these terrific web link pages or project pages with resources that are completely on theme. I think it's also the .edu domain because I see libraries that create link resources for certain subjects also receiving high rank and link weight. The important part of this though is the fact that these resources stay on task and are a part of a larger authority.

One that I ran into recently
[uwsp.edu...]

Now there’s a list to be on if you have a site about dog behavior, care or welling-being. It would be even better yet if they would have created a separate page yet to link to each dog related theme.

[chiyo]

thats a great response paynt and explains the google academic bent.

Your right, academic discipline does mean these guys are probably intuitive themers! They dont even have to think about it, and posisbly also review the sites they link to with more care.

Also Google started off as a uni project at stanford and possibly trialed using university pages. Certainly google picked up our academic content a long time before they became a big name.

[paynt]

chiyo I agree it probably did start out with the university connection. The library list connection is another great one I found recently, coming on stronger all the time. Who better to theme than professors and libraians.

[littleman]

Great thread.

Xoc at the bottom of the LS robot text you'll find a wildcard:
User-agent: *
Disallow: /
That probably would turn googlebot away.

Hotbot has zero restrictions
[hotbot.lycos.com...]

[Brett_Tabke]

They id what is research area and what is students area by the thrid level domain. Also, most student pages will have "users" or "~" in the url (url words of death on google).

[oLeon]

littleman,
that´s at the bottom of the robots.txt from Looksmart:

User-agent: Googlebot
Disallow:

User-agent: METAGOPHER
Disallow:

User-agent: *
Disallow: /

I agree with XOC, there´s no reason for Googlebot staying away.

[Brett_Tabke]

It is up to the agent to interp whether the wildcard applies to them as well. Reread the spec, I've not seen a definative answer. I think by rights, the agents should read that bottom entry even if mentioned elsewhere.

[NFFC]

There is certainly something going on with Looksmart & Google:

Pages found for Looksmart = 3,610 [google.com] compare with Yahoo = 2,280,000 [google.com].

Perhaps they have been banned for cloaking, look here [google.com]. ;)

[bobriggs]

Well I saw the Robots.txt in the title of this thread, but it started as Link points. Anyway, if this is off topic I apologize.

But I did notice the NBCi (Snap) posts.

Here is my site's robots.txt file:

User-agent: *
Disallow: /dealers/
Disallow: /cgi-bin/

The reason I'm giving this is that when I clicked "more results from this site" on google, it gave:

[mydomain.com...]  
www.mydomain.com/dealers/ 

as a listing.

It has since been deleted (from google's index) I checked my logs, and sure enough, it tried for that page, but received a 401 response header. (Authorization) I only noticed the listing after the site had first been listed in google's index.

The point is, that Snap's robots.txt excludes everything. But is that really going to keep google out?

And by the way, is my robots.txt formatted properly? I see a many with a blank line after the User-agents line.

[Brett_Tabke]

Ya, that is a fine format.

[grnidone]

> (Yahoo link, 100 points, ODP link 50 points, major authority hub link like a topic directory 5points, inbound link 2 points, insite link .50 points). Others don't believe insite links count - I do...

I wonder what the "points" on PDF files are going to be, now that Google can index those. I have three clients with pdf files referencing them.

-G

[jilla]

In terms of the 100 points from a Yahoo indexed page. Is any yahoo indexed page equal in weight to any other one indexed in yahoo or is there more weight given to a page at a top of a category heading on yahoo versus at bottom of the heap? I do notice that pages linked from yahoo sites do well in google but am wondering if certain yahoo pages count more than other ones.

Debbie

[Robert Charlton]

It's clear to me on Google that the Yahoo category affects rankings. I don't know enough to generalize, but, say, in the area "widget store" or "widget stores," I've seen a listing in Regional> Widget Dealers> Location> Directories boost a search for the plural "location widget stores" much more than a listing in Regional> Widget Dealers> Location... which will boost you for "location widget store" singular, but not as much for the plural.

Not quite a precise example, but it should give you the idea.

[Brett_Tabke]

No difference in listing location that I can detect jilla.

[jilla]

Thanks Robert and Brett. I did notice that sites I've linked to from a yahoo listed site that are unrelated to it in topic seem to do well. Robert,
I think you are saying where you are linked from does effect ranking though. Maybe it's that I use unpopular phrases and not very competitive ones. I find that anything I link to from a yahoo page does well (but also the algorythym is relevant).

I had a site that I linked to without a yahoo listed page (just used an open directory listing and looksmart indexed inbound links to it) which did worse though I used the same algo for the page.

Jilla