Forum Moderators: open

Message Too Old, No Replies

Google Bots

Doing their rounds

         

smsmasters

4:52 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



Yesterday (20th feb 2003) there were around 30 google bots indexing my forum...they were there for about 2hours or more...

Does this mean that my website will be on the search engine, if so, how long will it take?

Thank you! :)

Jesse_Smith

5:11 pm on Feb 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You'll be listed for a few days and then it will dump your site.

freshBot: 64.68.82.* bah, listed for only a few days, short dinner date then it dumps you.
deepcrawler: 216.239.46.* Good bot, it likes you, listed until death do you depart. So don't make him mad or it will divorce your site. You can divorce the Googlebot by using your robot.txt file. It's much cheaper and faster than going to the courts.

taxpod

8:00 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



I think I disagree. Late month freshbots seem to have the potential for getting into the index. One month I had 50,000 page hit by deepcrawler and 20,000 pages hit by Miss Fresh. After the update I had 60,000 pages in the index.

I believe others have posted that there is not a 100% disconnect between Miss Deep and Miss Fresh. It only my humble opinion and smsmasters might only be in the index for a short time but I cannot explain some of what I have seen without concluding that Miss Fresh and Miss Deep interact at some level.

smsmasters

8:27 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



so..have i got a good chance of being submitted?

lazerzubb

9:03 pm on Feb 21, 2003 (gmt 0)

deft_spyder

9:06 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



sms,

being submitted just means your url is in que to be visited, possibly. actually having the freshbot or googlebot on your site means that they are definitely talking some of your pages in for possible inclusion. inclusion doesnt guarantee ranking though.

you are on the right track, so make sure the pages they are taking are optmized for the kw's youd like them to show up at.

smsmasters

9:21 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



can u pls check my forum at www.smsmasters.co.uk and check the source pls...can u tell me if it is suitable?

sorry if posting my site link is against the rules..

smsmasters

9:27 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



64.140.49.66 this ip has been visiting my forum's regularly...is this google?

Stefan

10:50 pm on Feb 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



smsmasters, it isn't Google. It's a spy looking for academic plagiarism apparently.

deft_spyder

11:16 pm on Feb 21, 2003 (gmt 0)

10+ Year Member



sms,

sorry, but i dont have the time to go through your site looking at the seo stuff, but i can tell you that all of what you need to start making your pages search engine friendly in right here on this site.

look up "bretts quick rank in 12 months" in the site search. thats where you want to start. tons of what ive gleaned for seo is from reading other peoples posts with similar issues and adapting them.

Jesse_Smith

2:13 am on Feb 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



:::look up "bretts quick rank in 12 months" in the site search. thats where you want to start.

This is it.

************

In another post Google as a Black Box Giacomo
proposed that we talk too much theory and not enough
application of it. So, lets skip the theory and get to
what I know works from time proven methods on
Google. I know the following system works 100% of the
time with Google to attain rankings across a wide range
of keywords. This is what I do with clients to build a
successful site and has worked every time. The level of
success will depend largely on the subject matter, it's
potential audience, and it's level of competition on the
net.

The following will build a successful site in 1 years time
via Google alone. It can be done faster if you are a
real go getter, or everyones favorite a self starter.

A) Prep work and begin building content. Long before
the domain name is settled on, start putting together
notes to build at least a 100 page site. That's just for
openers. That's 100 pages of real content, as opposed
to link pages, resource pages, about/copyright/tos...etc
eg: fluff pages.

B) Domain name:
Easily brandable. You want "google.com" and not
"mykeyword.com". Keyword domains are out - branding
and name recognition are in - big time in. The value of
keywords in a domain name have never been less to
se's. Learn the lesson of "goto.com" becomes
"Overture.com" and why they did it. It's one of the
most powerful gut check calls I've ever seen on the
internet. That took serious resolve and nerve to blow
away several years of branding. (that is a whole 'nother
article, but learn the lesson as it applies to all of us).

C) Site Design:
The simpler the better. Rule of thumb: text content
should out weight the html content. The pages should
validate and be usable in everything from Lynx to
leading edge browsers. eg: keep it close to html 3.2 if
you can. Spiders are not to the point they really like
eating html 4.0 and the mess that it can bring. Stay
away from heavy: flash, dom, java, java script. Go
external with scripting languages if you must have
them - there is little reason to have them that I can
see - they will rarely help a site and stand to hurt it
greatly due to many factors most people don't
appreciate (search engines distaste for js is just one of
them).
Arrange the site in a logical manner with directory
names hitting the top keywords you wish to hit.
You can also go the other route and just throw
everything in root (this is rather controversial, but it's
been producing good long term results across many
engines).
Don't clutter and don't spam your site with frivolous
links like "best viewed" or other counter like junk.
Keep it clean and professional to the best of your
ability.

Learn the lesson of Google itself - simple is retro cool -
simple is what surfers want.

Speed isn't everything, it's almost the only thing. Your
site should respond almost instantly to a request. If
you get into even 3-4 seconds delay until "something
happens" in the browser, you are in long term trouble.
That 3-4 seconds response time may vary for site
destined to live in other countries than your native one.
The site should respond locally within 3-4 seconds
(max) to any request. Longer than that, and you'll lose
10% of your audience for every second. That 10%
could be the difference between success and not.

The pages:

D) Page Size:
The smaller the better. Keep it under 15k if you can.
The smaller the better. Keep it under 12k if you can.
The smaller the better. Keep it under 10k if you can - I
trust you are getting the idea here. Over 5k and under
10k. Ya - that bites - it's tough to do, but it works. It
works for search engines, and it works for surfers.
Remember, 80% of your surfers will be at 56k or even
less.

E) Content:
Build one page of content and put online per day at
200-500 words. If you aren't sure what you need for
content, start with the Overture keyword suggester and
find the core set of keywords for your topic area. Those
are your subject starters.

F) Density, position, yada...
Simple old fashioned seo from the ground up.
Use the keyword once in title, once in description tag,
once in a heading, once in the url, once in bold, once
in italic, once high on the page, and hit the density
between 5 and 20% (don't fret about it). Use good
sentences and speel check it ;-) Spell checking is
becoming important as se's are moving to auto
correction during searches. There is no longer a reason
to look like you can't spell (unless you really are
phonetically challenged).

G) Outbound Links:
From every page, link to one or two high ranking sites
under that particular keyword. Use your keyword in the
link text (this is ultra important for the future).

H) Insite Cross links.
(cross links in this context are links WITHIN the same site)
Link to on topic quality content across your site. If a
page is about food, then make sure it links it to the
apples and veggies page. Specifically with Google, on
topic cross linking is very important for sharing your pr
value across your site. You do NOT want an "all star"
page that out performs the rest of your site. You want
50 pages that produce 1 referral each a day and do
NOT want 1 page that produces 50 referrals a day. If
you do find one page that drastically out produces the
rest of the site with Google, you need to off load some
of that pr value to other pages by cross linking heavily.
It's the old share the wealth thing.

I) Put it Online.
Don't go with virtual hosting - go with a stand alone ip.
Make sure the site is "crawlable" by a spider. All pages
should be linked to more than one other page on your
site, and not more than 2 levels deep from root. Link
the topic vertically as much as possible back to root. A
menu that is present on every page should link to your
sites main "topic index" pages (the doorways and
logical navigation system down into real content).
Don't put it online before you have a quality site to put
online. It's worse to put a "nothing" site online, than
no site at all. You want it flushed out from the start.

Go for a listing in the ODP. If you have the budget,
then submit to Looksmart and Yahoo. If you don't
have the budget, then try for a freebie on Yahoo (don't
hold your breath).

J) Submit
Submit the root to: Google, Fast, Altavista, WiseNut,
(write Teoma), DirectHit, and Hotbot. Now comes the
hard part - forget about submissions for the next six
months. That's right - submit and forget.

K) Logging and Tracking:
Get a quality logger/tracker that can do justice to
inbound referrals based on log files (don't use a lame
graphic counter - you need the real deal). If your host
doesn't support referrers, then back up and get a new
host. You can't run a modern site without full referrals
available 24x7x365 in real time.

L) Spiderlings:
Watch for spiders from se's. Make sure those that are
crawling the full site, can do so easily. If not, double
check your linking system (use standard hrefs) to
make sure the spider found it's way throughout the
site. Don't fret if it takes two spiderings to get your
whole site done by Google or Fast. Other se's are pot
luck and doubtful that you will be added at all if not
within 6 months.

M) Topic directories.
Almost every keyword sector has an authority hub on
it's topic. Go submit within the guidelines.

N) Links
Look around your keyword sector in Googles version of
the ODP. (this is best done AFTER getting an odp
listing - or two). Find sites that have links pages or
freely exchange links. Simply request a swap. Put a
page of on topic, in context links up your self as a
collection spot.
Don't freak if you can't get people to swap links - move
on. Try to swap links with one fresh site a day. A
simple personal email is enough. Stay low key about it
and don't worry if site Z won't link with you - they will -
eventually they will.

O) Content.
One page of quality content per day. Timely, topical
articles are always the best. Try to stay away from to
much "bloggin" type personal stuff and look more for
"article" topics that a general audience will like. Hone
your writing skills and read up on the right style of "web
speak" that tends to work with the fast and furious web
crowd.

Lots of text breaks - short sentences - lots of dashes -
something that reads quickly.

Most web users don't actually read, they scan. This is
why it is so important to keep low key pages today.
People see a huge overblown page by random, and a
portion of them will hit the back button before trying to
decipher it. They've got better things to do that waste
15 seconds (a stretch) at understanding your whiz bang
flash menu system. Because some big support site can
run flashed out motorhead pages, that is no indication
that you can. You don't have the pull factor they do.

Use headers, and bold standout text liberally on your
pages as logical separators. I call them scanner
stoppers where the eye will logically come to rest on
the page.

P) Gimmicks.
Stay far away from any "fades of the day" or anything
that appears spammy, unethical, or tricky. Plant
yourself firmly on the high ground in the middle of the
road.

Q) Link backs
When YOU receive requests for links, check the site out
before linking back with them. Check them through
Google and their pr value. Look for directory listings.
Don't link back to junk just because they asked. Make
sure it is a site similar to yours and on topic.

R) Rounding out the offerings:
Use options such as Email-a-friend, forums, and
mailing lists to round out your sites offerings. Hit the
top forums in your market and read, read, read until
your eyes hurt you read so much.
Stay away from "affiliate fades" that insert content on
to your site.

S) Beware of Flyer and Brochure Syndrome
If you have an ecom site or online version of bricks
and mortar, be careful not to turn your site into a
brochure. These don't work at all. Think about what
people want. They aren't coming to your site to view
"your content", they are coming to your site looking for
"their content". Talk as little about your products and
yourself as possible in articles (raise eyebrows...yes, I
know).

T) Build one page of content per day.
Head back to the Overture suggestion tool to get ideas
for fresh pages.

U) Study those logs.
After 30-60 days you will start to see a few referrals
from places you've gotten listed. Look for the keywords
people are using. See any bizarre combinations? Why
are people using those to find your site? If there is
something you have over looked, then build a page
around that topic. Retro engineer your site to feed the
search engine what it wants.
If your site is about "oranges", but your referrals are
all about "orange citrus fruit", then you can get busy
building articles around "citrus" and "fruit" instead of
the generic "oranges".
The search engines will tell you exactly what they want
to be fed - listen closely, there is gold in referral logs,
it's just a matter of panning for it.

V) Timely Topics
Nothing breeds success like success. Stay abreast of
developments in your keyword sector. If big site "Z" is
coming out with product "A" at the end of the year,
then build a page and have it ready in October so that
search engines get it by December. eg: go look at all
the Xbox and XP sites in Google right now - those are
sites that were on the ball last summer.

W) Friends and Family
Networking is critical to the success of a site. This is
where all that time you spend in forums will pay off.
pssst: Here's the catch-22 about forums: lurking is
almost useless. The value of a forum is in the
interaction with your fellow colleagues and cohorts. You
learn long term by the interaction - not by just reading.

Networking will pay off in link backs, tips, email
exchanges, and it will put you "in the loop" of your
keyword sector.

X) Notes, Notes, Notes
If you build one page per day, you will find that brain
storm like inspiration will hit you in the head at some
magic point. Whether it is in the shower (dry off first),
driving down the road (please pull over), or just parked
at your desk, write it down! 10 minutes of work later,
you will have forgotten all about that great idea you
just had. Write it down, and get detailed about what
you are thinking. When the inspirational juices are no
longer flowing, come back to those content ideas. It
sounds simple, but it's a life saver when the ideas stop
coming.

Y) Submission check at six months
Walk back through your submissions and see if you
got listed in all the search engines you submitted to
after six months. If not, then resubmit and forget
again. Try those freebie directories again too.

Z) Build one page of quality content per day.
Starting to see a theme here? Google loves content,
lots of quality content. Broad based over a wide range
of keywords. At the end of a years time, you should
have around 400 pages of content. That will get you
good placement under a wide range of keywords,
generate recip links, and overall position your site to
stand on it's own two feet.

Do those 26 things, and I guarantee you that in ones
years time you will call your site a success. It will be
drawing between 500 and 2000 referrals a day from
search engines. If you build a good site with an
average of 4 to 5 pages per user, you should be in the
10-15k page views per day range in one years time.
What you do with that traffic is up to you, but that is
more than enough to "do something" with.

stephenboyle

9:13 am on Feb 22, 2003 (gmt 0)

10+ Year Member



>From every page, link to one or two high
>ranking sites under that particular
>keyword. Use your keyword in the link
>text (this is ultra important for the
>future).

I wonder why this would be important (in the future)? That is, what is the rationale for this particular tip?

chiyo

10:10 am on Feb 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



stephenboyle. I agree strongly with 95% of brett's "guide", and the advice to link out to authoritatve sites from your keyword hyperlink is key, and quote perspicious back then.

Im seeing more evidence that our domains that do this are getting a leg up compared to our domains that tend to not link out.

Regardless of all the talk about google's algo, one principle i have always followed is that Google, in the end in its main database, conforms to the base architecture of the web as it was designed - in short to provide an interlinking set of documents that provide reputation for each other and navigation to what you are looking for.

We saw that first expressed in the importance of reputation and external links in, with the G algo, and we are increasingly seeing it, I feel, in a weighting given to sites that link out appropriately as well.

The Web is a web after all, not a collection of islands with one way bridges.

It makes absolute sense that sites should drift to the top the more interconnected they are with the rest of the web in their topic area/field.

This is related to another pet theory of mine that we, by focusing on domains (as is natural given our webmaster and often commercial goals) we forget that Google may well focus far less on domains, and more on pages - and see "pages" as the key elemnts in a web of thousands of related and interconnected pages, rather than domains.

stephenboyle

10:59 pm on Feb 22, 2003 (gmt 0)

10+ Year Member



Yes, and yet the logical conclusion if Google really does see 'pages' rather than domains, then it should not matter whether links internally or externally, as all merely pages. Thus I wonder what specific reason the advice is to link to external high rank pages, as in tip G (above) "this is ultra important for the future".

I am athinking - is the advice to do this in the hope the linkee will reciprocate? Or is it done so that people find the site more useful in a general sense of having links to other supposed valuable data (valued by pagerank?)
and are thus more likely to add a link to it, because they find it a useful site?
Or does Google actually weigh in on the equation ,and rate sites with external links higher somehow? Thus does Google actually see 'domains' as an important aspect?
And so on.
And why does the links best go to 'high ranking sites'? This is obviously some calculated idea about getting help for one's own site's rank, so I wonder what is the specific future orientated rationale/methodology.

I don't mind the vague idea that links (on every page?) to high ranking sites are going to be ultra important in the future, but I wonder what specific practical reasons are or could be given for it. If the web is nothing but pages, and one has lots of quality pages, do external links really matter?

If there are links that one naturally wishes to link to I would think put them in whether they are high ranking or not, but to feel obliged to put links to high ranking pages just for the sake of getting some rank oneself does seem to require a specific reason for its future effectiveness.