homepage Welcome to WebmasterWorld Guest from 174.129.163.183
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Spiders
What makes Googlebot come...
thehittmann




msg:50486
 5:46 pm on Jan 4, 2004 (gmt 0)

What makes SE spiders come to your site more often?

I know that incomming links from higher PR sites has a bit to do with it. But the fast spider comes to my site everyday a few times a day. If I make a change without a doubt the very next day I can see my change on alltheweb. Googlebot comes about one a month and even though I keep building my incomming links...this doesnt seem to be increasing.

 

AjiNIMC




msg:50487
 9:10 pm on Jan 4, 2004 (gmt 0)

Keep changing your contents, add the date e.tc this helps.

Aji

millie




msg:50488
 12:06 am on Jan 5, 2004 (gmt 0)

>... add the date e.tc this helps<

Could you clarify this for me ... why does adding the date help and where - ie, do you mean Last Modified or in the page content?

Thanks, M

vrtlw




msg:50489
 12:15 am on Jan 5, 2004 (gmt 0)

I think Aji means a date #include statement.

With SSI (.shtml) extension you could use <!--#echo var="DATE_LOCAL" -->

To take it a stage further you could use an include statement to insert some text that changes on a regular basis (like a latest info column).

dasboot




msg:50490
 12:16 am on Jan 5, 2004 (gmt 0)

High PR = more spidering from Google. So if you are already a huge, massive, rich company with a megasite - you're o.k.

Does that help?

thehittmann




msg:50491
 2:26 am on Jan 5, 2004 (gmt 0)

Thanks everyone. I think I'll add a date like you mentioned. My site is not for a company. It is a music website.

g1smd




msg:50492
 11:01 pm on Jan 12, 2004 (gmt 0)

Google usually shows the latest version of a site in the cache, but every few weeks it can revert to showing an older cached copy. Having an embedded date on the page can help you to determine what Google is up to more readily.

Worth doing.

allanp73




msg:50493
 11:47 pm on Jan 12, 2004 (gmt 0)

Has anyone noticed that Google isn't indexing as often? I have a new client with a pr3 site which I made major changes to about a week ago and still the new index page hasn't been indexed. I got desparate and even submitted it to Google. I never experienced such a slow indexing.

thehittmann




msg:50494
 5:43 am on Jan 13, 2004 (gmt 0)

I have a medium size site (30 pages) that gets indexed about once a month, I have a tiny little blog that gets indexed roughly every 3 days. The pr of my blog is 1 but it gets spidered and indexed more often than my other which is PR 4 and climbing.

g1smd




msg:50495
 8:30 pm on Jan 14, 2004 (gmt 0)

Google has seen that the blog is updated more often than the rest of the site, that's why it keeps coming back to it.

trimmer80




msg:50496
 8:42 pm on Jan 14, 2004 (gmt 0)

anybody know how the bot tells when the page is updated. E.g. does it look at the file properties for last modified or compare content on the page with past content. The reason I ask this is I have sites that are dynamic and the content on all pages are updated via the database. Thus the file itself doesn't change but the content does.

MarkHutch




msg:50497
 8:45 pm on Jan 14, 2004 (gmt 0)

I think most go by the file date and time on the server. Last mod I believe.

trimmer80




msg:50498
 8:56 pm on Jan 14, 2004 (gmt 0)

if this is the case, i'm better off finding a small element that is in every page of my code, e.g. <HTML>, doing a find and replace on this to something unique <ABCDEFG> and then another find and replace back to <HTML>. Thus changing the last modified on all files in site.

Stefan




msg:50499
 3:59 am on Jan 15, 2004 (gmt 0)

Crawling frequency is very much related to PR.
Making regular changes to pages, and adding internal links to new good content pages, might help as well... it sure doesn't hurt.

sidyadav




msg:50500
 5:34 am on Jan 15, 2004 (gmt 0)

yep, you should get crawled often if:
  • you are a PR4 or higher.
  • you have a link from a PR4 or higher page, but you yourself don't have a PR4 or higher.
  • Google has calculated your update frequency and indexes your site very rapidly even though you don't have a high PR or have a link from one. It usually helps if you add the date at top of your page, something like the php date() [nz2.php.net] function.

    Sid

  • trimmer80




    msg:50501
     8:40 pm on Jan 15, 2004 (gmt 0)

    a php date function will not change the last modified date. Does this mean you believe that it compares new content to old and not the last modified?

    thehittmann




    msg:50502
     1:06 am on Jan 16, 2004 (gmt 0)

    if the content on the page has changed it will see it as being modified even if the server doesnt support the last modified header. But most do anyway.

    Global Options:
     top home search open messages active posts  
     

    Home / Forums Index / Google / Google News Archive
    rss feed

    All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
    Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
    © Webmaster World 1996-2014 all rights reserved