Forum Moderators: open

Message Too Old, No Replies

Will site page published 2003-03-20 be indexed by 2003-05-07?

How long does it take?

         

g1smd

2:31 am on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have a friend who has published a "bare bones" web page about an event that occurs on 2003-05-07. On that date, the page will have a live web cam, picture highlights, and all sorts of other information. At the moment, the page just contains text about what is going to happen. On the day, many people will be looking for information about this event, so it would be nice if it were indexed in Google. The question is, will this page, published today, be indexed in time, by 2003-05-07, or is it too late? How long does it take to get indexed? Many of the other pages of this site are already indexed for several years, so Google only has to notice the extra page and scan it.

jdMorgan

4:04 am on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



g1smd,

Well, it all depends on Google's update schedule. You've probably missed your chance to get deep-crawled for the next update, and so will get deep-crawled some time in early April. If Google's April update happens in late April, then you should be OK. As we've all just been reminded, though, sometimes they slip well into the next month.

You might also be picked up and remain well-visited by the freshbot if the page has high-enough PR and is updated frequently. In that case, the "fresh" listings might just float you through.

Only Google knows their schedule, so the true answer is, "No-one but Google knows." And actually, they might not know either, if the update schedule is determined by how long it takes their computers to calculate PR on the entire slice of the web that they index. In that case, the update might be scheduled "whenever the processing is done."

Based on the historic update schedule [webmasterworld.com], I'd bet $20 you'll make it. I wouldn't bet $200, though.

Good luck!
Jim

Powdork

6:23 am on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I agree with JD. Keep in mind the update must start prior to the third in order for it to propagate to the www servers in time. Also, from your 362 posts I'm guessing you know this, but you must have links pointing to your page from pages already in Google in order for it to make it in. As for freshbot visiting, that should depend on the pr of the sites linking to you since your friend's pages will not have pr until May.

Marcia

7:32 am on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



>>probably missed your chance to get deep-crawled for the next update

Jim, you're right, and I don't know if it'll stick, but Freshbot's been grabbing interior pages within /directories/ on a brand new site that doesn't even have any PR yet and probably won't for over a month. It's only a small site, but the crawling is going as deep as it gets. It's a big surprise!

The fresh listings were slipping in and out of www when looking earlier today, but for hours they've remained constant with www2. This will be a very exciting update, the one coming up <fingers crossed>.

g1smd

10:41 pm on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



>> from your 362 posts I'm guessing you know this <<

Actually, over 90% of my posts have been in either the HTML and Browsers or the New to Web Development or the ODP and Directories fora, very little in the Google forum.

I understand the basics of how Google works, but I work in telecomunications not SEO. The site in question is a hobbyist site, run by a friend. Apart from pointing the author to the W3C HTML validator, I have had very little involvement with the site itself. I had urged him (since Christmas!) to get a page on now, so that come the day it will already have been indexed.

HitProf

10:55 pm on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Good luck g1smd!

Make sure he gets some links from related or directory pages already listed in Google.

g1smd

6:23 pm on Mar 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have arranged three links to the site from other sites that are already indexed in Google, Yahoo, etc.

I hope that is enough links, and done early enough for the search engines to notice the new page.

I'll let you know on 2003 May 07.

freejung

6:43 pm on Mar 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I can tell you from personal experience that one link from a PR5 page is enough to get you thoroughly freshbotted. I think you may be able to get crawled with a PR4 link, but I wouldn't count on it for any links lower than that.

HitProf

7:57 pm on Mar 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Good work g1smd, le tus know when the site appears :)

g1smd

11:39 pm on Mar 22, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Thanks for the PR5 hint. I have arranged one of the links to be from a PR6 page on a hobbyist web site; at least I think it is PR6, squinting at the little bars in the Google Directory.

Using Mozilla 1.1, or Netscape 4.8 is there any other way to find out the PR of a page?

Powdork

10:11 am on Mar 23, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Use netscape 4.8 to search on Google for "How to download Internet Explorer":)
I am aware of no other way but there probably are some. Maybe the searchnerd site has some ideas.

CraigE

6:01 am on Mar 24, 2003 (gmt 0)



g1smd, good question.

Based on my log file analysis of a few new sites of mine, I have noticed that the time from when the spider visited the sites to the time I was indexed varied between 4-8 weeks. Hope this helps.

[edited by: Woz at 6:43 am (utc) on Mar. 24, 2003]
[edit reason] no Promos please. [/edit]

g1smd

8:26 pm on Mar 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I don't get to see the logs, neither does the site owner, as it is on a (small, UK-based, ISP and) free host. Just have to wait and see what happens. Hoping that the freshbot finds it soon. Now have five links to the page.

Powdork

9:56 pm on Mar 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hoping that the freshbot finds it soon. Now have five links to the page.

You may be in for a freshbot visit. Keep searching for your site on Google with searches that include text from your pages. Also search by url (leave off the www.) because sometimes Google can index the link to your page and know it exists without crawling and indexing the page. I think your a shoein to be crawled next time around but whether or not the April update will be moved to the regular Google servers by 5/7 is anybody's guess. The down side is that if you do get fresh crawled and the dance is still playing on the 7th, the fresh listings will probably not be given during the dance.

g1smd

10:35 pm on Mar 24, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Using one particular three word search term, that this site should be listed for, currently returns just three results, so it will be real easy to see when Google lists the site, as there should then be four results.

However, this three word search isn't quite what most people will be using on the day. They will probably use only one or two of the words, maybe combined with one or two other words (which are all covered by this page in the text) but which when used as search terms currently result in hundreds to thousands of site listings.

It will be interesting to see where this site falls when it gets indexed. It is unlikely to be anywhere near the top for the one word searches, but I'll keep looking at the results for the "easy" three word search term to see when the site appears; then I'll see how it does on single and two word searches.

4serendipity

2:40 am on Mar 25, 2003 (gmt 0)

10+ Year Member




I'd bet $20 you'll make it. I wouldn't bet $200, though.

I'll second this.

However, I'd start trying to attract greshbot in a big way. It sounds like you've secured enough linkage, but the pages will still need to be updated frequently (I'd do it once a day) to keep those fresh listings up.

Good Luck

Marcia

4:15 am on Mar 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



New site went up, one link from a page that was PR5 and just dropped down to PR4 - so it is possible from a 4 (that's on the high side of 4). It ended up with 12 pages indexed, now gone, and should be showing up the update after this coming one.

g1smd

11:22 pm on Mar 25, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



>> the pages will still need to be updated frequently (I'd do it once a day) to keep those fresh listings up. <<

Is it enough to run an FTP script that just re-uploads the page each day, or does the content also have to be changed in some way?


.

I love FTP scripts. You make a shortcut icon with a command line like:

C:\WINDOWS\FTP.EXE -s:c:\mydocu~1\scripts\upload-site01-files01.txt

You then make a text file, with the same name as you used in the above link. This file will hold the FTP instructions; inside that text file you put this:

open ftp.site.com
username
password

cd /webspace/folder
dir

send c:\mydocu~1\mywebsite\site01\folder\index.htm index.html
send c:\mydocu~1\mywebsite\site01\folder\page2.htm page2.html
dir

close

.

It usually takes a few goes to get it all working, but it is then a one click site update.

In the send instructions, the first filename is the location and name of the file on your machine, and the second filename is what you want to call the file when it is on the website. That's how you got old DOS systems to still have the web filename of .html on the site, even though the office computer didn't support long file names.

g1smd

8:41 pm on Mar 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Woooooooooooooooooooooooooooooooo. We are in! Surprised to have been found so quick.

The page is in, but it doesn't currently have a "Fresh" tag on it. Will it stick?

<edit>I don't know if the page was simply "found" by the spider, or whether they followed up on my submission of the site to Google, which I did yesterday evening. None of the other pages that I altered to point to the new page have been re-indexed as yet.</edit>

For my original "specific" three word search that listed only three sites in the results, there are now five listed, and we are number ONE.

For a common two word search that many people will be doing, we are number 14 out of 98 500 results. However, if you put the two words the other way around then the site is nowhere to be seen.

For another three word search, one that simply adds another word to the two word search above, we are number ONE again, out of 35 300 results.

If we use any of the three search words singly, but combined with the date of the event, then we are number ONE every single time, out of 92 or 237 or 3310 results.

Another three word search sees that page listed at position 39 out of 14 200 results.

A two word search, with the year "2003" added to the two words, sees that page listed as number 8 out of the listed 691 000 results.

For another three word search including one word I forgot that people might search with, we are not listed anywhere in the first 100 of the 7120 results. Quick site edit coming up to add another line of text including that word, which is currently not mentioned at all anywhere on the page.

The title that Google displays in the results is taken directly from the <title> tag on the site, and the Google description is taken from the meta description tag.

Now just need to keep the page stuck in Google, and try to improve some of the other words as well. That FTP script is going to run every night from now until 2003-05-07. Also going to look on Yahoo, and some other SE results, and see what they are doing.

<edit>Later: Have added another 40 word paragraph which includes the forgotten words, and another link to a useful site. Hmm, I have also discovered that the site is easily found for searches which include some other words and the date using these elements: 2003 05 07 or 2003 May 07, in any order, but the site is nowhere to be seen if the day number is typed as either a "7" or as the "7th". Now going back to fix that. Looking at the Yahoo results, they are almost as good as those from Google, first page for most searches again. What other major engines should I check?</edit>

freejung

12:02 am on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I doubt your listing is perminant at this point since we don't seem to have an update yet. Fresh tags were off for me this morning but the fresh version of my homepage was displayed in the SERPs. Now fresh tags are back for me, you might want to check them again. Since the update is soon, I wouldn't worry if I were you, you are probably seeing a preview of your post-update status via freshbot.

g1smd

6:41 am on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



>> I doubt your listing is permanent at this point <<

Hmm. Just checked. The site is currently nowhere to be found on Google using any of the search terms that worked 8 to 10 hours ago. Darn! However, the page has only been online for 6 days at this point. The site has dropped out of the Yahoo SERPs as well.

[edited by: g1smd at 6:51 am (utc) on Mar. 27, 2003]

GoogleGuy

6:47 am on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hmm, five days from initial post to it showing up in the index. Not bad. :)

My guess is that if the freshbot found you, you should be in good shape for the next full crawl. Not sure if you put the page up in time for our March index, but April is probably quite likely.

I try to never make promises (and I don't know what your site is) but the future looks bright for you..

g1smd

6:56 am on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



As the site will carry live images of an event that occurs on 2003-05-07, the only thing that is important is that it is findable from a few days before hand (say 2003-05-03 onwards) if possible, findable the day before and on the day of the event itself is top priority, and then still listed for at least a few days after the event. Being findable right now isn't a priority, as the page only has content saying what is going to happen, and of course, no images yet.

AhmedF

7:02 am on Mar 27, 2003 (gmt 0)

10+ Year Member



Just thought I would chip in.

One of the sites I own is a directory. The front page changes about 10 times a YEAR. The inner pages automatically update about 10 times a DAY

Nevertheless, it is a PR7 site, and the front page is always fresh-tagged :)

I believe it is PR more than anything else.

HitProf

2:24 pm on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



That's great news g1smd!

GG: if only G were so quick as to pick up changes on existing pages...

g1smd

8:16 pm on Mar 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmm. No sign of the site anywhere in the listings now. Nowhere to be seen. Even when searching just for the domain name it finds absolutely no records at all.

g1smd

9:35 pm on Mar 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Not listed anywhere yesterday.

Today, back in the listings with a 28 Mar 2003 tag next to the entry.

Rankings for dates including the 07 seem to have dropped one or two places on some searches, but after the editing a few days ago, the site is now findable for dates using 7 or 7th whereas before it was completely unlisted for these. All searches that had a first place listing are still first place. Now appear in the top 40 for most of the searches that we want to be found for, having added text including extra words a few days ago.

Today I have added some more text and have broken the page up with several <h2> and <h3> headings; will see what that does for us in a day or two.

The story so far: Looks like the site was found after 5 days (or my "Add URL" submission was followed up within 12 hours), then it was listed in Google for 12 hours, then it was 3 days unlisted, and now today over 12 hours listed, after freshbot revisited.

Will I have to keep on adding content to remain in the index, or will simply re-uploading the same files be enough? It looks like we are being listed when found by the freshbot. Does the content have to change, or just the file timestamp?. How long before we go in permanently? In a few days or so, or is it still a month away?

jady

10:01 pm on Mar 29, 2003 (gmt 0)

10+ Year Member



Marcia - same here, brand new 20 page site (with high PR links pointing to it) got deep crawled and is getting hit by freshbot. Funny thing is, even without a PR assigned, if you type for the site by name (Ex. Blue Widgets in USA) the site is found! Definately something I have never seen before on a new launch.. :)

jady

10:03 pm on Mar 29, 2003 (gmt 0)

10+ Year Member



To Add:

GG - is the "March Update" going to happen in April and "April update" in May?

Referring to March update we only have 2 days left...

BigDave

10:09 pm on Mar 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



g1smd,

You will be popping in and out, and moving around in rankings. That's why it's called everflux;)

Instead of trying to stuff everything into that one page, put together a couple of other pages for information surrounding the event.

Also try and get some more links coming in. The more you have, the more likely freshbot is going to keep finding you. And if you get those pages up and those links coming in before the deep crawl, the better positioned you will be in the April update if it happens in time.

GoogleGuy said:

I try to never make promises

An excellent policy anywhere in life. I always prefer to say "I'll try" than "I promise". If someone tries to pressure me into promising, I quickly change it to "I won't".

This 100 message thread spans 4 pages: 100