Forum Moderators: open

Message Too Old, No Replies

Will site page published 2003-03-20 be indexed by 2003-05-07?

How long does it take?

         

g1smd

2:31 am on Mar 21, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I have a friend who has published a "bare bones" web page about an event that occurs on 2003-05-07. On that date, the page will have a live web cam, picture highlights, and all sorts of other information. At the moment, the page just contains text about what is going to happen. On the day, many people will be looking for information about this event, so it would be nice if it were indexed in Google. The question is, will this page, published today, be indexed in time, by 2003-05-07, or is it too late? How long does it take to get indexed? Many of the other pages of this site are already indexed for several years, so Google only has to notice the extra page and scan it.

g1smd

6:23 pm on Apr 26, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmm, back in again, the main site is fresh tagged as 24 Apr 2003 and is number 1 in most of the listings. However, the Google cache shows an old version. Roll on the update.

Also included in the SERPs is the page that was recently marked as robots: noindex,follow, which seems strange. I didn't know a lot, but I now know that I understand less than I did before.

Jesse_Smith

12:47 am on Apr 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm not g1smd, but I think I'll reply anyway!

Check your logs over the last 10 days. If you see 216.239.46 geting the files, then the Googlebot got your site in the last deepcrawl, and you are guarenteed to be listed at the May update. Though seeing that the deepcrawl ended today, you will be very lucky if the May update starts before the 7th. so you might want to hunt for any other sites that are about the event.

g1smd

11:18 am on Apr 27, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



.....

Site is still listed, but without fresh tags, I changed the content back to an older version of the site yesterday again, hoping that freshbot gets the message. The Google cache therefore shows a version with a lot more content, but not quite as full as the version I'll upload tomorrow.

The only mishap is in timing: in missing being deep-crawled for the last update, by about two days. The site has had to rely on freshbot to stay in, and it now looks like the next update is after the event.

Oh yeah, and the deepcrawl bot went through about 5 or 6 days ago.

g1smd

6:30 pm on Apr 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Still listed this morning without fresh tags, and uploaded a new version of the site at that time, This evening the site is fresh listed with 26 Apr 2003 tags again. Cache contains same content as it previously did, which is the same as the context now online, i.e. Google did not change the cache contents when it refreshed the site listing. It didn't cache what it saw on the 26th.

g1smd

6:12 pm on Apr 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Still listed, and without fresh tags this evening.

An oddity on one particular three word search combination, one that has had 9 results each time for the last week or more. Tonight it says "showing results 1 to 6 of about 14", shows the first 6 results, but then there is no additional or similar results link to be seen anywhere. So, where are the other 8 results?

g1smd

8:50 pm on Apr 30, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google listed on the 28th: 100 visitors.

Dropped from Google on the 29th, just a few hours after the last posting, but still 50 visitors for that day. Inbound links are getting better. Site listed on lots of other sites, added by people running those sites, who probably spotted the site via Google searches for similar sites to their own I assume.

Not listed in Google today, even though content was changed late last night and two days ago. Over 80 visitors today, mostly from one new inbound link; one that is already delivering more traffic than Google ever did (even though the site was #1 in Google for most of the last month).

Google will only show its worth when the topic becomes newsworthy and the General Public start searching for information.

g1smd

1:44 am on May 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Still not listed. Freshbot seems not to be interested in the changed content.

Jesse_Smith

5:52 am on May 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



g1smd, check your listing in the www-sj server now! It's doing testing and all pages deepcrawled in the last deepcrawl are listed there. Though I don't think the update will be before the 7th, unless there doing it a lot faster this time around.

g1smd

11:49 pm on May 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Still no listing on the main servers. Freshbot definately dozed off.

Thanks for the -sj tip. Site is definately listed there, but not doing too well. I can't tell exactly which date the -sj cache has stored, but think that 2003-04-14 is about right; for another site they have content from 2003-04-17 listed.

I did a lot of work to the site a few days after that. Darn. A lot of -sj searches result in a page pointing to the site being listed (and doing quite badly compared to the recent fresh results) rather than the site itself. That pointer page is currently marked Robots: noindex,follow, but Google must have visited before that was added.

A few days ago several keywords were giving a number one spot for the main site; but the -sj searches drop the site down to the 40s and 50s in some cases now. At least it does show up now; it has been nowhere for the previous three days.

Still getting a lot of visitors; have three very good inbound links (and a dozen or more not so good), two of which were added by their site owners without asking, and which are producing 80% of the hits. Not a single visitor via a search engine in the last three days (except me via -sj).

I need to get the freshbot back, and have a fresh listing May 6th and May 7th. Nothing else matters.

Jesse_Smith

12:07 am on May 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



:::I need to get the freshbot back, and have a fresh listing May 6th and May 7th. Nothing else matters.

Don't get your hopes up, looking at this thread [webmasterworld.com], it looks like the update will not start in time. Just to have the update finisted in time it would of had to start today, since it lasts five days. It might be a good idea to go to a ton of Space sites and try to get them to link to you, and pay Yahoo to list you.

g1smd

9:05 pm on May 4, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Still no listing on the main servers. Freshbot must have died. Still listed on -sj as before.

Just four of the inbound links listed on other sites are producing 90% of the traffic. These should generate a good number of hits on the day.

hitchhiker

9:17 pm on May 4, 2003 (gmt 0)

10+ Year Member



Just 40 mins to go (in CET)! (There better not be a loud bang or something, else we'll all know where to look)

:P

BTW. How many records does this thread hold?!

g1smd

11:10 pm on May 4, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmm. Still doing OK-ish but not great in -sj results, nowhere to be seen in -ex, www2 or www3 results, but now listed as number 1 on www but without fresh tags.

The results (on www only) are a rollback of some sort, as they are using an older title for the site, one that was only used in the first week or so after the site went live, and not used for at least the last three or four weeks.

For that one three-word phrase that has mostly had about 8 to 10 results for the last month (initially only 4 results before the last update) it is now showing Results 1 - 10 of about 34.

Ummm?

>> Just 40 mins to go (in CET)! <<

Eh? It's only 2003-May-04 here, and in less that an hour it will be 2003-May-05; still two whole days to go.

g1smd

12:08 am on May 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The number one result, and the 34 entries (rather than 8), has now spread to -va, -ab, and -dc, but these results are not in -fi, or -in. Those have the same results as last week. No idea what is going on.

hitchhiker

2:25 pm on May 5, 2003 (gmt 0)

10+ Year Member



OOps, read your date wrong.. (all back 2 front for me)
Can you say what this event is about? - curiosity, sticky me or something!

g1smd

11:26 pm on May 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Hmm, now showing in -in as well as www2 and www3, and these all have good results (and are without fresh tags).

Still not showing in -fi at all.

A good listing, and still without fresh tags, in www, -va, -ab, and -dc, as before.

Still not doing quite as well in -sj.

Nearly 250 hits today, very few of them from any search engine; over 90% of visitors arrived via just four very high quality links from other sites about the same topic.

g1smd

3:03 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Ummmmm. Google still listed the site on May 6th, but with vastly different results in the different datacentres. Most visitors that day, came from 3 or 4 high quality links from other topical websites, making 1119 visitors for the day. Oh my!

g1smd

3:11 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Sometime in the last few hours the site has gained a 05 May 2003 fresh date tag on www, -va, -in, and -ex (though these datacentres each have slightly different results).

The results on -fi, and -ab are both good, but these are without fresh tags.

The results on -sj, www2 and www3 are awful, and do not have fresh tags either.

The fresh listing will probably carry the site through the crucial period (over the next 18 hours) when the most hits are expected.

Hoorah!

GoogleGuy

4:04 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hope it works out well for you, g1smd!

g1smd

6:46 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Oooooooooooooo. Nearly 5000 hits so far today.

Wish I was earning even a dime for every one; but it is all free.

g1smd

8:45 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



At just past mid-event over 9000 hits; most of them from China, the US, and UK (in that order).

HitProf

10:02 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I enjoy your postings g1smd, keep going :)

It's noon where I am now, what's your time and date?

And why do you expext so many visitors exactly today?

<added>stupid question, answer is in your first post</added>

Good luck!

g1smd

11:43 am on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Event is all finished; over 12 000 visitors passed through and still they keep coming.

Time zone is UTC.

Stayed up all night finishing the last minute preparations. It all went quite well with the site updated from 2 laptops, networked together. A couple of glitches needing a reboot, and some further editing of images after the event as well.

The webcam images themselves were spread over 11 different domains, but only the one page of information was live before the event. The links to the other domains were only added at the very last minute (extra domains to spread the load). That ensured that all visitors passed through the advertised page, and heads were counted there. Also have a several MB size log file to look at as well.

Not sure how much data was served, but I'm guessing perhaps at least 4 GB in an 8 hour period (it's only a small site). Loads of fun, and we are all completely wiped out now.

Here are the site stats for the day: (the address of the site is not traceable from this Image [nedstatbasic.net] ).

In the end, four high quality links provided 35%, 20%, 10%, and 7% of the total visitors to the site, and various Google domains provided less than 3% of the direct visitors. However, being listed in Google on and off for the last four weeks directly led to those other sites finding our site and linking to it very early on. Those other sites were all top ten results in the SERPs. Without Google, this wouldn't have happened.

Visitors by country: China: 24% (that really surprised us!), UK: 19%, US: 11%, Germany, Poland, Netherlands: 4% each, Sweden, Belgium, Spain: 3% each, and so on. Must have seen hits from just about every country of the world, even from some of the most remote and under-developed.

Next Event: 2004 June 08.

HitProf

1:47 pm on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks for your reports g1smd.

I think I have a pretty good idea of what type of event this was :)
I have been watching live webcam, don't know if this was through your site.

Glad it worked out well.

takagi

2:05 pm on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Nice report g1smd, I almost daily read the progress (or lack of it) in this thread. I'm happy for you and your friends it worked out so well.

For quite some time I've curious about the site you've been writing about. But with the hints in your last message I could find it. Event is over but the counter is still going up (now 14,230).

g1smd

2:46 pm on May 7, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month




If this thread is to have a purpose other than a simple "Daily Diary" about the trials and tribulations of getting a listing in the search engines, then I think it should still be just some very simple sort of message. At the moment, the only words that come are mind are: "Build it and they will come".

In the end, Google did have us listed and in the top three or four positions for most of the keyword phrases that we wanted to be ranked for. I have no idea if luck played a part with the timing (see the threads in Google forum, about -sj and pre-update, etc), or whether GoogleGuy stirred the soup in some way. A week ago, it was looking like that we had dropped out, freshbot had died, and the next update would be several days after the event. In the end it all turned out good, and we were freshed right up yesterday.

What did we learn from this? Next time: start a month earlier. We allowed 6 weeks and were (almost) two days too late. Next time, it'll be done nearly three months ahead.

I now see that an official site for the same event that we were involved in, have reported that they were getting ten thousand hits per minute. Is that correct, or even possible? That makes our 15 000 hits in 12 hours look a bit small. However, we are not disappointed; anything over a few hundred would have been judged a success, as originally we were just going to pipe pictures out for a public meeting about 20 km away from here. The full "web cast" idea came much later. It was all hosted on free ISPs, and I haven't yet had any feedback as to whether people had any problems getting through, or whether it all worked out fine.

g1smd

6:38 pm on May 8, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Final visitor count for 2003-05-07 was 16556. Yow!

Jesse_Smith

9:19 pm on May 8, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Is that uniques, or page views, and in one or two days? You should of included affiliet text links to order products, like books, about what they were all looking at.

GoogleGuy

10:46 pm on May 8, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Congrats, g1smd! Glad that things went so well for your site!

g1smd

10:48 pm on May 8, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The 16556 figure was just for page views (I assume) and was from 2003-05-07 00:00 UTC until 2003-05-07 24:00 UTC. It is also for just the first page of the site, which was the page that was listed in Google, and the one that other sites also linked directly to. That domain held only a single page.

The visitor figure will naturally be a little on the low side, as some people will have clicked the link onwards to the next page of the site before the external page counter had time to be loaded.

Page 2 of the site was hosted on another domain, and I do have the log file for that site. It is quite a few MB. I guess there are some tools that can be used to analyse that one. Something for a rainy day, perhaps.

Page 2 of the site was only posted to the web and linked from the first page just a few minutes before the event started, so no-one will have arrived there from any other place. Page 2 just listed all the mirror sites that our images were being posted to; these mirrors went live a few hours before the event, but were hosted in subdirectories. These again would not have been found except by following links from Page 2. They all have Robots: noindex,follow tags, so that only the first page is listed.

We had no marketing links, nothing for sale, just images and information. It did cross our minds when the hit counter was going up (at one point) by over a hundred a minute, that it would have been nice to have a quid or a dollar off every visitor; but heh, that isn't what the site was about. Everything was done at two peoples own private expense. It was done for the fun and love of it; core internet values from the old days, nowadays nearly lost under the corporate and affiliate stampede.

<edit>GoogleGuy posted while I was writing. Thanks a lot. It went really well, though the weather intervened for just over an hour in the middle. We were freshed up, the day before and that carried us through. Was that a coincidence, or did you stir the soup in some way?</edit>

This 100 message thread spans 4 pages: 100