Welcome to WebmasterWorld Guest from 34.229.24.100

Forum Moderators: open

Message Too Old, No Replies

New sites and Google

Put up two new sites, here's what happened

     
12:04 pm on Sep 12, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


Hello Googlehounds. We recently put up two new, small (<40 page) brochure style sites. The first was uploaded on Thursday the 4th, and the second on Friday the 5th.

I kept track of logs activity, fresh dates etc. as I was curious about how the new google would handle this. Here's what I found:



Some 'New Google' Data
Note: When uploaded, both sites were linked to from two pages (PR6 and PR5) of another site located on the same virtual host. These pages are regularily freshed. The log data was for the day it is listed under, therefore visits on the 5th should be reflected on the 6th, the 6th the 7th etc...

4th September - Thursday
Site A uploaded
Logs
Spidered Home page and robots txt

5th September - Friday
Site B uploaded
Logs
Site A: Spidered Home page and robots txt.
Site B: Spidered Home page and robots txt.
Serps
Site A index page shows up with Freshdates (5th)

6th September - Saturday
Logs
Site A: Spidered (deep crawl) robots txt.
Site B: Spidered (deep crawl) robots txt.
Serps
Site A and B index page shows up with Freshdates (6th)

7th September - Sunday
Logs
Site A: Spidered Home page and robots txt.
Site B: Spidered Home page and robots txt.
Serps
Both drop from the serps.

8th September - Monday
Logs
Site A: No Visits
Site B: No Visits
Serps
Site A and B index page shows up with Freshdates (8th)

9th September - Tuesday
Logs
Site A: Spidered Home page (2ce) and robots txt.
Site B: Spidered Home page and robots txt.
Serps
Fresh dates for 9th of September on index pages, all other pages show without freshdates.

10th September - Wednesday
Logs
Site A: No Visits
Site B: No Visits
Serps
Fresh dates for 9th of September on index pages persisting, other pages seem stable in the results.

11th September - Thursday
Logs
Site A: Spidered Home page and robots txt.
Site B: Spidered Home page and robots txt.
Serps
Fresh dates for 9th of September on index pages persisting, other pages seem stable in the results.

12th September - Friday
Logs
We'll See!
Serps
Fresh dates for 9th of September on index pages persisting, other pages seem stable in the results.



Extra Details
The two sites are now getting traffic from both Google and Yahoo!, and doing quite well for their keywords.

Toolbar
As soon as they were spidered they got whitebarred. First the indexpages and then the entire site (after the deep crawl on the 6th-early 7th the whole site showed a white bar but wasn't in the serps!). Looking forward to seeing what kind of PR they get.

Links
Although I did add some links in the middle there somewhere, it seems that the links from the pr5 and pr6 pages were key - these pages are regularily freshed - since the last PR update they haven't missed a freshdate. Other pr5 pages within the same site seem to only get freshdates once in a while - perhaps their '5' is a fraction less than that of the other '5' that gets consistent freshes.



Worth putting a link: A discussion on current spidering patterns observed in the 'new google' [webmasterworld.com]...

[edited by: mipapage at 2:16 pm (utc) on Sep. 12, 2003]

1:21 pm on Sept 12, 2003 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38250
votes: 111


What are the referrals?
1:32 pm on Sept 12, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


What are the referrals?

Can you be more specific, or explain a bit more? (I'm a greenish web designer)
Plua it's Friday and late afternoon here, I'm hungry, got a buzz going, and wrestling with some PHP.

[edited by: mipapage at 1:46 pm (utc) on Sep. 12, 2003]

1:39 pm on Sept 12, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member trillianjedi is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 15, 2003
posts:7256
votes: 3


Looking forward to seeing what kind of PR they get.

Both new home pages will be PR5. The PR6 inbound takes care of that.

These days, PR value in itself counts for less. Quantity of inbound links seems to make more difference, especially where anchor text is concerned (although I'm not discounting the value of quality links).

You'd probably be better off in the SERPS with 20 PR3 anchor text keyword links and a resultant, say, PR4, than the PR5 and PR6 link and resultant PR5.

Will be an interesting thread - thanks for posting your findings.

TJ

1:44 pm on Sept 12, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


You'd probably be better off in the SERPS with 20 PR3 anchor text keyword links and a resultant, say, PR4, than the PR5 and PR6 link and resultant PR5.

I couldn't agree more.

WRT PR, what I am more specifically interested in seeing is how deep the pr spreads - just for curiosities sake, really.

Will be an interesting thread - thanks for posting your findings.

Y'er welcome!
3:13 pm on Sept 12, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


I did similar at Sept 8: small site, just 15 pages. Linked to it from two PR 5 pages.

Homepage showed up in the index Sept 10. I'm still waiting for the other pages to show up.

A few weeks ago (experience with 2 other new sites, somewhat larger) other pages came up not exceeding two days after the homepage. Wonder why it takes longer this time ...

3:39 pm on Sept 12, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 16, 2003
posts:746
votes: 0


I have seen the same type of pattern when adding new sections to my site.

You'd probably be better off in the SERPS with 20 PR3 anchor text keyword links and a resultant, say, PR4, than the PR5 and PR6 link and resultant PR5.

Are you suggesting that the number of links is factored in addition to the resulting PR? Maybe the number of domains linking to your site is counted more heavily.
3:49 pm on Sept 12, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


Are you suggesting that the number of links is factored in addition to the resulting PR?

I think what was being suggested was that the more links you have, the more inbound anchor text you have.

especially where anchor text is concerned

I will understate the importance of inbound anchor text by saying that it is better than gold :}



Albert,

Wonder why it takes longer this time ...

In my logs (see 6th of Sept above) and judging from the thread that I linked to at the bottom of the first post, your site may not have been around for a deep crawl yet - someone posted today that they were being deep crawled, and the three week pattern shows deeper crawling patterns on weekends...

9:56 am on Sept 15, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


Fwiw - still stable in the serps, though the crawling pattern is, ummm, wacko, as has been mentioned in other threads [webmasterworld.com].

Brett - any answer to my Q on your Q (or can someone else enlighten me as to what he was after?)

10:09 am on Sept 15, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member marcia is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Sept 29, 2000
posts:12095
votes: 0


mipapage, I could be wrong but my take is that referrals would mean how's the traffic.

I've got a site that went "live" last weekend, showed up by Wednesday, then disappeared and has returned - only the homepage, no others in. White toolbar, but it's got a PR5, a PR4 and a PR3 link, based on experience and knowing where the links are I'd say it'll be a low end PR4 & interior pages could be PR3 starting out. That's based on not having additional links yet.

This and another site were slower to get the interior pages in - I've wondered whether PR and number of links has something to do with it. Just a speculation, I'm not watching too carefully right now.

I think that it does make a difference having links from a decent number of different sites.

10:30 am on Sept 15, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


Marcia,

thanks for the answer.

WRT traffic - it's been good, not great, but for the second site better than expected for now (trying out Brett's content plan, so time will tell). Amazing and eye opening to compare the search terms that I optimized for vs. the terms that bring traffic.

WRT internal pages getting listed, I think that I was lucky to get in just before what seemed to be a deep crawl - I'm not sure if it was PR of linking pages or just plain timing. I haven't seen a deep crawl like the one mentioned above in any of our sites (only 11) since that date.

Your site that disappeared then reappeared looks like what happened with mine, but without the deep crawl.

<added>Worth mentioning though that I did link to internal pages and not just to the root - I suppose this may have been a factor to getting internal pages listed, although google did hit only the root at first <added>

10:59 am on Sept 15, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


My site I mentioned in msg #6 has still only homepage in index, but no other page.

Site was spidered like this:
9 Sep 64.68.82.169 - robots and home
10 Sep 64.68.82.* - some deeper pages, again 11 Sep

12 to 15 Sep - only robots and home spidered.

SERPs show home with fresh tag 14 Sep.

Some traffic coming in already.

What do you think I can expect about my other pages showing up? Next major update?

BTW: has there been a sub-major update this weekend? I was hoping for my pages showing up ...

Until today I have just 1 inbound link from a PR 5 site (targetting homepage).

Would you think I can accelerate the process by putting some inbound links w/ KWs directly at my deeper pages?

Your comments are appreciated!

11:23 am on Sept 15, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


albert

I'm no wizard at this, but a few things:

  • We had a site that seemed to be penalized by the big G. After a note to Google, the home page of the site appeared, but internal pages didn't appear until I got some deep links.
  • I deep linked the two sites outlined above, and the internal pages are listed and were listed quickly (maybe due to deep linking, maybe luck wrt timing and the deep crawl).
  • the site has a site map, reachable from every page on the site (a link in an 'included' footer). I believe in site maps for search engines, and in the logs, it was hit early in the deep crawl.

I deep linked to both the home and internal pages of the two new sites from two pages of a differnt site, one page being PR6 the other PR5. The pr6 is in spanish and links to that language in the new sites, the pr5 is english and links to the english pages of the new sites. Point is, I think deep links are important.

I haven't seen a deep crawl in my logs since you posted.

1:01 pm on Sept 15, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


mipapage

thanks for your answer.

A sitemap exists already, and it was one of the first pages spidered.

I guess I will spend my site two deep links and see what happens.

1:21 pm on Sept 15, 2003 (gmt 0)

Junior Member

joined:Apr 30, 2003
posts:62
votes: 0


I put up a new site two or three weeks ago, linked in from two sites on same server, and got about 30 or so links from other sites.

Index is being repeat crawled but no deeper pages than that showing up yet.

I am finding it can take from a day or two to several weeks for new pages on my other older sites also.

I guess it cycles, and then gets around to it or doesn't.

1:23 pm on Sept 15, 2003 (gmt 0)

Junior Member

joined:Apr 30, 2003
posts:62
votes: 0


mipa...

yes, just submitted a deep link, I think it forces the issue with the robot which may get things going, we'll see!

5:48 pm on Sept 15, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:July 28, 2003
posts:925
votes: 0


mipapage, I think by referrals Brett meant the Googlebot IPs that spidered, this could be interesting to know.
8:50 pm on Sept 15, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 17, 2003
posts:169
votes: 0


Mipapage,

Have you had a deep crawl yet?

To get listed in Google, were your internal pages linked from other sites with high PR, or did you get links from a highly crawed page from within your site?


------------------------------------

"We had a site that seemed to be penalized by the big G. After a note to Google, the home page of the site appeared, but internal pages didn't appear until I got some deep links.

I deep linked the two sites outlined above, and the internal pages are listed and were listed quickly (maybe due to deep linking, maybe luck wrt timing and the deep crawl).

the site has a site map, reachable from every page on the site (a link in an 'included' footer). I believe in site maps for search engines, and in the logs, it was hit early in the deep crawl.

I deep linked to both the home and internal pages of the two new sites from two pages of a differnt site, one page being PR6 the other PR5. The pr6 is in spanish and links to that language in the new sites, the pr5 is english and links to the english pages of the new sites. Point is, I think deep links are important.

I haven't seen a deep crawl in my logs since you posted. "

9:49 pm on Sept 15, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


benc007,

Not sure what yer after there... There was a deep crawl on the 6th as outlined in the first post. In msg11 (together with 13) I mentioned that I linked to these sites from two pages within another site that had prs of 5 and 6 and were crawled regularily. Both new sites were whitebar pure of PR! Hope that answers your question...

WRT gbot IPīs, have to get those in the morning...

10:14 pm on Sept 15, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 11, 2003
posts:495
votes: 0


Point is, I think deep links are important

I said the same thing a couple times here. It encourages the bot to deep index your site quickly. If you continuously add and get links to new material, the bot will grow very attached to the site. Let the material grow old though, the indexing will slow down.

You probably won't see any PR until the end of this month or next. It doesn't take many links to get a decent PR. While some here go for massive quantity, I'll keep focusing on quality and will get the same or higher results with much less effort.

7:18 am on Sept 16, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


mipapage, I think by referrals Brett meant the Googlebot IPs that spidered, this could be interesting to know.

The bot IP's were all across the board, no rhyme, rythym or consistency that I could see other then that the were dominantly from 64.68.82 with only three visits from .86 and .87.

7:39 am on Sept 16, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


My site (see msg #6 and 12) is completely indexed this morning (UTC 8:35). Good SERPs for targetted KWs and phrases.

Only one inbound link to the home until yesterday. Yesterday evening I directed 3 inbound links to 3 deeper pages (but don't think that's the reason for showing up in the index this morning).

Since deep pages were spidered at Sep 10 and 11, would that mean that there was some sub-major update last night? Or is this pattern just normal?

Just curious.

11:33 am on Sept 16, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:July 28, 2003
posts:468
votes: 0


My site is but three months old and I got 'deep crawled' as well for the first time on the 13th - most every day since then. Crossing my fingers...

One question - how can you tell WHEN the page shown in the 'google cache' was taken? I ask because though my site's been spidered my pages aren't listed yet - if there is something wrong wondering what will happen.

2:39 pm on Sept 16, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


Josefu,

AFAIK there's no way to know when a page was cached, except the page is just fresh showing up - in that case the fresh tag in G's SERP shows the date.

HTH

8:32 pm on Sept 16, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:July 28, 2003
posts:468
votes: 0


Thanks for that albert, I can't find that date though : P

Google has visited yet again today! But...

[error] [client 64.68.88.145] File does not exist: /home/********/public_html/salon/boutique/images/select/select.html
[Tue Sep 16 08:13:59 2003] [error] [client 64.68.88.151] File does not exist: /home/********/public_html/salon/boutique/images/original/original.html
[Tue Sep 16 05:59:37 2003] [error] [client 64.68.88.7] File does not exist: /home/********/public_html/salon/images/gallery/gallery.html
[Tue Sep 16 03:15:05 2003] [error] [client 64.68.88.131] File does not exist: /home/********/public_html/salon/boutique/images/couture/couture.html

...I have no access from, and no links to, anything within any /image/ files. Perhaps this comment should be elsewhere, but this is what's happening. Should I start a new thread?

10:33 pm on Sept 16, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 25, 2003
posts:365
votes: 0


Josefu,

you might like to look around here:

[webmasterworld.com ]

esp this one: [webmasterworld.com ]

12:32 am on Sept 17, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 17, 2003
posts:169
votes: 0


"I'm no wizard at this, but a few things:
We had a site that seemed to be penalized by the big G. After a note to Google, the home page of the site appeared, but internal pages didn't appear until I got some deep links."

Mipa - I also have a site that seems to be penalized by G. It has zero backlinks and zero PR, and the site has never been in the SERPs. I thought maybe linking to all internal pages (about 1,000 pages) from the site map will help. If not, then maybe linking to the internal pages from another site with high PR.

Any suggestions?

7:31 am on Sept 17, 2003 (gmt 0)

Senior Member from ES 

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 11, 2003
posts:1381
votes: 0


benc007,

It has zero backlinks and zero PR, and the site has never been in the SERPs

How long has the site been up?

If not, then maybe linking to the internal pages from another site with high PR.

Linking deep is good, no matter where your site is (or isn't!) in Google.