|Understanding Dominic - Part 2|
| 12:38 am on May 16, 2003 (gmt 0)|
Earlier this week, albert [webmasterworld.com] started a thread called “Understanding Dominc [webmasterworld.com].”
In that thread, he compiled a list of all of GoogleGuy’s recent comments concerning update dominic. The excerpts were a big help for many of our members. However, the following discussion quickly ended up much like the various threads that albert originally had to dig through in order to compile his list.
Trying to wade through all the various update threads has clearly caused some frustration [webmasterworld.com] for many of our members. So in order to try and alleviate some of that frustration, I thought we might try a new version of albert’s original post. This version includes all of albert’s original excerpts. It will also be updated periodically. However, we are going to keep this thread “read only.”
Anyone interested in following GoogleGuy’s contributions to the current update discussions can bookmark or flag this thread. If/when we post updates, this thread will appear on the active list.
If you have any questions or comments relating to anything in this thread, I would greatly appreciate it if you post them in one of our current update threads.
Update dominic – Part 9 [webmasterworld.com]
Serious Google update analysis thread [webmasterworld.com]
SJ started testing a new index with a sightly different build of backlinks.
GoogleGuy msg #44 May 5
|rfgdxm1, every index has to pass a really stringent battery of tests before it becomes visible. SEOs might notice a slightly different build of backlinks, but things like that could be balanced by factors that improve search quality more in other areas. The other thing to bear in mind is that it's easy to re-sync something like backlinks or spam snapshots once you're convinced that an algorithm or method is an improvement. |
SJ results will show up at other data centers soon
GoogleGuy msg #107 May 5
|Critter, it wouldn't surprise me to see SJ results start to show up at other data centers soon. |
Test of new method with a known base of backlinks, bringing in more up-to-date backlink and spam info later on
GoogleGuy msg #134 May 5
|Traveler, good question. From the first few posts of that 500+ thread, several people mentioned that they have some very new results in SJ. It's natural that we would test new methods by using a known base of backlinks, but that shouldn't be discouraging to people--backlinks are the sort of data that Google could bring back in over a relatively short time frame. And the same thing goes for known snapshots of spam--that can be brought in fairly quickly as well. SEOs notice whether a backlink comes from two months ago or one month ago, but typical users would care more about fresher pages. |
SJ index is not old
GoogleGuy msg #146 May 5
|Critter, the SJ index isn't an older index. You can verify that by doing a topical query such as SARS. The results are more fresh in SJ than they are in our regular index. |
About backlinks from forums
GoogleGuy msg #160 May 5
|Critter, if it's the site listed in your profile, it looks like you only have 5-6 domains that link to your site. A few of those are forum links that might not have made it into the base of backlinks. Getting links from places like the Open Directory Project would help, for example. |
About guestbook links
GoogleGuy msg 165 May 5
|Much more likely that those guestbook links just aren't given weight now, rfgdxm1. |
Backlinks and spam snapshots will be added later
GoogleGuy msg 178 May 5
|mcavic, I think I did say that newer backlinks and spam snapshots would be pending to be applied over time. Or at least I tried to. :) |
SJ data will show up at other data centers first, new data / filters after that
GoogleGuy msg #298 May 5
|albert, what you said, except I wouldn't be surprised to see SJ show up at other data centers first, and then to start applying the newer data/filters after that. |
Less Backlinks for all sites
GoogleGuy msg #108 May 6
|[i]Don't be alarmed if the number of reported backlinks goes down. That's actually to be expected in the update. Most of it affects all sites uniformly, so it comes out in the wash as being equal. The better way to measure it is how your rankings/traffic change. |
SJ results will show up at other data centers
GoogleGuy msg #43 May 6
|What rfgdxm1 said. I think you'll see SJ results appear at more data centers over time. |
GoogleGuy msg #11 May 8
|I'm still hanging around. There's not that much new info to convey, but I'm here. |
A few more backlinks were added
GoogleGuy msg #197 May 10
|I think we added a few more backlinks in yesterday. I'm assuming people have read HitProf's thread on backlinks too? rfgdxm1, sorry to hear that you don't like the SJ index. I also checked your ingredient theory in your spam report. People had suggested that a long time ago at the GooglePlex, but that's not the primary addition for SJ. |
It was only a minor update (so far)
GoogleGuy msg 203 May 10
|Twas a minor update in backlinks, MyWifeSays. I still expect SJ results to be seen at more data centers first. |
sj/fi index will shift to other data centers. After more backlinks and other data
GoogleGuy msg #52 May 14
|webdev, I think almost all of these questions have been answered several hundred threads ago. As late as this morning, I posted saying that I expected sj/fi index data to make its way to other data centers and to various sites. Once that data appears more broadly, we'll gradually be pulling in more backlinks and applying other data. |
sj/fi data centers have been approved
GoogleGuy msg #59 May 14
|steve128, the sj/fi data centers have been tested and approved. What I said several hundred posts ago was that you can expect an index like that to show up at more (and possibly all) data centers in the future. |
More pages and backlinks to be added
GoogleGuy msg #73 May 14
|Ltribe, I expect more pages and backlinks to be brought in with time. |
Help spread the word ...
GoogleGuy msg#86 May 14
|Maybe I'll just set things up to auto-post every 50 posts or so. :) |
Please help spread the word so people know what to expect and don't worry too much.
SJ and/or FI - and other data centers
GoogleGuy msg #377 May 14
|I wouldn't draw huge distinctions between sj and fi. When I say "I expect the sj index to spread to other data centers," that could be sj or fi. |
SJ and FI similar, but emphasizing different things like topicality - same with different data centers
Google Guy in this thread msg #16 May 14
|There's a lot of backlinks on the web. :) It will take some time to bring them all in. To clarify a couple things: |
- it helps to think of sj and fi as similar. It's better conceptually to think of them as cut from the same cloth.
- chiyo, every data center has different machine characteristics. So similar/identical indices might look slightly different at different data centers. This goes back to the point above. Don't think of it as if we build a different index with a different theme for different customers. Our partners get the same scoring/data that we use. That said, the global index that we build can emphasize different things more, such as topicality or more diverse file types.
So sj/fi are different in several ways. I would expect that difference to spread to other data centers. Then things will resume moving forward.
Hope that helps,
Different nature of SJ and FI is expected to spread to more data centers
Google Guy in this thread msg #28 May 14
|trillianjedi, I'm saying that sj and fi are of a different nature than previous indices, and that I expect that different nature to spread to more data centers (ex, anyone? :) |
I've been using "sj" to denote this different nature, but I appreciate the chance to clarify.
Measuring the size/usefulness of an index
GoogleGuy msg 4 May 14
|There are lots of different ways to measure the size/usefulness of an index. Nice job to Allergic for noticing something that most people usually don't. :) |
Timeframe to bring in backlinks - and: should we watch sj/fi
GoogleGuy msg 43 May 14
|x_m, you could always do that yourself by just not querying sj/fi. :) |
trillianjedi, just to clarify on the other point, I mentioned that backlinks could be brought in on a relatively short timeframe, but remember that we are talking about terabytes of data here--the web is a big chunk of data. I think I also replied to someone else at one point that bringing in those backlinks wasn't the sort of thing that could happen in a day or two. Hope that sheds more light on things
| 9:45 pm on May 17, 2003 (gmt 0)|
Backlink reduction due to improved estimation of link counts
GoogleGuy msg #7 May 9
|HitProf, I really like your insights. I think our newer systems do a much better job of estimating link counts. |
-dc entered the game
GoogleGuy msg #2 May 17
|Good deal. Sounds like things are on schedule. |
Recent indexing changes - Google gives no guarantee for static search results
GoogleGuy msg #3 May 17
|Hollywood, we never promise that we're going to return the same static set of results for any length of time for any query. The only thing we promise is that we're going to try to return the best results that we can. I would not base your business deals on the assumption that Google search results will remain static for long periods of time. That's just not something that we promise. |
Bringing in backlinks is no overnight thing
GoogleGuy msg #2 May 17
|Bringing in backlinks and other data will be a gradual process over time. It won't be an overnight thing once all the data centers have the sj-type index. Once that index is everywhere, I'm looking forward to us bringing in fresher data--but I want to set expectations that it will take some time. |
Timeframe is more than days, less than months. - Calm down, and remember last september
GoogleGuy msg #16 May 17
|I would say more than days, less than months. That's just my personal take. A lot of people are paying attention to every microdetail at this point. I would say that stepping back a level of detail would give better insight and less stress. Suppose you're on a long bus trip. If you scrutinize the road for every bump, pebble, or sharp turn, you're going to be more stressed than you need to be. If people are newer to WebmasterWorld, I'd recommend going back to last September, when we improved our scoring. If you re-read those threads, you'll see the same sorts of reactions that you see now. People claimed the sky was falling. You saw 5-10 people making pretty alarmist claims as loudly as possible. People suggested that scoring changes were a secret attempt to boost AdWords sales. The imminent destruction of Google was predicted several times. A few personal attacks on GoogleGuy took place. :) |
We got through that, and re-reading those threads will give you a different perspective on these more recent threads. Take that, plus the fact that I've said 5-6 (9-10?) times that we'll be bringing in better filters and fresher data, and most people should feel better, I hope.
We still use the scoring improvements from last September, but you don't hear people worrying about it now. I'm looking forward to doing a similar post for our next major improvement, whenever that comes: "You know, if you go back to May, you'll see several webmasters were worried about that change too. Would you believe that there were over 4000+ posts from people who were anxious about that change? ..."
Anyway, each webmaster is free to do whatever they want, of course. I do think that people would feel less stressed if they took a step (or two) back though. I think you've seen that somewhat; many of the senior members of WebmasterWorld haven't been doing tons of posts in the dominic discussions.
Spam showing up right now will be short-lived
GoogleGuy msg #24 May 17
|tigger, sometimes weekends are easier for making it over here. :) |
Hey nutsandbolts, let's try something new. If you have feedback about this index, do a spam report with your nickname, mention webmasterworld, but also include "dominic" in the comments. If people have general feedback or specific examples about this index, that's the best way to get the examples to us. I'm not worried about spammers that were banned and are back for a brief time--that will be short-lived. But if you have comments about searches that seem better/worse (preferably not just searches for your own site), send a spam report with "dominic" somewhere in the comments. We'll read what people have to say when they can give specific searches.
[edited by: Marcia at 3:39 pm (utc) on May 19, 2003]
[edit reason] Modified nesting for formatting. [/edit]
| 3:35 pm on May 19, 2003 (gmt 0)|
Look at the overall picture. - Do you show well at sj/cw/fi and the like? - Spam will disappear again
GoogleGuy msg 39 May 17
|Kirby, I'm not worried about spammers that have been handled before and appear to be back. Most of those will be gone again. But I do want to hear spam reports like "this type of search seems to work better or worse with the new system." |
worker, if I were playing Yoda like Alphawolf suggests, I'd say: worry less about PR on the toolbar and more about rankings.
And less about rankings for high-profile phrases and more about overall rankings. And less about rankings and more about traffic. And less about traffic and more about conversions. Maybe that doesn't sound much like Yoda though. I think the shorter answer would be that if you're showing well at datacenters like sj/cw/fi, I wouldn't worry much about what the PR says.
I just wanted to say thanks to the other posters who are taking a step back and looking at the overall picture. Please help to remind people that the longer-term view will keep folks less stressed and more productive. :)
About micro-level views, larger landscapes, and a post showing deep understanding
GoogleGuy msg #54 May 17
|teeceo, when bringing up a new system, you want to work from a known base of data. I fully expect that after that, we'll be working to bring in newer sites. And your anchortext, rfgdxm1. ;) FWIW, BigDave has (in my experience reading his posts--I don't know who he is) a deep understanding of Google's workings and perspective. I would give his comments as much consideration as you would mine. |
mrdch, I understand why webmasters may be anxious--mainly my post was meant to bring out a few points that people may have missed. WebmasterWorld is great for the micro-level view ("I dropped from #4 to #16, but only in cw!"), but not always for the larger landscape.
On-page factors: Well optimized individual pages bring traffic from different important key phrases
GoogleGuy msg #78 May 17 (referring to msg #71)
|annej, I really wish every webmaster would do the log analysis that you just did. :) |
Still a lot of work left to do
GoogleGuy msg #109 May 18
|Stefan, there was a lot of work at Google behind these changes. We're trying the make the transition as gentle as possible, but there's still a lot of work left to do. |
Specific feedback about improved or worsened searches welcome
Google Guy msg #181 May 18
(Remark: quote shortened)
|This is just my take, but I don't think it helps anybody to have folks call other people dancing monkeys, or post claiming "B£*LLSH!T" in all caps, or virtually jump out of windows. If people have constructive comments for this index, I gave a method back in msg #24 of giving us specific searches or types of searches that you consider good or bad. I just checked, and I don't mind telling you that so far it's a single digit number of reports. I'm guessing that number will go up at least some after this reminder post :) but if you have specific searches or suggestions to pass on, that's probably the best way to get them to us. |
Current status of update
GoogleGuy msg #205 May 18
(Remark: read accurately, also posts referred to)
|Sure, Anon27 and deanril. I think the plan will be |
1) deploy the new index/system across all data centers
2) begin pulling in more data (i.e. newer backlinks, pages, and spam updates)
3) once new data is into the system, begin pulling in new algorithms that have been waiting in the wings
I believe the current status is that we're around step 1.5 or thereabouts; something like 7 or so data centers have the new index/system. I expect the current pace of switching data centers to continue about as it has been. I would expect step two to occur over roughly the same timeframe as a typical index cycle (thus the "more than weeks, less than months" comment).
Step 3 is longer-term and ongoing, but I'm really excited about what we'll be able to do to improve quality across the board.
Hope that helps,
(Staza, that should help with your questions: yes, and I believe so to #1 and #2.)
[edited by: Marcia at 3:44 pm (utc) on May 19, 2003]
[edit reason] Formatting of nested elements. [/edit]
| 8:31 pm on May 21, 2003 (gmt 0)|
About hidden text
GoogleGuy msg #22 May 19
|Thanks for the feedback. If people want to mention specific searches/sites, they can do a spam report and put "dominic" in the comments somewhere--and hopefully mention your nickname. |
Just today, I've fielded questions from two sites that had hidden text and were taken out for 30 days. One site owner knew that they had had hidden text and one didn't. If you had hidden text on your pages and you think that might be the problem, you'll want to double-check that all the hidden text is gone. The first set of hidden-text penalties will be expiring over the next week or two. Sounds like that probably isn't an issue, but it's something to bear in mind.
Don't gun for #1 at "that major keyword"
GoogleGuy msg #36 May 20
|paynt, feel free to use that. I think I'm paraphrasing something Danny Sullivan would say when I talk like that. :) Just the idea that paying more attention to users, what they're finding and what they want, rather than gunning for #1 on that "major keyword" is one thing that demonstrates a wise SEO. :) |
TLD / language, give specific comments to Google
GoogleGuy msg #31 May 20
|Hey Napoleon, I think I've already commented on our SJ index at length, and given people some idea of what to expect. I believe the change for non-U.S. IP addresses is geared to allow users to separate language from TLD, and I believe that it was tested for several weeks on one TLD, with us watching for feedback. Given that it's a UI change on the home page, the amount of negative feedback has been very low--a good sign. I know that the UI team is interested in the best way to present international users with a choice of which TLD to use and which language to use. If you write to firstname.lastname@example.org with some subject line such as "UI for TLD/language," I'm sure it will reach open ears on the UI team--they want feedback on how to make things smoother for everyone. I'll be curious to read suggestions for how to make that UI better myself. As far as things like expired domains, changes such as SJ are exactly the sort of thing that will allow us to pull in better algorithms that can take full advantage of that data. |
Anyway, I hope that addresses some of your questions. We're always looking for feedback on what we can do better. In another thread, I gave a way to give specific feedback to Google about the SJ index: do a spam report with "dominic" in the comments. I've since reminded people about that method a couple times. Yet the number of reports via that form has been less than even the number of posts on this thread. After 3000+ posts about this index on WebmasterWorld, we're sitting at around 30 concrete suggestions about what's bad/good about the SJ index. By that measure, SJ is definitely an improvement over past indices.
More changes to come
GoogleGuy msg #42 May 20 re msg #37 (shortened)
|NovaW, I would certainly expect to more changes like the ones you mentioned, which is a good thing. |
links=PR0 like red herring
GoogleGuy msg #45 May 20
|Napoleon, the links = PR0 sounds like a red herring, but I'll be happy to root around and read it. :) |
Keep on doing what's right
GoogleGuy msg #11 May 20
|Oh well. People knock on Google sometimes. If you just keep doing what you know is right, things seem to work out just fine. :) |
About link pages
GoogleGuy msg 158 May 20
|Sorry, that's what I was trying to say earlier. :) Links pages don't get PR0, and we index them. |
Funny one about link farms
GoogleGuy msg 161 May 20
|They're mostly located in the Midwest, I think. Lots of irrigation and sunshine to raise those little links. The links need time to mature, or else they aren't as good. Once the links are harvested, they're shipped throughout the world to different companies that make HTML editors, and those companies can feed the links right in. |
If you drive across the U.S., you can still see link farmers from time to time.
Links will be 'organic'
GoogleGuy msg #170 May 20
|walthamstow, just remember that you want all-organic links. :) |
| 9:28 pm on May 23, 2003 (gmt 0)|
Phase 2 Begins
Status: index is at all data centers. Subtle spam filter comes soon. Fresh data coming in next days. Timeframe: more than weeks, less than months.
GoogleGuy msg #53 May 23
|Some fresh data might be incorporated next-day, but I wouldn't expect everything freshbot found yesterday to make it in a day. There's a lot of data that needs to be fetched and cross-checked--I would expect the full data to show up more on the timeframe of what you would expect from a crawl/index. Step 1 is done (index is at all data centers). I know that one subtle spam filter is going in soon, but Napoleon, I would start counting on the weeks-but-not-months comment beginning from the time that the index switchover finished. Again, just trying to give webmasters iinformation so they have the right expectations: more than weeks, less than months. |
But I'm glad that people have noticed that freshie has been crawling deeply in addition to normal freshie duties.
Timeframe again. And changes expected for country language / TLD / redirection next week.
GoogleGuy msg #58 May 23
|I think we're on the same wavelength, Napoleon. |
P.S. About the questions you raised about country language/tld/redirection, I think we've got some changes scheduled soon (next week) that should bring it in line with what many users expect again. Just wanted to let you know that that was coming too.
Traditional update(s) expected for a little while longer. After, things will be more gradual.
GoogleGuy msg #65 May 23
|crobb305, some things will be filtering in sooner (I know a few more autospam filters will happen earlier), and I wouldn't be surprised to see fluctuations in backlinks and pages, but I would hold on to the idea of an update that brings in more data for a little while longer. In time, I do think things will be more gradual. However, we're still in the transition period for this system, so I wouldn't be surprised to see a traditional update for a little while longer. Hope that helps. |
| 10:42 pm on May 23, 2003 (gmt 0)|
At least another traditional update expected
GoogleGuy msg #81 May 23
|Sure, reneewood. I would expect at least another update of the form where the crawl/index cycle finishes and then data centers are updated in the traditional dance. |
Next update: what will/should be brought in. Freshbot looks like Deepbot.
GoogleGuy msg #88 May 23
|dnbjason, I've learned never to make promises, but I would say the odds are good. :) rfgdxm1, more anchor text, backlinks, spam filters, etc. should be brought in by the next update. I think someone else remarked that freshbot is looking more like deepbot these days. |
Definitely expect newer data affecting ranking
GoogleGuy msg #91 May 23
|I would definitely expect newer data that we bring in to affect rankings. |
| 10:30 pm on May 25, 2003 (gmt 0)|
About toolbar, status of index, and backlinks
GoogleGuy msg #43 May 23
|MurphyDog and notsleepy, I wouldn't worry. I've seen at least a couple grey bars for well-known sites, so I'm asking around about that. I wouldn't worry about the toolbar display during this transition. shaoye, the index includes many new pages and new backlinks, but also includes an older snapshot of backlinks for now as well. |
What exactly should happen with next update
GoogleGuy msg #102 May 23
|Let me clarify. The next update should bring in more backlinks, data, etc. |
Those dang prepositions. :)
Fluctuations before next update
GoogleGuy msg #109 May 23
|rfgdxm1, I wouldn't go as far as saying that SERPs aren't going to change at all until the next update. Some amount of fluctuations might occur before then (e.g. a couple of small spam filters, or some fresher data). |
Older data for cross-verification and worst case backup
GoogleGuy msg #113 May 23
|TheAutarch, that data can be used for cross-verification. It's also there as a worst-case back-up, but I don't think we'll need to use it. |
Some new data such as spam filters come before next index, larger set of data comes with next index
GoogleGuy msg 120 May 23
|jojojo, some data will be brought in, such as spam filters. A larger set of data will be brought in with the next index. |
| 8:05 pm on Jun 10, 2003 (gmt 0)|
Thanks for the killer post WebG.