Forum Moderators: open
I'm programmer first and a reluctant webmaster second. I'm used to having to find workarounds for bugs in Windows. Complaining to Microsoft is truly a wast of time.
It seems to me that there has been much discussion lately about panalties arising from hidden text - even to the point of pages being removed from the Google index. Since I don't use hidden text, I cannot comment from my own experience, but I can look at this issue objectively.
There are three possible policies a search engine can take when encountering hidden text/links, etc.
1: Take no special actions.
2: Ignore all such text, etc. Quite literally, pretend it isn't there.
3: Apply some sort of penalty.
Now lets look at how/why hidden text might be created.
1: As a deliberate attempt to spam search engines.
2: By accident. (Probably applies mostly to zero-length links).
3: Deliberately but without any intention to spam. For instance, a Webmaster might leave himself a todo list in white text in an empty table cell.
Now lets look at how a spammer might fool a search engine.
1: Use very bright gray text against white, etc. (Presumably search engines that worry about hidden text implement thresholds).
2: Use javascript to change background/text colors. (I imagine this can be done with tables but I've never tried it.)
Let's be realistic about this. There is not a snowball's chance in hell of a search engine being able to analyse all the javascript on a page to see if it is being used to hide text. It therefore follows that a spammer will be able to fool search engines using hidden text with no difficulty at all.
It therefore follows that applying penalties where hidden text is found is a total waste of time (if the intention is to combat spam). This just leaves two sensible policies for hidden text.
1: Take no special actions.
2: Ignore all such text, etc. Quite literally, pretend it isn't there.
Now none of this is rocket science. Indeed, if the boffins at Google can't work this out for themselves, they must be suffering from a combined IQ of a retarded chimpanzee.
So, let's see what GG has to say. Do Google boffins shuffle around on their knuckles? Judging from the mess Google is in right now it's easy to laugh and just say yes but the answer in reality is probably no. Nevertheless, given that it is child's play to use hidden text in a manner that can't be detected by search engines, I think it behoves GoogleGuy to say definitively what the policy is. After all, as I have explained, penalties are more likely to catch the innocent than the spammer.
widgets blue widgets red widgets lumpy widgets etc etc,
which generally doesnt make much sense.
i have recently edited my initial post for clarification, which my have coincided with your posting
I asked for an informed opinion.It is informed I asked someone at Google.
Congratulations on making me smile!
I've just had lunch but here's something for others to chew on.
Suppose I create a frameset with fixed sizes. Suppose I then deliberately make sure that the text exceeds the boundaries of the frame. Voila, it's hidden.
Now Google's more advanced algos might have no great difficulty spotting this but would they actually do so? Also bear in mind it would depend on the choice of frameset page which is entirely independent of content page.
I hear people saying, well, you'd deserve to be punished wouldn't you. Perhaps I would, if it were deliberate. However, an accidental mistake such as this could go unnoticed on a web page, quite literally for years. The content of the page may be excellent and popular with visitors and users who, when they arrive, do not feel cheated. Yet, Google, in their infinite wisdom would like to deprive their users of the right (sorry, I should have said 'privilege' not 'right') to find the website.
Does that seem sensible to people?
Of course, Google's algos are not obliged to treat such accidents as deliberate, but if they do not, the use of algos to detect hidden text to defeat spammers simply becomes a total wasted excercise.
As I have said before, I am a reluctant webmaster and no expert in website design. So if I can think of ways that a website might be banned for accidentally falling foul of Google's measures of perfection, I imagine there are many more.
Forget all my semi-legal arguments for a moment and simply consider these two questions.
Will the issue of hidden text have spammers worried?
MY OPINION : NO WAY
Will good sites end up being removed from the Google index for arbitrary reasons?
MY OPINION : ABSOLUTELY YES, we've already heard of cases where it has happened.
So are Google right to adopt this policy?
MY OPINION : Well, I think people know that already!
it doesn't render pages however... it doesnt get image files...
Will the issue of hidden text have spammers worried?
MY OPINION : NO WAYWill good sites end up being removed from the Google index for arbitrary reasons?
MY OPINION : ABSOLUTELY YES, we've already heard of cases where it has happened.So are Google right to adopt this policy?
MY OPINION : Well, I think people know that already!
I totally agree with you, kaled. Also my own case [webmasterworld.com] is a show case to it.
how does your argument fit in with the fact that a website using hidden text for the purpose of SEO may severely limit a blind persons ability to access it through a screen reader program, as the hidden text will make little or no sense whatsoever when read aloud?
I think you are a little confused.
I've been advocating the, perhaps, legitimate use of hidden text for keywords. Assuming that hidden text is ignored by screen reading software for blind people (which would be sensible but I don't know if it is true) then the page would make more sense by permitting the use of hidden text.
However, essentially, we are all missing the most important point here. If Google used the sort of page analysis strategies that I have advocated in earlier postings (and we are not talking rocket science or massive CPU usage) they would have absolutely no need to take arbitrary decisions thereby removing good sites from the index.
Google's action to ban hidden text using algos demonstrates that even huge corporations can make mind-numbingly stupid decisions.
I find it a little worrying that so many webmasters support Google in this but what worries me most, is that webmasters that agree with me are prepared to just quietly accept it.
kaled, if someone complains that hidden text is lowering the quality of our search results, don't you think we should take some action on that? For that matter, why would you think we couldn't detect hidden text put in by JavaScript? You said that there's a "snowball's chance in hell" of that happening? That's a pretty big assumption, wouldn't you say?
I appreciate your rhetoric (boffins at Google have the combined IQ of a retarded chimpanzee, huge corporation makes mind-numbingly stupid decisions, mentions of first amendment violations, etc. etc.). Personally though, I'm less inclined to hop into threads that have overheated rhetoric.
The chimpanzee comment was a quote from the first post of this thread. :) kaled said that Googlers "must be suffering from a combined IQ of a retarded chimpanzee." Which is one reason why I haven't bothered to post in this thread very much. I've got other things to do besides discuss statements like that. :)
Regards,
Brent
Assuming that hidden text is ignored by screen reading software for blind people (which would be sensible but I don't know if it is true) then the page would make more sense by permitting the use of hidden text.
Absolute, utter nonsense. If it's coherent content, clean standards-obeying HTML, it's useable by a person with sight and a blind person. ALT tags will help with images.
If as you say hidden text is impossible to solve as a problem, how is a software company that makes reader software for the blind going to stack up in resources to Google? People with sight are the ones "blind" to "hidden" text on a page, so true blind users end up getting assaulted with repetitive lists of hidden keywords (using the extreme example).
I find the concept that sighted person needs more help than a blind person to establish relevancy as a justification for hidden text is more than a little twisted and offensive.
Hidden text = direct appeals to GoogleGuy. They are both an attempt to elbow in front of the guy next to you. I am as happy that this forum is moderated to delete mikeH's post or quote the TOS as I am that Google is taking action on hidden text.
You misquote me. What I said was "Even if you have complete freedom of expression, it does not follow that anyone else is obliged to publicize what you say."
Please note that there is a substantive difference between "publish" and "publicize". That said, it's also true that no one else is obliged to publish what you say!
Your example of the F*** word etc. not being blocked if they are visible says to me that the only preference Google is expressing about our content is "get it out where people can see it." That is not at all the same thing as an editorial judgement about said content.
GG - I figured you gave up when he had the gall to post your confidential reply about his obvious hidden text tactic - then continued with an illogical argument based on flawed assumptions...
Regards,
Brent
If this refers to me posting a confidential reply, let me state categorically that no-one has replied to me confidentially on this or any other thread. Who's assumptions are flawed now? Also, even if my arguments are based on flawed assumptions, they are not illogical, just, apparently, beyond your comprehension. (Sizzle!)
Quote by GoogleGuy
kaled, if someone complains that hidden text is lowering the quality of our search results, don't you think we should take some action on that? For that matter, why would you think we couldn't detect hidden text put in by JavaScript? You said that there's a "snowball's chance in hell" of that happening? That's a pretty big assumption, wouldn't you say?
The actions you are taking will not improve search results in the long term. There may possibly be a short term effect and this might conceivably be beneficial, though I doubt it very much. In the long term however, there will be no improvement in results.
As for my chimpanzee metaphor, you certainly cannot say that I was not prepared to back up my position with reasoned argument. I have discussed this subject from a variety of angles. You have not.
As for my "Snowball's chance in hell" comment with regard to Google's ability to detect all hidden text by algos, I stand by it. Quite apart from the use of cloaking (a term I only discovered the meaning of the other day) there are very simple methods that Google would have enormous difficulties with if attempting to apply them to the whole index. Applying a CPU intensive scan when a spam report is processed is a different issue. Essentially, you are simply talking about a tool to assist manual handling of spam reports.
I have also outlined methods by which keyword spam could be handled in a far more sensible way but you have not commented on these.
For that matter, why would you think we couldn't detect hidden text put in by JavaScript?
You would not ADD hidden text using Javascript. What would be the point. Search engines ignore Javascript don't they? (Except for Google's magic hidden-text algos.)
No, you would use Javascript to hide the text. I'm not going to get into a huge discussion on the technicalities of how this might be done but I will supply a simple example.
1: Create page with black background and red text.
2: Set the font color of the text you wish to hide to white.
3: Write Javascript to change the background color to white when the page loads.
4: Having tested it, use the Javascript eval function and a little obfuscation to confuse Google algos.
Now, this simplest of examples could be handled by algos without huge difficulty but it would be very expensive in terms of CPU power. Some people have suggested that rendering the page could be used to solve this problem, but again, for reasons too complex to discuss here, that is a non-starter. (It would also be massively expensive in CPU power and you would have to employ OCR techniques to compare the rendered page with the expected output results).
In another thread (and possibly this one too) you state that I obviously made up my mind long ago. Well, if a couple of days is long ago, then I'm guilty as charged.
If I were to go over all my previous postings on this thread, I would probably count a dozen, perhaps more, well argued reasons why banning sites for hidden text is stupid. Unless I've missed something, you haven't made a serious attempt to answer a single one of these.
Your sole, and extremely lame explanation is that it will improve relevancy for users. As I have reiterated time and time again, this is nonsense. Spammers will get around your best algos but good sites, in many cases designed by part-time webmasters like myself, will in some cases be banned even though they may be perfectly good, useful sites with excellent content.
Explain this to me.
My most visited page is Z******. On current statistics, more than 50% of visitors download the software on this page. At the bottom of this page is a short list of keywords to assist the USER to find this page. Clearly from the download rate, visitors are very happy with what they find there. So why would Google wish to ban this page from their index if I choose, for whatever reason, to hide the keyword list. The page would be no less relevant to the user would it? Banning this page would not help your users would it?
Let me remind you (GG) once again, in previous posts, I have outlined keyword-relevancy strategies that could be easily deployed, would not require stupid penalties to be issued, would help users and would be INHERENTLY IMPOSSIBLE to spam but you haven't commented on these suggestions either.
Go back and take a look. My suggestions would mean that if a spammer filled a page with half-a-dozen keywords (repeated) Google could actually, give it a near zero relevancy for those keywords without having to treat it as any sort of special case.
At this point I'm sure that you are tempted to say something like, "We know all about page-content analysis and don't need help from an amateur like you." However, the simple fact is this, if Google were already using these sorts of strategies, there would be no reason to ban hidden text in order to improve relevancy. Indeed, no-one would have ever complained about hidden text in the first place.
There is only one possible reason to ban hidden text, and that is to prevent totally irrelevant sites using common search keywords to try to get their popup adverts in front of users. However, whilst I have experienced this problem on other search engines, I have not noticed it on Google, presumably because they stop it manually. Given all the other reasons for allowing hidden text, and that this problem is apparently under control at Google I can see no reason why hidden text should be banned. In any case, dodgy websites such as these simply need to use a Javascript redirect - they have no need to use hidden text at all.
On the issue as to whether the HT algorithm will improve Google's results, opinions differ. Google thinks so. "Freedom of speech" means that Google, just like anyone else, has the absolute right to give their opinion concrete expression, in the form of algorithms generating, for instance, estimations of site relevance. This issue has been argued in court, and free speech won.
On the absurd and ignorant assertion that Google is a monopoly, a few facts should suffice:
-- Of the four most-visited web portals, two have access to other search engines: MSN uses only Inktomi, and Yahoo OWNS Inktomi. So the customer count is 3 to 2 in favor of Google. That's not a monopoly in anybody's lexicon. (As a matter of fact, the ratio of searches is somewhere between 3/2 and 5/2, again nowhere close to a monopoly. The fact that more people CLICK on Google results is a result either of Google giving proportionately better results, or of more informed surfers freely choosing Google for deliberate searches -- which in turn argues that if Google were a monopoly (which it isn't) it would have been a natural monopoly.
-- There is a company that, according to the court, DOES have a monopoly (M$ and browsers), and in fact M$ does tie their browser to both their portal and their search results; which so far as I can see from previous court cases, is unquestionably illegal.
-- Google has no business relationship with webmasters at all. Therefore it is especially absurd to babble about how a monopoly (which Google isn't) chooses or treats its customers (which webmasters aren't).
-- As to Google deciding whether Buddhism is a religion, or Islam is a satanic cult, or Bill Gates is the most hated man on earth: to make THAT argument into even a straw man, you'd have to add the straw.
First, Google has no motive and no method for doing that: they aren't in the business of theming -- whereas there are abundant business reasons to check hidden text on sites returned by search results whose quality has been questioned.
Second, whether or not you or I liked their categorization of any particular "topic," Remember free speech? If Google wants to do that, then you can use another search engine, because Google still isn't a monopoly--whack the right eight keys on your keyboard, or do one mouse click in the right place, and you can escape the dreaded Google monopoly and enter the free and open world of msn.com.
Patrick
ps: I suspect at least one of my sites has been banned from Google recently
The arguments I have put forward would apply to any general-purpose search engine intent on banning hidden text. However, no-one would care if a search engine that no-one has ever heard of did this.
As for my having the right to use another search engine - from the webmaster's perspective, this is so trite as to not warrant a reply.
buckworks,
I'm sorry if you feel I misquoted you, and I apologise. However, in the context of the internet, it is my opinion that the difference between publish and publicize is a fine one.
For instance, if I were to write a book, the publisher would organise all the publicity as well as printing and distribution etc. However, if I choose to publish the book on the internet (i.e. users download it) then what part of the analogy should apply to the search engine? Is it the shop, the printer, the publicist?
Again, for misquoting/paraphrasing you, I apologise. It is most annoying when someone misquotes you, but take it from me, it is also very annoying when people misunderstand points you've tried hard to make as clear as crystal.
Regards,
Kaled.
PS
I am all in favour of things being transparent and above-board. Personally, I don't use hidden text, never have, unless, of course, you include certain meta tags - if those don't constitute hidden text, I don't know what does? I wonder if Google will ban these too. It would be fun watching the index vanish into a puff of e-smoke!
2. OCR is not required and in fact would be less effective than hooks in the rendering engine.
3. This software will make mistakes and it can be beaten. It will not be easy to beat for those who can't even comprehend how they are doing it. It will not be a common technique, because anything that becomes common will be accounted for. If you have the time and money to figure out how to beat it, you might just as well put your efforts into other spam areas that will remain easier for now.
4. The detection software does not have to run on the entire index. Most pages NEVER make it into the first 5 pages of the SERPs. Just running on the spam reports will have a huge impact. Once they are have extra testing cycles that are avilable after the spam reports, it is quite simple to find high probability areas to go after. They could start working their way down the first two pages of the SERPs in the most popular searches.
5. Accessibility guidelines do not support hidden text. Alt is the proper place to put information about your images.
6. Free results on a website is not and cannot ever be a monopoly under the definitions in Title 15. Other search engines might be able to file anti-trust claims against google because they win most of the contracts with other portals, but that is not a monopoly on searches, that is trade in *selling* those results. Google is incredibly far from having a monopoly in that area.
7. Google has first amendment protection on their results, just as you have first amendment protection on your site. Congress cannot pass any laws that restrict Google's rights to publish the results as they see fit. They would have to come up with a constitutional amendment. If people are bothered enough with this action to come up with a constitutional amendment, it isn't going to matter anyway, they will have stopped using google.
Amendment ICongress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.
In this case, google can even claim to be "the press" in that they publish their opinion of the importance of websites. They get extra protection for that.
8. Personal insults are just plain rude. When you toss those into your posts, and then put in direct pleas (that are against the TOS of this site) you do not deserve any sort of answer from the person that you insulted. And yes, your comments were a personal insult to GoogleGuy, as he is one of those "idiots" at google. You can try to make your point without getting insulting.
Indeed the thread is a hot topic, and as GoogleGuy said, the topic itself can be "overheated rhetoric", but IMO in your instance, on a scale of worriedness I don't think you should worry much.
If Google does lend creedence to a "hidden text algorithm", it's been suggested it would only be used on spam reports.
Regardless of how it's used, I'm sure it's well tested, and in the case of flash sites you'd think that Google would be well aware of the millions of flash sites that are out there.
I can't see Google implementing an algorithm like this and "forgetting" about the use of flash or other technologies that are legitimately used. It might be a fine line between what is "spammy hidden text" and "hidden text" but in the case of flash I'd hope you'd not fall into the category of spammy.
my 0.02 anyway.
There's been loads of discussions on the board about how flash sites could do with more text to rank better in Google....I can't remember hidden text being mentioned ;)
I can't comment on Google's algos, but I can certainly understand why you are worried.
This monolithic plot by Google to purify the web will undoubtedly create many innocent casualties.
I doubt GoogleGuy will come back with an assurance that your site will be ok, since he would be confirming a method by which text could be hidden. So for your own peace of mind, you should consider doing some restructuring so that text versions appear in search engines and flash versions are hidden. However, if you have a lot to do, I'd wait for the dust to settle before making big changes.
I thought this thread was was about to die a couple of days ago. If it keeps running and enough people whinge about the new policy, perhaps Google will have a change of heart. I've done my best to argue the case against the hidden-text penalty. If people continue to put forward erroneous (my opinion) arguments in favour of the HT penalty, I may reply, but this thread has already used up silly amounts of my time.
Regards,
Kaled.
1. Google does not use googlebot to run the hidden text check. This is done by a seperate piece of software that a) reads external image, CSS and JS files. b) is not required to abide by robots.txt because it is not a bot.
2. OCR is not required and in fact would be less effective than hooks in the rendering engine.3. This software will make mistakes and it can be beaten. It will not be easy to beat for those who can't even comprehend how they are doing it. It will not be a common technique, because anything that becomes common will be accounted for. If you have the time and money to figure out how to beat it, you might just as well put your efforts into other spam areas that will remain easier for now.
4. The detection software does not have to run on the entire index. Most pages NEVER make it into the first 5 pages of the SERPs. Just running on the spam reports will have a huge impact. Once they are have extra testing cycles that are avilable after the spam reports, it is quite simple to find high probability areas to go after. They could start working their way down the first two pages of the SERPs in the most popular searches.
1: Never thought otherwise, but I understand why you might believe I did.
2: Yes, You are probably right, but I've only had a limited time to ponder this. Bear in mind this, you could place hidden text at the bottom of the page and use advanced methods to hide it when it is scrolled into view. That would be tricky to detect.
3: If it makes mistakes and bans sites incorrectly, then users and webmasters suffer.
4: If you only run the detection software on high PR pages, some interesting results will follow.
Please bear in mind that I initiated this thread because OTHER PEOPLE believed they were losing money as a result of being banned unfairly. For the most part I have been extraordinarily polite. I am certain that people who are losing money as a result of this policy have thought far worse things than I have said.
Regarding my comments and the law, I was inviting expert opinion. It sounds like you have done some research.
I find it interesting that absolotely no-one has disparaged my suggestions for dealing with spam by using better keyword/page-content analysis. As I have said several times, if this were adjusted, there would be no need to apply penalties. Spamming by over-use of keywords could be made impossible. Hidden or visible, it would make no difference.
Just for the record, I do not have a problem with sites being banned that use hidden text to attract visitors that are looking for something else entirely. But the only way to do this fairly is to do it manually.
For the most part I have been extraordinarily polite.
?
Regards,
Brent
Thread closed.
On the issue as to whether spammers are worried, this thread pretty well proves that they are.
I'm not a spammer. I'm not aware of a single post in this thread by a spammer. Spammers have had a certain amount of advanced warning, and, as professionals, will be well ahead of the game.
If I were a spammer, frankly, I would relish the challenge of beating Google's best algos. I hope they are difficult beat - it'll just be more fun for those that enjoy that sort of thing.
I certainly believe in 5 years or so people will find this approach a bit strange.
I am new to the forum and have a question concerning all of this.
I am an amateur, I won't even try to skirt around that one. My website is pretty well listed on Inktomi related SE's. Just out of sheer curiosity I tried adding hidden keywords to the bottom of my home page (white on white) because I had seen similar high-ranking sites do the same thing.
Well, recently I submitted my site to Google and in the mean time started reading all of the bad press about hidden text. At, the time I didn't know it was forbidden.(Out of sheer stupidity.)
Just when I went to check my latest site visitors, I noticed that GoogleBot finally knocked on my door.
Host: 64.68.82.37 Agent: Googlebot/2.1 (+http://www.googlebot.com/bot.html)
This is the only one I have received so far. So I took head and immediately went and removed the hidden text from my homepage and re-uploaded it.
My question is, "Is it already too late?", or "Will they do a deep crawl before banning a website?"
I really want on Google.
Also, if I were to be banned before even being listed on Google, would I have to re-submit for them to index my site again or would they automatically do it after a month or so?
Thanks for any help you can provide. Lesson learned.
