Forum Moderators: goodroi

Message Too Old, No Replies

Block specific words from being indexed

         

Jer1024

10:18 pm on Apr 11, 2005 (gmt 0)

10+ Year Member



I have a customer ( a lawyer ) who wants the words 'Deferred Adjudication' on his site, but doesn't want the words to show up on search engines if possible. ( He doesn't want that kind of business I guess )

Hey who am I to question his motives, I just try to accomidate.

I could of course replace the text with images, but that's a bit extreme. I have to do something though....

thx

Span

1:50 pm on Apr 12, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hi, I guess the only other way is to use JS:

<script type="text/javascript">
document.write("Deferred Adjudication");
</script>

sifredi

1:54 pm on Apr 12, 2005 (gmt 0)

10+ Year Member



I know it's against good SEO practice, but you can easily strip the documents from sensitive words and show an alternative page to the search engines. As long as the changes are small, they really shouldn't care.

Javascript can be seen by google, so that is still a bit risky.

Jer1024

4:10 pm on Apr 12, 2005 (gmt 0)

10+ Year Member



Thanks.

Hmmm...

I could the user agent to determine if it a bot.
( like in a log program )

Dim varAgent,strBots,arrBots,lenBots,chkBots,boolBots,strDefered

boolBots = False

varAgent = Request.ServerVariables("HTTP_USER_AGENT")
strBots = "bot,slurp,crawler,spider,archiver,grub,crawl,Crawler,Bot,Slurp,Spider,Downloader"

arrBots = Split(strBots,",")
lenBots = UBound(arrBots)

For i = 0 to lenBots
chkBots = instr(varAgent,arrBots(i))
If chkBots > 0 Then
boolBots = true
End If
Next

If boolBots = False Then
strDefered = "defered adjudiction"
Else
strDefered = ""
End If

Then on the page:

Blah Blah Blah <%= strDefered %> Blah Blah Blah

The Search Engines should never know the difference...

spoonyjo

6:42 pm on Apr 13, 2005 (gmt 0)

10+ Year Member



I have a similar issue (also with an attorney who does not want to come up under certain searches). My first thought was to use JS (since using images would be so clumsy) -- but since JS is visible to google that seems like a waste of time.

SEO isn't my thing (or I'm slow) so the solution outlined above doesn't help me out too much. Any advice on this or on how to implement the solution above?

Thanks-

Philosopher

7:12 pm on Apr 13, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Why not just break the words up by inserting a comment tag?

Defe<!--comment-->rred Adjudi<!--comment-->cation

Instead of using a comment tag, you could add some random text where I have the comment tag above and use css to set it's display to none.

Def<span style="display:none;">blablabla</span>erred Adjudi<span style="display:none;">blablabla</span>cation

Jer1024

12:54 am on Apr 14, 2005 (gmt 0)

10+ Year Member



The above description uses .asp ( server side VBSCRIPT ). The script is written, then is sent to the user as html.
( Which is why the client/robot can't see it as anything other than normal text)

For other people to use I can change the script so it catches any of the words in a list to make it easy.
Copy and paste the lines below

***** From Here
<%

Dim varAgent,strBots,strWordsList,arrBots,arrWordsList,lenBots,lenWordsList,chkBots,boolBots,j

boolBots = False

varAgent = Request.ServerVariables("HTTP_USER_AGENT")

' Words designed to catch a search robot
strBots = "bot,slurp,crawler,spider,archiver,grub,crawl,Crawler,Bot,Slurp,Spider,Downloader"

' Words you might use and want to remove
strWordsList = "legal phrase 1, legal phrase 2, legal phrase 3, legal phrase 4, legal phrase 5"

arrBots = Split(strBots,",")
lenBots = UBound(arrBots)

arrWordsList = Split(strWordsList,",")
lenWordsList = UBound(arrWordsList)

For i = 0 to lenBots
chkBots = instr(varAgent,arrBots(i))
If chkBots > 0 Then
boolBots = true
End If
Next

Sub Write_words()
If boolBots = False Then
Response.Write(arrWordsList(j))
j=j+1
Else
strDefered = ""
End If
End Sub
%>

<html>

This is <% Call Write_words() %> it should be followed by<br>
<% Call Write_words() %> which if this were a bot would not be seen. <br>
Oh yeah, This is <% Call Write_words() %> followed by<br>
<% Call Write_words() %> <br>

****** To Here

** Very important. Notice how I placed the phrases in order on the list as I would use them on the page. **

strWordsList = "A,B,C ..etc.. %>
I want to hide this <% A %> and I want to hide this <% B %> and I want to hide this <% C %> etc...

Also if you want to test this script replace:
varAgent = Request.ServerVariables("HTTP_USER_AGENT")
with:
varAgent = "IRLbot/1.0 (+http://irl.cs.tamu.edu/crawler)"

Just a bot that visited my sites USER_AGENT

larryhatch

6:24 am on Apr 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Hello Jer1024 & Philosopher:

I like the clean simplicity of Philo's solution, and suggest this little twist:

Def<!--inition-->erred Adju<!--stment in-->dication ..

Anyone looking might assume that the <!-- slipped in by accident,
that the words Definition, erred, adjustment, and indication were intended.

When rendered on-screen, 'Deferred Adjudication' appears of course.
I doubt any SE on Earth would bother unscrambling that,
or that there would be any penalty for doing so. - Larry

Jer1024

4:43 pm on Apr 14, 2005 (gmt 0)

10+ Year Member



****

never mind. I get it now...

Reid

10:15 am on Apr 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



from all these tricks and scripts semms like a whole lot harder to me than just inserting an image of the phrase.

I like the inserted comments one though.

sifredi

10:57 am on Apr 15, 2005 (gmt 0)

10+ Year Member



I would do it in php. Here is a simple way (the blue color is where the html document goes, and the red is the replacement array):
<?php

$html =<<<EHTML

<h1>HTML document</h1>
<p>Your html document goes here</p>

EHTML;

$rewrite = array(
"html"=>"xhtml",
"here"=>"there"

);

if (preg_match("#(google¦slurp@inktomi¦yahoo! slurp¦msnbot)#si", $_SERVER['HTTP_USER_AGENT']))
{
foreach($rewrite as $val=>$key)
{
if (preg_match("/$val/",$html))
{
$html = preg_replace("/$val/","$key",$html);
}
}
}

#############
echo $html;
#############

?>

Jer1024

12:06 pm on Apr 15, 2005 (gmt 0)

10+ Year Member



Inserting an image might be easier but, often it won't look right for one reason or another.

The comment idea is simple and should work. Although I thought SE's ignored commented script altogether.

Originally when I started making websites I used Flash quite alot and it exports a list of commented keywords based on what's on the .SWF.

For a long time our site was visited but SE's seemed to ignore the commented script. It wasn't until I placed the .SWF file on top of some hidden text with absolute positioning that I started seeing results.

Philosopher

7:34 pm on Apr 15, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The engines do ignore things found in comment tags, but it's the fact that the comment tag is there at all. It simply breaks the word into two segments. The engine sees both parts of the word as separate instead of one word.