Welcome to WebmasterWorld Guest from 54.147.63.124

Forum Moderators: open

Message Too Old, No Replies

GoogleBot chats?

GoogleBot triggered my online chat script.

     
8:19 am on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


I just had a weird scene. I use the online chat from HumanClick/LivePerson, and guess who just came in for a chat? Googlebot!

It didn't respond though, but it initiated the chat skript like a human visitor would do.

I typed in a few words to see if probably a human editor was behind all this, then I released the bot out of the chat.

The funny thing is that it needed to run a JavaScript application for this AND to accept cookies as well. Seems the bot is really picking up on scripts now...

10:59 am on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


normal GB or GB/Test? Would be really surprised. Do you have the IP?
2:27 pm on Apr 28, 2004 (gmt 0)

Administrator from US 

WebmasterWorld Administrator brett_tabke is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 21, 1999
posts:38048
votes: 12


human at the plex...they surf too.
3:47 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


I typed in a few words to see if probably a human editor was behind all this, then I released the bot out of the chat.

Guessed it's not a human being than ...

3:56 pm on Apr 28, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:July 28, 2003
posts:188
votes: 0


Doesn't that chat type have a "push" feature for sending pages to the visitor?

Next time think to start pushing all the pages you've been trying to get indexed.... :)

3:58 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 29, 2000
posts:1133
votes: 0


Well I hope by now the GBot has learned to speak English. I just cant stand those conversations in binary lol.
4:06 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Googlebot/Test - picked the robots.txt and then hit directly into the chat.

In theory, my chat program has a push-function, but it is only available at a higher payment-level so I have switched it off.

5:32 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


did you do a reverse-lookup on the IP to check if it's really Google?
Just imagine what happens if GB starts indexing chats ... I see the next way for Spamming :)
5:37 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 22, 2002
posts:1001
votes: 0


ThomasB - Spamming over chats? To late - Spimming is already established!
8:19 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


but not for better SE rankings I thought ... I'd call it chamming.
8:24 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Unfortunately I do NOT have the IP address. I wasn't fast enough since it only showed up in the chat window and I didn't do a screenshot - the chat window deletes visitors who have left the site pretty fast.

It resolved however to crawler14.google(bot?).com and started with 62 or 64. I know, that's not much help...

I don't think they are really trying to index chats. It seems rather that their experimental javascript-indexing still has some issues to resolve.

Maybe GoogleGuy can shed some light on this? (Was this enough to summon him? :-)

8:31 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


I guess it would be pretty hard to determine if it's a chat or not unless you watch out for flush's on the client site (is that possible?)

But having a good robots.txt is then more important then ever.

8:36 pm on Apr 28, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


You're absolutely right about the robots.txt - I wouldn't have thought about it. Blind spot so to say. I'm going to adjust it right away.
4:54 am on Apr 29, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Apr 16, 2004
posts:395
votes: 0


is having a "good" robots.txt important if you don't really care what it goes through or caches?

I mean, if you WANT things to be cached.. can a robots.txt help?

8:30 am on May 4, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Now it's MSN-BOT who triggered my chat skript!

IP: 204.95.98.253

Still not fast enough the get the whole browser identification string.

3:24 pm on May 4, 2004 (gmt 0)

Senior Member from MY 

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 1, 2003
posts:4847
votes: 0


are you sure there are no direct links to an internal page of your chat app? left by you or someone else? maybe the chat app has provision for non javascript-enabled browsers? (try an old browser / lynx on it - can you reach chat?)
8:30 pm on May 4, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


The problem is that the chat itself is hosted by an external company (LivePerson, previously HumanClick). I found that - also in the GoogleBot example before - the chat is DIRECTLY triggered and not a page visited before. So maybe the bot is browsing the HumanClick/Liverperson site instead?

The chat on my page is initiated like this:


<a href='http://hc2.humanclick.com/hc/1234567/?cmd=file&amp;file=visitorWantsToChat
&site=1234567&byhref=1' target='chat1234567' onClick="javascript:window.open('http://hc2.humanclick.com/hc/1234567/?cmd=file&file=visitorWantsToChat&site=1234567
&referrer='+document.location,'chat1234567',
'width=472,height=320');return false;">
<img src="chat-icon-here"></a>

Where '1234567' is my chat-ID with them.

Is that any help?

[edited by: Marcia at 10:33 am (utc) on May 7, 2004]
[edit reason] Side scrolling. [/edit]

8:42 pm on May 4, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
posts:1562
votes: 0


looks like a normal page. No reason not to follow it except if it's robots.txt prohibited imho
9:47 am on May 7, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Got it again - but this time I was better prepared:

msnbot/0.11 (+http://search.msn.com/msnbot.htm)
204.95.98.253

How can I exclude the chat in robots.txt, since it is not in my page tree at all but hosted externally?

12:08 pm on May 7, 2004 (gmt 0)

Senior Member from MY 

WebmasterWorld Senior Member vincevincevince is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 1, 2003
posts:4847
votes: 0


Yes, your chat command has both the <A HREF and the OnClick
i.e. it will open it in a javascript window, or failing that, it will follow in the normal way

that's what google uses

if you remove HREF="...." from the link, then it will still work for users who have javascript empowered browsers, but it won't be followed by google

failing that, set the NO FOLLOW metatag in your page...

or, if you don't want google not to follow anything from that page...

put your link into a small <IFRAME > on its own small page, with the NO FOLLOW on the IFRAMEd page, not your main page. or even, write a page that your link opens, and that page REDIRECTs to your chat script, with a NO FOLLOW meta tag

1:32 pm on May 7, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Hmm... I guess I try the HREF-deletion first. As far as I can see it was never used by non-Java-visitors before.

Since the tag is automatically on EVERY page, the NOFOLLOW is not an option.

I'm not familair with Iframes at all, but I guess I can find a tutorial somewhere?

1:50 pm on May 12, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


MSNBot is now a daily visitor in my chat since I haven't found time to change the link yet.

What is surprising (and totally unrelated to this thread) is that the frequency with which MSN visits my site has significantly increased over the last few weeks.

Are they deep-spidering the net to get food for their own search?

2:12 pm on May 13, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Weeeeellll.... I changed the code now, took the "href" portion out and it still works. Now let's wait for the next visit of one of them bots...

The problem is however, that the cursor doesn't turn into a hand anymore when I slide it over the trigger image.

Any help?

2:20 pm on May 13, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 8, 2003
posts:548
votes: 0


Try adding style="cursor:pointer;" to the tag.
2:33 pm on May 13, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Actually adding href="" helped as well. Any advice against it?
11:56 am on May 14, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 8, 2003
posts:548
votes: 0


Maybe href="" doesn't work on all browsers. It shouldn't harm to do both, href="" and style="...".
6:10 pm on May 19, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Sept 11, 2002
posts:2024
votes: 0


Well... I'm slightly puzzled. MSN-Bot still chats, even though I removed all of the misleading href-tags.

IP: 204.95.98.253
ID: msnbot/0.11