How does Blue Coat Systems Find My Private Site?

It's my own site for family & friends & Blue CS finds a new uploaded page!

[erlandc]

Hi,
So I'm wondering how they find a new page, let's say www.whatever.com/dk.html within a couple days!

i.p. is 199.19.249.196

how do they do that?

thanks!
e

[Samizdata]

Domain registrations are held in a publicly accessible database.

Some businesses make it their business to investigate.

...

[toplisek]

I have seen abuses of this also on my own.
How to defend and see also on my side into are held in a publicly accessible database?

[erlandc]

I'm not talking about domains, I'm talking about a new webpage that the co. in question that sees my new page. I've fed them a 403 but still it bothers me and wondering how they do it? How do they?

[jimbeetle]

Have you checked out the company? It offers proxy and gateway services and such to businesses. You might actually be feeding that 403 to a friend or family member who works for a company that uses its service.

[erlandc]

Hi Jim,

I did check out blue coat, and maybe you're right about a family or friend using it's service, I don't know.

Here's the creepy thing, when I create & upload a new page, I check my log files and my visitors check the new page over several minutes, whereas blue coat just get the HTTP/etc, and nothing else! What's with that?

[26/Jan/2011:04:30:19 -0800] "GET /bh.html HTTP/1.1" 403 4243 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322; MS-RTC LM 8; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

[edited by: mademetop at 9:08 pm (utc) on Jan 26, 2011]
[edit reason] IP address removed [/edit]

[SteveWh]

I had something very similar a few years ago. I'd create a new file on my server that no one could possibly know about, then look at it in my browser, and within a minute or two I'd see in cPanel > Latest Visitors that a mysterious IP had also fetched the file.

It turned out that my antivirus program was intercepting my browser requests and sending the URLs to the AV company. The company fetched the file from their own IPs (there were a bunch of them), and presumably checked it for viruses.

They still do it, but I've just gotten used to it.

If I really want a file to be inaccessible/secret, I must put it behind password protection, even if it will only be on the server for a few seconds.

Maybe BlueCoat offers a browser security service similar to that, and you or someone who does know about your file is using that AV product, or browser extension, or whatever.

whereas blue coat just get the HTTP/etc, and nothing else! What's with that?

Anyone who gets the 403 won't load the auxilliary CSS, JS, files, etc. because the 403 page probably doesn't have the links to them that the real page has. That might not actually be what you were asking about, though(?)