Welcome to WebmasterWorld Guest from 54.227.127.109

Forum Moderators: Ocean10000 & incrediBILL & phranque

How to block an entire company's domain name using htaccess?

     
4:36 pm on Nov 29, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


So I have a company example.com and I want to block everything come out of that main domain eg.. example.example.com etc.

Is there anyway in htaccess in apache to solve this issue? Thanks!
5:09 pm on Nov 29, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


I do this all the time when I see a history of bad bot behaviour coming from a specific company. Here are my steps:
-https://www.whois.com/whois/ look them up, find their IP address range
-within htaccess, add statement "deny from your.ip.address.here/24", where your.ip.address.here is the numeric IP address from your whois query, and the /24 is the required CIDR range

Alas, life is not that clear cut. Companies can have multiple IP ranges for the same host provider, they can jump countries, they are multi-national and operate amongst many host providers. There are many ways for specific companies to remain hidden while still getting access to your site. They do not want to be found.

I have never been able to ban a top level domain name by name, within my htaccess.
5:30 pm on Nov 29, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Torontoboy the whois website you gave doesn't always give the ip address range of any domain.. any other suggestions?

I realize that access-logs only seem to have ip addresses..
5:38 pm on Nov 29, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


Yes, you are correct. Try the unix command "host example.com" which should return an IP address.

$ host google.com
google.com has address 172.217.2.174
google.com has IPv6 address 2607:f8b0:400b:809::200e
google.com mail is handled by 30 alt2.aspmx.l.google.com.
google.com mail is handled by 20 alt1.aspmx.l.google.com.
google.com mail is handled by 10 aspmx.l.google.com.
google.com mail is handled by 40 alt3.aspmx.l.google.com.
google.com mail is handled by 50 alt4.aspmx.l.google.com.
$

NetRange: 172.217.0.0 - 172.217.255.255
CIDR: 172.217.0.0/16
NetName: GOOGLE

deny from 172.217.0.0/16
5:48 pm on Nov 29, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Nope neither does the above command work. Well I've figured out somewhat by looking for ASN using company name
6:19 pm on Nov 29, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


research "linux dns lookup" and choose your tool. [cyberciti.biz...]
6:33 pm on Nov 29, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14326
votes: 563


It is theoretically possible to say outright
Deny from example.com
but I strongly advise you not to do this, as it throws your server logs into lookup mode (I can never remember the correct technical term) so everything else becomes unreadable.

born2run, are you looking to block people who are actually at example.com, or just example.com referers? If you already know they've committed transgressions, wouldn't that be visible in logs with a nice plain numeric IP that you can turn around and deny?
7:34 pm on Nov 29, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


It is theoretically possible to say outright
Deny from example.com
but I strongly advise you not to do this, as it throws your server logs into lookup mode

That mode is what I call a mistake. All my IP addresses disappear, replaced my untraceable domain names. Been there, not so useful. It would be convenient to bad by host name as well as by IP address.
9:54 pm on Nov 29, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14326
votes: 563


That mode is what I call a mistake.

I prefer to restrict the word “mistake” to things that will actually throw a 500-class error, or that will not have the intended result. That's why I said “strongly advise not to”. (You can achieve the same unreadable-logs result by accidentally dropping a comma into a Deny from line; I tend to do this every year or two.)

Apache 2.4 has a wider range of access-control options, but here we appear to be in 2.2.
10:33 pm on Nov 29, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11123
votes: 111


I want to block everything come out of that main domain

You want to block requests that are referred by that domain or you want to block visitors representing the company that owns that domain?
9:11 am on Nov 30, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Hi guys want to block all employees visiting from company abcd. Even if someone here can let me know how to find ip addresses used by this company I’d be very grateful..
11:06 am on Nov 30, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 19, 2002
posts:3340
votes: 33


The problem is that you could block all the ip's registered to that company - assuming you can discover them.
however itr doesn't stop employees working from home or in a coffee shop etc.
additionally if they are running bots they may well be running them from the amazon cloud or suchlike.
... mind you, every bit helps and every obstacle you put in their way is something.
12:26 pm on Nov 30, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11123
votes: 111


Even if someone here can let me know how to find ip addresses used by this company I’d be very grateful.

if there's a pattern to their requests, such as a specific set of requests or unique user agent string, you may be able to find this information by analyzing your server access log file.
if they are accessing through some isps such as mobile networks it will be difficult to block by IP address without blocking all access by that provider.
this might work if you don't care about visitors from a provider in a location you don't care about but you might not want to block verizon for example or your local/national provider.
12:56 pm on Nov 30, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


Hi guys want to block all employees visiting from company abcd. Even if someone here can let me know how to find ip addresses used by this company I’d be very grateful..

The above methods could work if they are not too intelligent. If they are tech savvy they could use Opera VPN, any other VPN, or Tor browser, and you'll never find them.
1:38 pm on Nov 30, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


It's ok if they use VPN, tor etc.. but as per phranque, I used logresolve to convert access-logs to domain names. Then I'll see what ips are coming from which domain.com etc. Thanks so much! :-)
1:42 pm on Nov 30, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


Their internet service provider will be different from their web site's domain name host provider. These two are not correlated.

For example I am Canadian but my host provider is from the US. My internet service provider is local and is Canadian. The two are unrelated.
2:23 pm on Nov 30, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Yes I need the internet service provider's ips for the company's employees visitors to my site. I understand web site hosting IP will be different.
8:45 pm on Nov 30, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:14326
votes: 563


Now, wait a minute. The company in question is so small that it uses a commercial ISP and doesn't have its own IP block? If so, what exactly are they doing that's so egregious you need to block the whole company? And that's not even talking about the company employees who might happen to be browsing on their smartphones, using whatever data plan they're individually on. Those would be flatly impossible to block.
5:25 am on Dec 1, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


lucy24 I wouldn't know what internet setup they have. I just wanna block their desktop access. I don't care about smartphones, vpn, tor browser etc. It's ok. Thanks all! :-)
12:12 pm on Dec 1, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5462
votes: 3


In North America
Using the ARIN Search
Paste in one of the IP's from you logs. th

View/click Parent
View/click Organization
Click on embedded link at end of name
Scroll down to related Networks
Copy and paste the list into a text editor and save some where.
If the list is large (frequently even if it is not there will be smaller duplications that are contained in larger blocks), than you'll be require to sort and combine ranges (eliminating duplicates)
Deny the IP ranges
3:16 pm on Dec 1, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


Using the ARIN Search [arin.net...]
Paste in one of the IP's from you logs. th
View/click Parent
View/click Organization
Click on embedded link at end of name
Scroll down to related Networks

Thanks. This works quite well. It is exhaustive.
2:19 pm on Dec 6, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Torontoboy thanks.. I don't have knowledge of the company's IPs and from access-logs the company name doesn't show up even when I run the access-logs using logresolve.. so is there anyway to find via companyname.com domain?

Thanks!
5:55 pm on Dec 6, 2017 (gmt 0)

Junior Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 191
votes: 13


Find someone within the company and ask them which internet company is providing internet access. In particular sysadmin and dev people know this information. There are a variety of social engineering methods that are applicable. We are all human. If there is a will there is a way.
2:32 pm on Dec 15, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


Hi, another question so if example.com is using Akamai cdn, is there anyway to find out their real ip address? Thanks!
3:21 pm on Dec 15, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11123
votes: 111


is there anyway to find out their real ip address?

"real" ip address for what?
a cdn is used for hosting content.
it is not used for browsing or crawling your site.
9:00 pm on Dec 15, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:7921
votes: 562


Usually one blocks because of a behavior. What behavior are you trying to block?
10:31 am on Dec 16, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


tangor behavior is dishonesty in business.. so that's why I was looking at way to block all visitors coming from example.com's office
12:40 pm on Dec 16, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11123
votes: 111


I was looking at way to block all visitors coming from example.com's office

their cdn is irrelevant for this purpose.
3:24 pm on Dec 16, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:693
votes: 8


I agree phranque what other methods are there technically speaking? I did logresolve to my access-logs but none showed example.com..
6:10 pm on Dec 16, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:7921
votes: 562


If you are trying to stop a competitor from stealing your content, presentation, pricing, etc., you can bet they are doing it from a variety of locations, likely none of which are example.com, and all will be successful. You can't stop dishonesty with .htaccess

Not the answer you're looking for, but it is the right answer. Invest your time and angst in things you can actually control.

If it is content theft, and you can prove something actionable, DMCA is your most powerful route. Shut them down.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members