Welcome to WebmasterWorld Guest from 54.196.86.89

Forum Moderators: Ocean10000 & incrediBILL & phranque

Find country from Apache logs

     
10:55 pm on Apr 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


Hi I want a linux command that'll resolve ips by their resolved names (if available) and then sort by times they have accessed my site.

I need this to find out some ips in a particular country that's sending huge number of bot traffic without much clue. Thanks!
11:08 pm on Apr 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


I'm searching for logresolve scripts in google. I want only the IPs sorted by counts (but not print the counts). Then I"ll do a logresolve on the ip list.
11:09 pm on Apr 26, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11283
votes: 133


i would write a perl script to do that.
11:18 pm on Apr 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


I have this command that works:

cat access.log | awk '{print $1}' | sort -n | uniq -c | sort -nr | head -20

this gives a list like this (includes count):

898 54.166.172.33

I don't want the 898 .. just the IP address.. so in the command above what's printing the number of counts? Thanks!
11:28 pm on Apr 26, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


It's uniq -c but if I remove -c it doesn't seem to work..
11:57 pm on Apr 26, 2018 (gmt 0)

Full Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 329
votes: 23


The unix host command will give you the host name:

$ host 54.166.172.33
33.172.166.54.in-addr.arpa domain name pointer ec2-54-166-172-33.compute-1.amazonaws.com


or in-address arpa + "domain name pointer" + host name

I usually write a shell script. To each his own.

-unique first, then sort, then process unique IPs to host names
-kill using htaccess!
12:19 am on Apr 27, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


So I managed to create a list of unique IP addresses from access log file. The list contains only the unique ip addresses. Is there any way I can run a script to resolve these ip addresses? If you can please share the script. Thanks!
1:15 am on Apr 27, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:11485
votes: 692


Have a look at Analog [mirror.reverse.net]
2:08 am on Apr 27, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Dec 19, 2004
posts:819
votes: 10


Thanks keyplyr.. I also found a script on google that resolves a file of list of ip addresses:

======
#!/bin/sh
for IP in `cat ./ipaddress.txt`;
do
printf "%-4s", $IP
nslookup $IP | grep -v nameserver | cut -f 2 | grep name | cut -f 2 -d "=" | sed 's/ //';

done
======
2:26 am on Apr 27, 2018 (gmt 0)

Full Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 329
votes: 23


Shell script works.
2:57 am on Apr 27, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11283
votes: 133


then sort by times they have accessed my site.

i misunderstood this.

Is there any way I can run a script to resolve these ip addresses?

see TorontoBoy 's response above...
2:59 am on Apr 27, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11283
votes: 133


i didn't see some of the responses when i posted.

host or nslookup works...
3:40 am on Apr 27, 2018 (gmt 0)

Full Member from CA 

Top Contributors Of The Month

joined:Feb 7, 2017
posts: 329
votes: 23


input IP: 197.231.221.211
script output: 197.231.221.211,exit1.ipredator.se

It might be helpful to output the netname instead: CYBERDYNE-VPN01

$ host 197.231.221.211
211.221.231.197.in-addr.arpa domain name pointer exit1.ipredator.se


Try a whois lookup on some of these IPs. The host name may not be helpful. Also some suspect IPs often don't have a host name set up, on purpose, so you might get something odd returned from the script. Bots IPs will try any trick they can to not be identified.

$ whois 197.231.221.211
% This is the AfriNIC Whois server.

% Note: this output has been filtered.
% To receive output for a database update, use the "-B" flag.

% Information related to '197.231.221.0 - 197.231.221.255'

% No abuse contact registered for 197.231.221.0 - 197.231.221.255

inetnum: 197.231.221.0 - 197.231.221.255
netname: CYBERDYNE-VPN01
descr: Cyberdynes VPN users, block 01.
country: LR
admin-c: NW2-AFRINIC
tech-c: NW2-AFRINIC
status: ASSIGNED PA
mnt-by: CyberdyneSA-MNT
source: AFRINIC # Filtered
parent: 197.231.220.0 - 197.231.223.255

<snip>
nic-hdl: NW2-AFRINIC
mnt-by: GENERATED-WVURFBJ8EPYM0NQF6GHLKDUQS7QK9DL3-MNT
source: AFRINIC # Filtered




[edited by: not2easy at 4:13 am (utc) on Apr 27, 2018]
[edit reason] see Charter/ToS [/edit]