homepage Welcome to WebmasterWorld Guest from 54.237.184.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 39 message thread spans 2 pages: 39 ( [1] 2 > >     
Undefined Error in Logs
yaashul




msg:4480181
 4:47 pm on Jul 30, 2012 (gmt 0)

Many of my chrome user are trying to access a file /undefined. I dont have anything like that in website code. I tried to surf myself using a very same google chrome and didn't get any error in logs while searching for my ip.

Here r the copies of few of those logs

122.177.173.208 - - [30/Jul/2012:05:30:45 -0700] "GET /undefined HTTP/1.1" 404 1045 "MY SITE URL" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
122.160.115.76 - - [30/Jul/2012:05:30:51 -0700] "GET /undefined HTTP/1.1" 404 1046 " MY SITE URL " "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
122.177.173.208 - - [30/Jul/2012:05:30:57 -0700] "GET /undefined HTTP/1.1" 404 1045 " MY SITE URL " "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
14.96.249.134 - - [30/Jul/2012:05:31:10 -0700] "GET /undefined HTTP/1.1" 404 1046 " MY SITE URL " "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
122.177.173.208 - - [30/Jul/2012:05:31:20 -0700] "GET /undefined HTTP/1.1" 404 1045 " MY SITE URL " "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
122.160.115.76 - - [30/Jul/2012:05:31:26 -0700] "GET /undefined HTTP/1.1" 404 1046 " MY SITE URL " "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"
223.231.172.231 - - [30/Jul/2012:05:31:39 -0700] "GET /undefined HTTP/1.1" 404 1046 " MY SITE URL " "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57 Safari/536.11"

 

MxAngel




msg:4480304
 8:40 pm on Jul 30, 2012 (gmt 0)

It's most likely a troublemaker trying to probe your 404 response.

At one stage two IP's were listed on Project Honey Pot. Do the IP's have other requests?

For example on my site I see some of them trying to exploit the timthumb vulnerability and then they end with a GET /undefined

Also there's a little bit of more information in the following topics:

[webmasterworld.com...]
[webmasterworld.com...]

[stackoverflow.com...]

g1smd




msg:4480311
 9:16 pm on Jul 30, 2012 (gmt 0)

Is any of the navigation generated by javascript or some other script?

Use Xenu LinkSleuth to crawl the site and check for errors.

incrediBILL




msg:4480395
 2:51 am on Jul 31, 2012 (gmt 0)

It's from airtelbroadband.in which has an IP pool so I've had to block them completely a couple of times to stop bad activity, nothing new.

keyplyr




msg:4480401
 3:14 am on Jul 31, 2012 (gmt 0)

Actually, it can be from any person on any range that is using the safety settings that check for virus/malware/etc in Chrome browser, and possibly other browsers.

/undefined means the web page hasn't been classified as safe or otherwise yet. There's a couple more prepended tags that mean other things but I can't remember right now.


I forward them:

RedirectMatch 301 ^/undefined$ http://www.mysite.com/$1

incrediBILL




msg:4480412
 3:58 am on Jul 31, 2012 (gmt 0)

it can be from any person on any range


True, it may come from any range, but in the examples above it was from airtelbroadband.in

Doesn't mean you won't need to block other infiltrations, but I've been happily blocking that range for ages to keep the nasty things coming from it off my servers.

It's like blocking China, Russia, Nigeria, Vietnam, Ukraine, etc. If you don't do any business with those countries you're much safer from spam, hacking attempts, etc. if you don't let those areas of known high volume activity have access to your servers in the first place. The problem with airtelbroadband.in is that their modem pool randomly sends just as many good visitors as it does hackers, spammers and scrapers so ultimately I blocked them.

Then you deal with the rest :)

keyplyr




msg:4480421
 4:30 am on Jul 31, 2012 (gmt 0)


True, it may come from any range, but in the examples above it was from airtelbroadband.in

Doesn't mean you won't need to block other infiltrations, but I've been happily blocking that range for ages to keep the nasty things coming from it off my servers.

It's like blocking China, Russia, Nigeria, Vietnam, Ukraine, etc. If you don't do any business with those countries you're much safer from spam, hacking attempts, etc. if you don't let those areas of known high volume activity have access to your servers in the first place. The problem with airtelbroadband.in is that their modem pool randomly sends just as many good visitors as it does hackers, spammers and scrapers so ultimately I blocked them.

Then you deal with the rest :)


But that has nothing to do with the "undefined" appendage.

But thanks for the heads-up about airtelbroadband.in. I'll take a look at them.

incrediBILL




msg:4480425
 4:58 am on Jul 31, 2012 (gmt 0)

But that has nothing to do with the "undefined" appendage.


How true, cocktail?

I'll throw out a bone on that part of the topic as I was just mesmerized by the IPs themselves in the OP.

The best guess anyone has ever had was that it's caused by either a flaw in a CSS file pr javascript of a missing image or some other file and the browser is using "undefined" to request this missing image which is most likely reference by an empty variable, hence "undefined". Check the CSS file for any missing bullets on custom item lists for starters and then peruse your javascript.

It's probably some just an empty object on some property causing a glitch.

Can you say Firebug?

It's probably going to take a while to find IMO.

yaashul




msg:4480432
 5:26 am on Jul 31, 2012 (gmt 0)

incrediBILL,

I m getting more than 4000 request from different ip and most of them are from USA... I just choose these logs which is from airtelbroadband. Even I use to surf using airtel in office and I dont get such error.

I dont think it has anything to do with ISP.

keyplyr




msg:4480434
 5:29 am on Jul 31, 2012 (gmt 0)



The best guess anyone has ever had was that it's caused by either a flaw in a CSS file pr javascript of a missing image or some other file and the browser is using "undefined" to request this missing image which is most likely reference by an empty variable, hence "undefined". Check the CSS file for any missing bullets on custom item lists for starters and then peruse your javascript.

It's probably some just an empty object on some property causing a glitch.


I already explained what it is.

yaashul




msg:4480439
 5:34 am on Jul 31, 2012 (gmt 0)

Will it be possible that I have something wrong in javascript. But I am lil confused here. Why my ip logs never show in the error_log while I surf using google chrome.

yaashul




msg:4480442
 5:40 am on Jul 31, 2012 (gmt 0)

keyplyr,

If CSS has this issue why this is not coming while I surf? I mean my ip never showup in error logs.

keyplyr




msg:4480458
 6:37 am on Jul 31, 2012 (gmt 0)


@ Yaashul

I'm only speaking of the "undefined" appendage to your files in your logs. It has nothing to do with your web site code (CSS or javascript or HTML.) The extra word "undefined" is an error generated by the safety settings in the Chrome (and possibly other) browsers.

"Undefined" basically means that the safety checker has not determined whether the web page is safe or not, so it is undefined. It should not be getting into your server logs but it does sometimes. This is why I refer to it as an error.

I see it in my server logs occasionally as well. Instead of returning a 404 and stopping the user from accessing the web page, I eventually put up a redirect (see my above post msg:4480395)

This is nothing to worry about. You are not being hacked or scraped. Eventually Chrome will correct this (maybe.)

incrediBILL




msg:4480462
 7:22 am on Jul 31, 2012 (gmt 0)

I already explained what it is.


Not that I don't agree with your explanation, I couldn't find anything that definitively claimed it was the problem and I found others claiming my explanation.

I've got a couple of high traffic sites and a bunch of new sites and I've never seen this "undefined" stuff on any of my sites, ever.

Doesn't mean I dispute it happens but I'd sure like to find a specific reproducible case study to definitively be able to state the cause.

keyplyr




msg:4480471
 8:26 am on Jul 31, 2012 (gmt 0)

You'll see they all are Chrome browsers. I've been seeing them for about 6 months. Sometimes they are followed by other tell-tale signs that this is the Chrome security check. As I said above, there are also a couple other appendages that sometimes occur. Sorry I forget the others, but they are along the same lines: undefined, warning, safe, etc (those are not the exact terms, but you get the idea.)

I don't use Chrome very much as I have all my tools with Firefox. But along the same lines, I've seen the red warning page alerts when Firefox has a web page classified as a threat. When Chrome does this, I believe these "undefined" (and other appended terms) end up in the server logs when the user then clicks through that warning, gaining the added parameter (appended term.) This shouldn't happen, but it does sometimes.

MxAngel




msg:4480599
 3:12 pm on Jul 31, 2012 (gmt 0)

Interesting about the security check ...

I get them as much on IE8 / IE9 as Chrome. Been seeing them for a while too already. Haven't seen any with Firefox yet

IP:2.33.246.35
UA: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; SIMBAR={D3D36BC0-6DB4-11E0-B5E8-001B24851215}; GTB7.3; FunWebProducts; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; FunWebProducts)
GET/a_category_on_the site/undefined
Referer: a_URL_on_the_site.html

IP:182.73.19.90
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.47 Safari/536.11
GET/undefined
Referer: Main_site_url

yaashul




msg:4480631
 5:44 pm on Jul 31, 2012 (gmt 0)

I use Xenu LinkSleuth and I couldn't find any broken links on the pages /udefined error is showing

keyplyr




msg:4480641
 6:09 pm on Jul 31, 2012 (gmt 0)


@ yaashu. Why should you? Maybe you're misunderstanding all this. There is nothing wrong with your web pages.

g1smd




msg:4480668
 7:24 pm on Jul 31, 2012 (gmt 0)

>
RedirectMatch 301 ^/undefined$ http://www.mysite.com/$1

The $1 backreference value is always undefined.

keyplyr




msg:4480676
 7:45 pm on Jul 31, 2012 (gmt 0)



But what does that have to do with anything? (unless your just making a joke.)

g1smd




msg:4480680
 8:12 pm on Jul 31, 2012 (gmt 0)

Just pointing out that the code in
msg:4480401 has problems.
keyplyr




msg:4480691
 8:39 pm on Jul 31, 2012 (gmt 0)



Problems? No, it works as desired.

dstiles




msg:4480692
 8:39 pm on Jul 31, 2012 (gmt 0)

I have only one "undefined" in this month's (July) security log. It was with an MSIE 8 UA. The hit came from a TalkTalk/Opal (UK) "infrastructure" range that I've blocked for a long time. It runs, amongst other things, a so-called "anti-virus spider" (which hits AFTER the page has been looked at by the visitor! - see recent threads hereabouts).

I'm not saying my hit WAS a spider but it's in the range 62.24.128.0/17 which I've flagged as "includes lots of bots, webmarketing and HuaweiSymantecSpider" and the header was bot-ish.

g1smd




msg:4480695
 8:44 pm on Jul 31, 2012 (gmt 0)

Problems? No, it works as desired.

It's coding errors like this that can cause all sorts of issues on a site and take a very long time to diagnose.

lucy24




msg:4480728
 11:05 pm on Jul 31, 2012 (gmt 0)

###! Typed faster than me again.

RedirectMatch 301 ^/undefined$ http://www.mysite.com/$1

The $1 backreference value is always undefined.

Since $1 doesn't refer to anything, what's it doing in the code at all? That is, what's the difference between (AHEM!)

http://www.example.com/$1

and

http://www.example.com/

other than that the first version puts the server to extra work looking up something that doesn't exist?


But what I really meant to ask was: In the original post, what exactly does "MY SITE URL" in the Referer represent? Is it the exact content of the Request, minus only the "/undefined" element, or is it always www.example.com/ (bare domain) regardless of request?

keyplyr




msg:4480742
 12:07 am on Aug 1, 2012 (gmt 0)

Alright, alright... I changed it:

RedirectMatch 301 ^/undefined http://www.example.com/

Actually, what the $1 does do is redirect any added "undefined" on any other page to the index page... so:

http://www.example.com/widget.html/undefined

gets redirected to:

http://www.example.com/


But since these "undefined" params have so far only been present on base URLs, then the extra characters aren't really necessary.


[added]
Added the slash here. It's in the code but missed it in the cut'n paste here. The $1 did in fact do what it was intended to do. The "1" means the first, or base. Been in use for many years. Might not work on your server config though.

[edited by: keyplyr at 12:50 am (utc) on Aug 1, 2012]

g1smd




msg:4480747
 12:21 am on Aug 1, 2012 (gmt 0)

Actually, what the $1 does do is

... nothing. In the example code $1 never has a value, and should be removed.

Additionally, the canonical URL for the root page of the site is www.example.com/ with a trailing slash. Do not forget the slash.

lucy24




msg:4480848
 9:27 am on Aug 1, 2012 (gmt 0)

RedirectMatch 301 ^/undefined http://www.example.com/

Actually, what the $1 does do is redirect any added "undefined" on any other page to the index page...

Uhm, no, it doesn't. The code has a beginning anchor ^ so it will only work at the root level. To redirect all pages with trailing "undefined" you have to leave off the anchor.

The $1 means "reuse the first captured group". Remember, we're in RedirectMatch, which speaks RegEx. Different from vanilla Redirect. Since there is no captured group, there's nothing to reuse. Matter of fact, some servers might get seriously upset. They don't mind empty groups like (blahblah)? but they can have strong opinions about groups that haven't been defined at all.

You do not want to upset your server. If jdmorgan were here, he could come up with 500 reasons why not ;)

keyplyr




msg:4480857
 9:52 am on Aug 1, 2012 (gmt 0)

I'm not here to argue Lucy. In fact jdMorgan wrote that code about 10 years ago on one of my old sites he worked on, just with a different param. One more time, it worked the way I explained. I tested several times yesterday before I removed it. The beginning anchor was added later to accommodate the OP when I cut'n pasted. End of conversation. Geeez.

wilderness




msg:4481085
 10:00 pm on Aug 1, 2012 (gmt 0)

keyplr, FWIW some of the syntax that Jim provided a decade ago was even changed in methods by he in later years, and as newer versions of Apache came out.

When doing some obscure searches, I've seen these old-new-syntax conflicts time and again.

I've lines in place that I've been using more than a decade and lucy and g1smd insist that they are either dysfunctional or greedy and yet the lines are still in place.
Go figure ;)

This 39 message thread spans 2 pages: 39 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved