How to find subdomain crawls in logs

Google is crawling, but can't see subdomain in logs.

[jbgilbert]

Identifying googlebot visits to a domain named
www.domain.com is easy,

BUT if there is a subdomain off that root domain like
subdomain.domain.com, I can't seem to locate in the log files the googlebot crawls.

Anybody know the secret to this?

[MrSpeed]

How are you viewing the logs?
Sometimes subdomains have different log files that the main domain.

[jbgilbert]

Mr Speed,
Good thought... I just started a ticket at my hosting company to see if this is the case.

I checked cPanel "subdomain stats" and webalyzer has reports, so the information is out there somewhere -- just not in the root domain log files I download.

Any other thoughts?

[MrSpeed]

I think the only way you can do it is to FTP in to get the logs for the subdomains.

In my version of Cpanel there are instructions at ftp manager/ftp accounts. Basically it says:

You can download your raw access logs at the following URLs using the login and your account password:
ftp://ftp.maindomain.com/subdomain.maindomain.com

I think you may need to create an ftp account like this:
[e-mail]

This is how it is on my host with the cpanel x skin.
Your host may be a little different.

[BroadProspect]

you can ask them to include the requested domain in the access log
/BP

[SyntheticUpper]

<snip>

[edited by: SyntheticUpper at 5:58 pm (utc) on Mar. 26, 2004]

[SyntheticUpper]

<snip - Google has sold out IMO, there's no point in making a point >

[edited by: SyntheticUpper at 7:05 pm (utc) on Mar. 26, 2004]

[gpmgroup]

On a w2k or a w2k3 (IIS) Server you would normally set up multiple identities for each domain. (Setting up a host header record for each subdomain). Under such a scheme all log entries would normally go to the same log file.

[SyntheticUpper]

<snip>

[edited by: SyntheticUpper at 7:05 pm (utc) on Mar. 26, 2004]

[gpmgroup]

Under IIS the default log settings are "Method" "URI Stem" and "URI Query". These do not give the subdomain infomation you are looking for.

What you need to do is under advanced settings for the logging properties select "Referrer"

Whilst this will not give the subdomain infomation for the first object retreived it will for subsequent objects, and from this you should be able to work out which hits are coming from which subdomains.