Welcome to WebmasterWorld Guest from 18.206.194.83

Forum Moderators: phranque

Message Too Old, No Replies

Using wget to mirror site

Works on server A, but fails on B

     
3:43 pm on Nov 8, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 5, 2002
posts:367
votes: 0


I've got a script that runs a wget command to mirror a full site. The command is:

wget -S -m -nH ftp://username:password@REMOTE_HOST_IP

On server A, this results in a full download of the files on the REMOTE_HOST_IP. However, the exact same command on server B results merely in a index.html file displaying a listing of what's on REMOTE_HOST_IP, but it does not download the files itself...

Any ideas?

12:52 am on Nov 9, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member jtara is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Nov 26, 2005
posts:3041
votes: 0


I note that you are using FTP to access the site. FTP will access a certain file system directory on the disk, and it's descendants.

There is not necessarily a correspondence between URLs and file system directories, though.

I have to assume that the files (other than index.html) are in some other locations, and there is some URL redirection going-on in the HTTP server configuration.

Another possibility is that the FTP account does not have permission to access the files, save for index.html.

12:34 pm on Nov 9, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 5, 2002
posts:367
votes: 0


I note that you are using FTP to access the site. FTP will access a certain file system directory on the disk, and it's descendants.

There is not necessarily a correspondence between URLs and file system directories, though.

Yep, i use FTP. But like i said, the exact same command works 100% fine when executed from another server.

I have to assume that the files (other than index.html) are in some other locations, and there is some URL redirection going-on in the HTTP server configuration.

Not sure I can follow what you mean...

Another possibility is that the FTP account does not have permission to access the files, save for index.html.

Permission should be o.k., since the exact same user/pass combination is used when the wget command is executed from anoter server...

6:02 pm on Nov 9, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member jtara is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Nov 26, 2005
posts:3041
votes: 0


the exact same command works 100% fine when executed from another server.

Ah. I misunderstood. I thought you were backing up two different sites. You're backing up one site to two different backup servers.

4:42 pm on Nov 13, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 5, 2002
posts:367
votes: 0


You're backing up one site to two different backup servers.

Yup... Sorry for the mixup. Any idea though?

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members