Forum Moderators: open

Message Too Old, No Replies

Strange problem downloading dmoz rdf files

         

SlowMove

12:01 am on Jul 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I'm trying to download a copy of the rdf dump. At first, I tried using IE to save the 283m data file from [rdf.dmoz.org...] As it was downloading, the size of the file transferred exceded the 283m, and I finally stopped the transfer when it exceeded 500m. I read on another board that IE tries to decompress the file during the download, and that Opera will just download without modifying the file. So, I tried using Opera and had the same problem. I even tried using a Perl script that works well for transferring small files, but for some reason it couldn't handle the download. Any ideas?

dhaliwal

6:45 am on Jul 20, 2004 (gmt 0)

10+ Year Member



hi slowmove

use something like flashget or other download managers

BTW, do you like to share with me, what directory software you are using, i am using one, but it can't handle more than 20 k records,
can you tell me what you are using,

moltar

6:47 am on Jul 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I had exactly the same problem as you described. I later used ReGet to download it with no problem at all.

tschild

7:48 am on Jul 20, 2004 (gmt 0)

10+ Year Member



I used to have the same issue when trying to download with MSIE. Using wget there is no problem. Using MSIE the download should be OK too if you stick it out until the file length displayed has grown to the uncompressed length (ca. 1.8 GByte).

SlowMove

10:08 am on Jul 20, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks. I actually got it to work with Download Accelerator

g1smd

11:09 pm on Jul 30, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



The RDF this week has zero UTF-8 errors, and that is in over 2.4 GB of data (content 1.9GB and structure 500MB).

Previous weeks in recent months had zero to ten errors each week (mostly 1 to 3 errors each time).