bash - WGET ignoring --content-disposition? -
i trying run command download 3000 files in parallel. using cygwin + windows.
downloading single file via wget in terminal :
wget --no-check-certificate --content-disposition --load-cookies cookies.txt \ -p https://username:password@website.webpage.com/folder/document/download/1?type=file
allows me download file id 1 singularly, in correct format (as long --content-disposition
in command).
i iterate on rest api call download entire folder (3000 files). works ok, quite slow.
for /l %i in (0,1,3000) wget --no-check-certificate --content-disposition --load-cookies cookies.txt \ -p https://username:password@website.webpage.com/folder/document/download/%i?type=file
now trying run program in cygwin, in parallel.
seq 3000 | parallel -j 200 wget --no-check-certificate --content-disposition --load-cookies cookies.txt -p https://username:password@website.webpage.com/folder/document/download/{}?type=file
it runs, file-name , format lost (instead of "index.html"
, example, may "4@type=file"
file-name).
is there way me fix this?
it unclear them named. let assume want them named: index.[1-3000].html
seq 3000 | parallel -j 200 wget -o index.{}.html --no-check-certificate --content-disposition --load-cookies cookies.txt -p https://username:password@website.webpage.com/folder/document/download/{}?type=file
my guess is caused --content-disposition being experimental, , wget
used cygwin may older wget
used loop. check run:
wget --version
in cygwin , outside cygwin (ie. run loop).
Comments
Post a Comment