| Line | Revision | Contents |
| 1 | 7 | Download Machine |
| 2 | 35 | |
| 3 | 7 | simple, non interactive, non graphical, batch download tool |
| 4 | 17 | for FTP, FSP and HTTP downloads with resume support |
| 5 | 7 | Licensed as Open Source Software (GPL) |
| 6 | ||
| 7 | 36 | Radim Kolar (hsn@users.sourceforge.net) |
| 8 | 35 | http://dmachine.sourceforge.net/ |
| 9 | 7 | |
| 10 | How to use: | |
| 11 | =========== | |
| 12 | ||
| 13 | 17 | 1. Configure it by editing dmachine.cnf. |
| 14 | Normally not needed. | |
| 15 | ||
| 16 | 2. Run the background download daemon: | |
| 17 | 62 | java -jar dmachine-0.23.jar |
| 18 | 17 | |
| 19 | 3. Put some URLs into queue file: | |
| 20 | 7 | |
| 21 | 91 | Default queue filename is 'queue', but it can be changed in dmachine.cnf, you |
| 22 | can even have multiple queue files watched for changes. | |
| 23 | ||
| 24 | Open queue file with any ASCII editor and place required URL to it, one per line. | |
| 25 | I use copy & paste between Firefox and text editor. | |
| 26 | ||
| 27 | Smart Cache proxy server can be configured to write download requests to specified | |
| 28 | queue file. See manual at http://scache.sourceforge.net for more details. | |
| 29 | 7 | |
| 30 | 62 | 4. Save queue in editor and wait |
| 31 | 7 | |
| 32 | 91 | 5. Queue files are checked by daemon, so you can add more URL to it and |
| 33 | 7 | save it. |
| 34 | ||
| 35 | 6. DM has also drop directory support. This is a special directory; any | |
| 36 | file in this directory will be processed as queue and then deleted. | |
| 37 | ||
| 38 | Special commands in queue file | |
| 39 | ============================== | |
| 40 | 91 | Queue file is not just simple URL list one per line, there are also few |
| 41 | special commands for fine-tuning download operation. Queue files can | |
| 42 | have comments, just place # at start of line. | |
| 43 | 7 | |
| 44 | 1. Specifying an alternative URLs for file | |
| 45 | ||
| 46 | Download Machine uses only filename part of URL. If you are place: | |
| 47 | ||
| 48 | http://www.server.com/file.zip (and) | |
| 49 | http://mirror.server.com/mirrored/file.zip | |
| 50 | 35 | |
| 51 | 91 | into queue file, DM will think that these two are alternative URLs for the |
| 52 | 7 | same file. |
| 53 | 35 | |
| 54 | 7 | If the same file is on the net with different filename you can use: |
| 55 | http://www.server.com/stuff/file-1.2.3.zip#file123.zip | |
| 56 | http://www.server.org/stuff/file-123.zip#file123.zip | |
| 57 | ||
| 58 | 2. Custom Referer header | |
| 59 | ||
| 60 | 11 | Some servers checks Referer header. Download Machine sends fake Referer |
| 61 | 7 | header, which works in most time, but sometimes you will need a custom |
| 62 | Referer header. | |
| 63 | ||
| 64 | If you place into queue file: | |
| 65 | 11 | Referer http://216.42.31.135/ http://www.zephyr.com/ |
| 66 | 7 | every request to http://216.... will gets second (zephyr) URL as Referer. |
| 67 | ||
| 68 | 3. Custom cookie header | |
| 69 | 35 | |
| 70 | 7 | Some servers needs Cookie header to be sent. Cookies are often used for |
| 71 | 11 | authorization purposes and sometimes server can refuse a download if |
| 72 | 7 | no valid cookie was found in request. First argument is server name |
| 73 | and second is cookie value in format name=value[; ...] | |
| 74 | ||
| 75 | cookie http://www.fileplanet.com/ auth=542345123456 | |
| 76 | 35 | |
| 77 | 11 | 4. Check also other directory if file is already downloaded |
| 78 | 7 | |
| 79 | check_also <regexp> <directory> | |
| 80 | ||
| 81 | example: | |
| 82 | check_also *.mp3 /home/hsn/mp3/Krishna_Das/ | |
| 83 | ||
| 84 | 5. Save file to alternate directory | |
| 85 | ||
| 86 | save_to <regexp> <directory> | |
| 87 | ||
| 88 | example: | |
| 89 | save_to *.mp3 /home/hsn/mp3/Krishna_Das/ | |
| 90 | 35 | |
| 91 | 11 | Limitations: |
| 92 | ============ | |
| 93 | ||
| 94 | 17 | Access to Password-protected HTTP/FTP files fails for sure because |
| 95 | Java 1.1 can't parse URL with passwords. | |
| 96 | 94 | HTTP chunked encoding is not supported. It might corrupt some downloads. |
| 97 | 7 | |
| 98 | C/ |
Loggerhead is a web-based interface for Bazaar branches