You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+14-31Lines changed: 14 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,9 +8,6 @@
8
8
9
9
---
10
10
11
-
> [!WARNING]
12
-
> Danzo has seen a significant refactor that changed how commands were called in previous versions. This was done to support an easier CLI interface and make it easier to add additional downloaders in the future.
13
-
14
11
## Quickstart
15
12
16
13
This section gives a quick peek at the capabilities and the extremely simple command structure. For detailed descriptions, see [Usage](#usage).
@@ -150,17 +147,15 @@ The output filename will be inferred from the URL and Danzo will use 8 connectio
> The value for `-c` can be arbitrary. Danzo creates chunks equal to number of connections requested. Once all chunks are downloaded, they are combined into a single file. If the decided number of chunks are too small, Danzo falls back to a single threaded download for that file.
150
+
> ✎ The value for `-c` can be arbitrary. Danzo creates chunks equal to number of connections requested. Once all chunks are downloaded, they are combined into a single file. If the decided number of chunks are too small, Danzo falls back to a single threaded download for that file.
155
151
156
152
You can customize the number of connections to use like so:
157
153
158
154
```bash
159
155
danzo "https://example.com/largefile.zip" -c 16
160
156
```
161
157
162
-
> [!WARNING]
163
-
> You should be careful of the disk IO as well. Multi-connection download takes disk IO, which can add to overall time before the file is ready.
158
+
> ⚠ You should be careful of the disk IO as well. Multi-connection download takes disk IO, which can add to overall time before the file is ready.
164
159
>
165
160
> For example, a 1 GB file takes 54 seconds when using 50 connections vs. 62 seconds when using 64 connections. This is because combining 64 files takes longer than combining 50 files.
166
161
>
@@ -202,8 +197,7 @@ Single-connection downloads store a `OUTPUTPATH.part` file in the current workin
202
197
203
198
These partial downloads on disk are useful when a download event is interrupted or failed. In that case, the temporary files are used to resume the download.
204
199
205
-
> [!WARNING]
206
-
> A resume operation is triggered automatically when the same output path is encountered. However, the feature will only work correctly if the number of connections are exactly the same. Otherwise, the resulting assembled file may contain faulty bytes.
200
+
> ⚠ A resume operation is triggered automatically when the same output path is encountered. However, the feature will only work correctly if the number of connections are exactly the same. Otherwise, the resulting assembled file may contain faulty bytes.
207
201
208
202
To clear the temporary (partially downloaded) files, use the command with the `clean` flag:
> Failed chunks are automatically retried up to 5 times before failing the entire file. Additionally, Danzo automatically runs a clean for a download event once it is successful.
210
+
> ⚡︎ Failed chunks are automatically retried up to 5 times before failing the entire file. Additionally, Danzo automatically runs a clean for a download event once it is successful.
218
211
219
212
</details>
220
213
@@ -241,8 +234,7 @@ Downloading a file from a Drive URL requires authentication, which Danzo support
241
234
- Danzo will exchange this for an authentication token and save it to `.danzo-token.json`.
242
235
- If you re-attempt the use of these credentials, Danzo will reuse the token from current directory if it exists, refresh it if possible, and fallback to reauthentication.
243
236
244
-
> [!TIP]
245
-
> The API Key method only works on files that are either publicly shared or shared with your user. It cannot be used to download private files that you own. So for your own files, use the OAuth device code method.
237
+
> ⚡︎ The API Key method only works on files that are either publicly shared or shared with your user. It cannot be used to download private files that you own. So for your own files, use the OAuth device code method.
246
238
247
239
Danzo can be used in this manner to download Google Drive files:
> Danzo does not perform multi-connection download for Google Drive files; instead it uses the simple download method. For Google Drive specifically, this does not present a significant loss in bandwidth. This is done because Google can throttle multiple connections after a while.
251
+
> ⚠︎ Danzo does not perform multi-connection download for Google Drive files; instead it uses the simple download method. For Google Drive specifically, this does not present a significant loss in bandwidth. This is done because Google can throttle multiple connections after a while.
261
252
262
-
> [!NOTE]
263
-
> Users who have never logged into GCP may be required to create a new GCP Project. This is normal and doesn't cost anything.
253
+
> ✎ Users who have never logged into GCP may be required to create a new GCP Project. This is normal and doesn't cost anything.
> In an effort to create a successful and simple integration, Danzo lets `yt-dlp` dictate the file extension for a given output. As such, the `-o` flag will not have an effect on the extension. Audio downloads will always have a `.m4a` download, while a video may have `.mp4`, or `.webm`.
271
+
> ✎ In an effort to create a successful and simple integration, Danzo lets `yt-dlp` dictate the file extension for a given output. As such, the `-o` flag will not have an effect on the extension. Audio downloads will always have a `.m4a` download, while a video may have `.mp4`, or `.webm`.
283
272
284
273
A download type can be appended to the URL to control Danzo's behavior. These defaults were chosen based on heuristics and observed popularity.
> YouTube downloads require `yt-dlp` to be installed on your system. If it's not found, Danzo will automatically download and use a compatible version. Additionally, since the STDOUT and STDERR are directly streamed from `yt-dlp` to `danzo`, YouTube videos are not tracked for progress the way HTTP downloads are. When downloading a single YouTube URL, the output from `yt-dlp` will be streamed to the user's STDOUT. But if the URL is part of a batch file, then the output is hidden and the progress appears stalled until finished.
289
+
> ✎ YouTube downloads require `yt-dlp` to be installed on your system. If it's not found, Danzo will automatically download and use a compatible version. Additionally, since the STDOUT and STDERR are directly streamed from `yt-dlp` to `danzo`, YouTube videos are not tracked for progress the way HTTP downloads are. When downloading a single YouTube URL, the output from `yt-dlp` will be streamed to the user's STDOUT. But if the URL is part of a batch file, then the output is hidden and the progress appears stalled until finished.
302
290
303
291
Danzo also supports downloading music from YouTube and automatically add metadata from the Deezer or the iTunes API, when the appropriate ID is provided. Example:
304
292
@@ -318,8 +306,7 @@ Danzo supports downloading streamed content from M3U8 manifests. This is commonl
318
306
319
307
Danzo downloads the M3U8 manifest, parses the playlist (supports both master and media playlists), downloads all segments, and merges them into a single file.
320
308
321
-
> [!NOTE]
322
-
> Danzo requires `ffmpeg` to be installed for merging the segments.
309
+
> ✎ Danzo requires `ffmpeg` to be installed for merging the segments.
AWS session profiles are used to allow for flexibility and ease of access. As a result, specifying the flag (`--profile`) allows using a profile of the user's choice. Additionally, when not set, Danzo uses the `default` profile.
357
344
358
-
> [!WARNING]
359
-
> For successful authentication, Danzo needs to use a profile that is configured for the same region as the S3 bucket.
345
+
> ⚠︎ For successful authentication, Danzo needs to use a profile that is configured for the same region as the S3 bucket.
360
346
361
-
> [!NOTE]
362
-
> For S3 downloads, the `connections` flag determines how many objects will be downloaded in parallel if downloading a folder.
347
+
> ✎ For S3 downloads, the `connections` flag determines how many objects will be downloaded in parallel if downloading a folder.
363
348
364
349
</details>
365
350
@@ -394,8 +379,7 @@ Danzo can clone repositores sourced by various providers. While this is not part
394
379
395
380
As such, given a situation where a server needs to be prepared for operation by cloning a set of 8 repositories, 5 different tool assets, and an S3 folder; it would be slow to write a script incorporating several tools to get the environment ready. Danzo would be the perfect fir for such a scenario due to its batch-download capability via a YAML configuration. It is primarily for this purpose that an operation as simple and atomic as `git clone` was replicated in Danzo.
396
381
397
-
> [!WARNING]
398
-
> While Danzo as a tool is focused on conducting very fast downloads, it is important to note that in some cases where a git repository may be more than 1.5-2 GB in size, Danzo may experience easily noticeable slowdowns compared to plain old `git clone`. This is expected and usually, it's recommended to enforce depth (continue reading) when cloning repositories that large.
382
+
> ⚠︎ While Danzo as a tool is focused on conducting very fast downloads, it is important to note that in some cases where a git repository may be more than 1.5-2 GB in size, Danzo may experience easily noticeable slowdowns compared to plain old `git clone`. This is expected and usually, it's recommended to enforce depth (continue reading) when cloning repositories that large.
399
383
400
384
Danzo supports the use of Personal Access Tokens as well as SSH keys when cloning repositories. The syntax has been simplified to refer to repositories with one of the following:
> Repository cloning is another download provider that does not use `-c` or number of connections. Number of workers, `-w`, is still applicable as usual in batch (YAML config) mode.
412
+
> ✎ Repository cloning is another download provider that does not use `-c` or number of connections. Number of workers, `-w`, is still applicable as usual in batch (YAML config) mode.
0 commit comments