Skip to content

Commit

Permalink
docs: wordsmith fetch & fetchpost description and usage text
Browse files Browse the repository at this point in the history
[skip ci]
  • Loading branch information
jqnatividad committed Nov 6, 2024
1 parent 4951e28 commit 0cfddbb
Show file tree
Hide file tree
Showing 3 changed files with 7 additions and 7 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
| [explode](/src/cmd/explode.rs#L2)<br>🔣👆 | Explode rows into multiple ones by splitting a column value based on the given separator. |
| [extdedup](/src/cmd/extdedup.rs#L2)<br>👆 | Remove duplicate rows from an arbitrarily large CSV/text file using a memory-mapped, [on-disk hash table](https://crates.io/crates/odht). Unlike the `dedup` command, this command does not load the entire file into memory nor does it sort the deduped file. |
| [extsort](/src/cmd/extsort.rs#L2)<br>🚀📇👆 | Sort an arbitrarily large CSV/text file using a multithreaded [external merge sort](https://en.wikipedia.org/wiki/External_sorting) algorithm. |
| [fetch](/src/cmd/fetch.rs#L3)<br>✨🧠🌐 | Fetches data from web services for every row using **HTTP Get**. Comes with [HTTP/2](https://http2-explained.haxx.se/en/part1) [adaptive flow control](https://medium.com/coderscorner/http-2-flow-control-77e54f7fd518), [jaq](https://github.com/01mf02/jaq?tab=readme-ov-file#jaq) JSON query language support, dynamic throttling ([RateLimit](https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-06.html)) & caching with available persistent caching using [Redis](https://redis.io/) or a disk-cache. |
| [fetch](/src/cmd/fetch.rs#L3)<br>✨🧠🌐 | Send/Fetch data to/from web services for every row using **HTTP Get**. Comes with [HTTP/2](https://http2-explained.haxx.se/en/part1) [adaptive flow control](https://medium.com/coderscorner/http-2-flow-control-77e54f7fd518), [jaq](https://github.com/01mf02/jaq?tab=readme-ov-file#jaq) JSON query language support, dynamic throttling ([RateLimit](https://www.ietf.org/archive/id/draft-ietf-httpapi-ratelimit-headers-06.html)) & caching with available persistent caching using [Redis](https://redis.io/) or a disk-cache. |
| [fetchpost](/src/cmd/fetchpost.rs#L3)<br>✨🧠🌐 | Similar to `fetch`, but uses **HTTP Post** ([HTTP GET vs POST methods](https://www.geeksforgeeks.org/difference-between-http-get-and-post-methods/)). Supports both HTML form (application/x-www-form-urlencoded) and JSON (application/json) content types. |
| [fill](/src/cmd/fill.rs#L2)<br>👆 | Fill empty values. |
| [fixlengths](/src/cmd/fixlengths.rs#L2) | Force a CSV to have same-length records by either padding or truncating them. |
Expand Down
6 changes: 3 additions & 3 deletions src/cmd/fetch.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#![allow(unused_assignments)]
static USAGE: &str = r#"
Fetches data from web services for every row using HTTP Get.
Send/Fetch data to/from web services for every row using HTTP Get.
Fetch is integrated with `jaq` (a jq clone) to directly parse out values from an API JSON response.
(See https://github.com/01mf02/jaq for more info on how to use the jaq JSON Query Language)
Expand Down Expand Up @@ -43,8 +43,8 @@ If you don't want responses to be cached at all, use the --no-cache flag.
NETWORK OPTIONS:
Fetch recognizes RateLimit and Retry-After headers and dynamically throttles requests
to be as fast as allowed. The --rate-limit option sets the maximum number of queries per second
(QPS) to be made. The default is 0, which means to go as fast as possible,
automatically throttling as required.
(QPS) to be made. The default is 0, which means to go as fast as possible, automatically
throttling as required, based on rate-limit and retry-after response headers.
To use a proxy, set the environment variables HTTP_PROXY, HTTPS_PROXY or ALL_PROXY
(e.g. export HTTPS_PROXY=socks5://127.0.0.1:1086).
Expand Down
6 changes: 3 additions & 3 deletions src/cmd/fetchpost.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#![allow(unused_assignments)]
static USAGE: &str = r#"
Fetchpost fetches data from web services for every row using HTTP Post.
Fetchpost sends/fetches data to/from web services for every row using HTTP Post.
As opposed to fetch, which uses HTTP Get.
CSV data is posted using two methods:
Expand Down Expand Up @@ -58,8 +58,8 @@ If you don't want responses to be cached at all, use the --no-cache flag.
NETWORK OPTIONS:
Fetchpost recognizes RateLimit and Retry-After headers and dynamically throttles requests
to be as fast as allowed. The --rate-limit option sets the maximum number of queries per second
(QPS) to be made. The default is 0, which means to go as fast as possible,
automatically throttling as required.
(QPS) to be made. The default is 0, which means to go as fast as possible, automatically
throttling as required, based on rate-limit and retry-after response headers.
To use a proxy, please set env vars HTTP_PROXY, HTTPS_PROXY or ALL_PROXY
(e.g. export HTTPS_PROXY=socks5://127.0.0.1:1086).
Expand Down

0 comments on commit 0cfddbb

Please sign in to comment.