Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mime-type checking; better panic handling #2304

Merged
merged 8 commits into from
Nov 21, 2024

Conversation

jqnatividad
Copy link
Collaborator

beyond just checking the file extension, actually go the extra mile and do mime-type inferencing.
can be turned off by setting QSV_SKIP_FORMAT_CHECK env var.

Leveraged the file-format crate that is already being used by qsv for the sniff command.

Also, added better panic UI/UX with human-panic crate.

resolves #2301

that can be suppressed with QSV_SKIP_FORMAT_CHECK env var
with a better UI/UX
warning: immediately dereferencing a reference
    --> src/util.rs:1995:54
     |
1995 |             match simd_json::serde::from_slice(&mut **&mut s_slice) {
     |                                                      ^^^^^^^^^^^^^ help: try: `s_slice`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#deref_addrof
     = note: `#[warn(clippy::deref_addrof)]` on by default

warning: immediately dereferencing a reference
    --> src/util.rs:2139:54
     |
2139 |             match simd_json::serde::from_slice(&mut **&mut s_slice) {
     |                                                      ^^^^^^^^^^^^^ help: try: `s_slice`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#deref_addrof

warning: deref which would be done by auto-deref
    --> src/util.rs:1995:48
     |
1995 |             match simd_json::serde::from_slice(&mut **&mut s_slice) {
     |                                                ^^^^^^^^^^^^^^^^^^^ help: try: `&mut s_slice`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#explicit_auto_deref
     = note: `#[warn(clippy::explicit_auto_deref)]` on by default

warning: this expression creates a reference which is immediately dereferenced by the compiler
    --> src/util.rs:2126:63
     |
2126 |         csv_to_jsonl(&tempfile_path, &get_stats_data_types(), &statsdatajson_path)?;
     |                                                               ^^^^^^^^^^^^^^^^^^^ help: change this to: `statsdatajson_path`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow
     = note: `#[warn(clippy::needless_borrow)]` on by default

warning: deref which would be done by auto-deref
    --> src/util.rs:2139:48
     |
2139 |             match simd_json::serde::from_slice(&mut **&mut s_slice) {
     |                                                ^^^^^^^^^^^^^^^^^^^ help: try: `&mut s_slice`
     |
     = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#explicit_auto_deref
@jqnatividad jqnatividad merged commit 15d00aa into master Nov 21, 2024
14 of 15 checks passed
@jqnatividad jqnatividad deleted the 2301-check-mime-type_better-panic-handling branch November 21, 2024 21:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

crash on binary data, native support for compressed csv?
1 participant