csv repair and troubleshooting

[SoftTools59654]

csv repair and troubleshooting

Is it possible to add a tool that checks the syntax of csv files and fixes those that are problematic?

In most cases, either the number is low or high, or it does not exist in that line at all. In data with millions of records, it is impossible to check manually

In data with a high number of records, at the time of conversion, if there is an error in one or more lines of a csv file, it is no longer possible to convert that file to parquet or other data because csv syntax is problematic. And no data is converted

[jqnatividad]

@SoftTools59654
You can already check if a CSV is valid with the qsv validate command.

And if the problem is readily fixable (like a CSV record having too few or too many fields), qsv validate will tell you to run the qsv fixlengths command.

Maybe, you can even combine the two commands in a simple shell script to do "auto-fixlengths"...