Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]: Incorrect column info in the Dataset page of an uploaded CSV file with , in column headers #4994

Closed
liunelson opened this issue Sep 30, 2024 · 1 comment · Fixed by #5093
Closed

[BUG]: Incorrect column info in the Dataset page of an uploaded CSV file with , in column headers #4994

liunelson opened this issue Sep 30, 2024 · 1 comment · Fixed by #5093
Assignees
@dvince2
Labels
bug Something isn't working

Comments

@liunelson
Copy link
Member

Describe the bug

Question 1 of Problem 4 of the monthly demo scenarios asks the user to extract some datasets to Terarium. They can be simply downloaded from the paper website and uploaded to Terarium.

However, the Dataset page of the flow rates CSV file has two minor bugs:

  1. The data table is parsed correctly (4 columns) but the "Column information" section shows 6 columns; the issue is probably the , in the column headers.
  2. "Enrich metadata with AI" failed to provide summary info for 4 of the 6 columns in the info section, probably due to the above issue.

Screenshot 2024-09-30 at 5 47 55 PM

To Reproduce
Steps to reproduce the behavior:

  1. Upload this CSV file flow rates in the wastewater treatment plant_Total_flow.csv
  2. Go to its Dataset page
  3. Note the number of columns in the info section vs. the data table
  4. Note the lack of summary info for 4 of the columns in the info section.
@liunelson liunelson added the bug Something isn't working label Sep 30, 2024
@dvince2
Copy link
Collaborator

dvince2 commented Oct 7, 2024

Related to: #4970

@dvince2 dvince2 mentioned this issue Oct 7, 2024
Closed
dvince2 added a commit that referenced this issue Oct 8, 2024
@dvince2
[BUG]: Incorrect column info in the Dataset page of an uploaded CSV f…
28a15c3 
…ile with `,` in column headers #4994

* Changing our CSV parsing to use apache commons
* Updating pinned version of `commons-csv`, `commons-text` and adding new dependency to `commons-io` for CSV ingest.
* Moving statistics generation to `DatasetService`
* General dataset code cleanup
@dvince2 dvince2 mentioned this issue Oct 8, 2024
Merged
@dvince2 dvince2 linked a pull request Oct 8, 2024 that will close this issue
[BUG]: Incorrect column info in the Dataset page of an uploaded CSV file with , in column headers #4994 #5093
Merged
dvince2 added a commit that referenced this issue Oct 9, 2024
@dvince2
[BUG]: Incorrect column info in the Dataset page of an uploaded CSV f…
8755647 
…ile with `,` in column headers #4994

* New error messages
dvince2 added a commit that referenced this issue Oct 9, 2024
@dvince2
[BUG]: Incorrect column info in the Dataset page of an uploaded CSV f…
f1fc1aa 
…ile with `,` in column headers #4994 (#5093)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dvince2 dvince2

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
2 participants
@liunelson @dvince2