Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue]: If I have already index a .txt file, can I still index a .csv file and merge it to the existing graph? #784

Closed
Edwin-poying opened this issue Jul 31, 2024 · 3 comments
Labels
awaiting_response Maintainers or community have suggested solutions or requested info, awaiting filer response

Comments

@Edwin-poying
Copy link

Edwin-poying commented Jul 31, 2024

Hi, I have questions about the following situations:

  1. If I have already index a .txt file, can I still index a .csv file and merge it to the existing graph?
  2. Can I indexing both csv and txt file at the same time?

Thanks

@Edwin-poying Edwin-poying added the triage Default label assignment, indicates new issue needs reviewed by a maintainer label Jul 31, 2024
@Edwin-poying Edwin-poying changed the title [Issue]: Indexing both .txt and .csv files at the same time [Issue]: If I have already index a .txt file, can I still index a .csv file and merge it to the existing graph? Aug 1, 2024
@natoverse natoverse added awaiting_response Maintainers or community have suggested solutions or requested info, awaiting filer response and removed triage Default label assignment, indicates new issue needs reviewed by a maintainer labels Aug 1, 2024
@natoverse
Copy link
Collaborator

I don't think this will work as it currently stands. Incremental ingest (see #741) currently works because it takes advantage of the cache: you can add new files, but prior files are still read; they just get the benefit of the cache for processing to avoid expensive recompute (to a point). In this case, however, you have to update your reader config such that the ingest won't even look for those originals because they are of a different type. At the moment my suggestion would be a pre-processing script to get the new files in the same format, but I'll add a note to the incremental issue that we should take this use case into consideration.

@natoverse
Copy link
Collaborator

For the second part of your question, no, the config only supports one file type at a time.

@Edwin-poying
Copy link
Author

I see,
Thank You @natoverse

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting_response Maintainers or community have suggested solutions or requested info, awaiting filer response
Projects
None yet
Development

No branches or pull requests

2 participants