Skip to content
This repository has been archived by the owner on Feb 17, 2024. It is now read-only.

Reloading indices from hdfs generates corrupted lucene files if hdfs lags #90

Open
phymbert opened this issue Sep 28, 2021 · 0 comments · May be fixed by #92
Open

Reloading indices from hdfs generates corrupted lucene files if hdfs lags #90

phymbert opened this issue Sep 28, 2021 · 0 comments · May be fixed by #92
Assignees
@phymbert
Labels
bug Something isn't working
Milestone
0.2.1

Comments

@phymbert
Copy link
Owner

Reloaded partition failed to be opened by the search reader because indices are corrupted.
Solution: add an optional step to download file from hdfs locally before unzipping it
@phymbert phymbert added the bug Something isn't working label Sep 28, 2021
@phymbert phymbert added this to the 0.2.1 milestone Sep 28, 2021
@phymbert phymbert self-assigned this Sep 28, 2021
phymbert added a commit that referenced this issue Sep 28, 2021
@phymbert
Fixes #90 Option to download indices files locally from hdfs first
d79ea0c
@phymbert phymbert linked a pull request Sep 28, 2021 that will close this issue
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@phymbert phymbert

Labels
bug Something isn't working
Projects
None yet
Milestone
0.2.1
Development

Successfully merging a pull request may close this issue.

#90 #91 hdfs copy to local & temporary files phymbert/spark-search
1 participant
@phymbert