Skip to content

Commit

Permalink
Atomic download of files
Browse files Browse the repository at this point in the history
Prevents serving of partial downloads, especially after early-exit
  • Loading branch information
EpicWink committed Nov 15, 2024
1 parent f3a77a0 commit b0bdbc2
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion src/proxpi/_cache.py
Original file line number Diff line number Diff line change
Expand Up @@ -759,9 +759,11 @@ def _download_file(self, url: str, path: str):
return
parent, _ = os.path.split(path)
os.makedirs(parent, exist_ok=True)
with open(path, "wb") as f:
download_path = path + ".download"
with open(download_path, mode="wb") as f:
for chunk in response.iter_content(chunk_size=16 * 1024):
f.write(chunk)
os.replace(download_path, path)
key = self._get_key(url)
self._files[key] = _CachedFile(path, os.stat(path).st_size, 0)
logger.debug(f"Finished downloading '{url_masked}'")
Expand Down

0 comments on commit b0bdbc2

Please sign in to comment.