Skip to content

ZipUrl does not download the whole archive anymore

Vuillaume requested to merge zipurl into master

Fixes #63 (closed)

Benchmark using a large archive on Zenodo (6.9GB)

import requests
archive_url = 'https://zenodo.org/record/5674820/files/29November2020.zip'

%%time
r = requests.get(archive_url, allow_redirects=True)
open('archive.zip', 'wb').write(r.content)

Wall time: 9min 54s

from eossr.utils import ZipUrl

zipurl = ZipUrl(archive_url)

zipurl.extract('29November2020/DJI_0026.JPG')

Wall time: 5.21 s

Edited by Vuillaume

Merge request reports

Loading