Skip to content

implement retries on https calls to the archive #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
KathleenLabrie merged 1 commit into from
Oct 5, 2026
Merged

implement retries on https calls to the archive #149

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion astrodata/testing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

import numpy as np
import requests
import requests.adapters
import urllib3.util # For http retries in requests
from astropy.io import fits
from astropy.table import Table
from astropy.utils.data import download_file
Expand Down Expand Up @@ -655,6 +657,13 @@ def download_from_archive(
if url is None:
url = GEMINI_ARCHIVE_URL + filename

# Set up a requests session and configure re-tries
reqses = requests.Session()
retries = urllib3.util.Retry(total=5, backoff_factor=1)
reqses.mount(
"https://", requests.adapters.HTTPAdapter(max_retries=retries)
)

# Now check if the local file exists and download if not
try:
download_it = True
Expand All @@ -670,7 +679,7 @@ def download_from_archive(
fileinfourl = url.replace(
"/file/", "/jsonfilelist/present/filename="
)
fileinfo = requests.get(
fileinfo = reqses.get(
fileinfourl, headers={"User-Agent": "astropy"}
).json()
goa_md5 = fileinfo[0].get("data_md5")
Expand Down
Loading