Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOP-5189: Proxy requests to raw GH content #635

Merged
merged 17 commits into from
Nov 21, 2024
14 changes: 11 additions & 3 deletions snooty/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -324,13 +324,21 @@ def get(
datetime.timedelta(hours=1) if cache_interval is None else cache_interval
)

url_netloc = urllib.parse.urlparse(url).netloc
is_raw_gh_content_url = url_netloc == "raw.githubusercontent.com"
target_url = (
f"https://docs-frontend-stg.netlify.app/.netlify/functions/fetch-url?url={url}"
if is_raw_gh_content_url
else url
)

if self.cache_dir is None:
res = requests.get(url)
res = requests.get(target_url)
res.raise_for_status()
return res.content

# Make our user's cache directory if it doesn't exist
filename = urllib.parse.quote(url, safe="")
filename = urllib.parse.quote(target_url, safe="")
self.cache_dir.mkdir(parents=True, exist_ok=True)
inventory_path = self.cache_dir.joinpath(filename)

Expand All @@ -350,7 +358,7 @@ def get(
mktime(mtime.timetuple()), usegmt=True
)

res = requests.get(url, headers=request_headers)
res = requests.get(target_url, headers=request_headers)

res.raise_for_status()
if res.status_code == 304:
Expand Down