Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOP-5189: Proxy requests to raw GH content #635

Merged
merged 17 commits into from
Nov 21, 2024
3 changes: 2 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ on:
push:
tags:
- 'v[0-9]*'
- '*'
rayangler marked this conversation as resolved.
Show resolved Hide resolved

jobs:
test:
Expand Down Expand Up @@ -35,7 +36,7 @@ jobs:
path: dist/${{ steps.build_package.outputs.package_filename }}

release:
if: startsWith(github.ref, 'refs/tags/v')
if: startsWith(github.ref, 'refs/tags')
needs: test
runs-on: ubuntu-latest
steps:
Expand Down
14 changes: 11 additions & 3 deletions snooty/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -324,13 +324,21 @@ def get(
datetime.timedelta(hours=1) if cache_interval is None else cache_interval
)

url_netloc = urllib.parse.urlparse(url).netloc
is_raw_gh_content_url = url_netloc == "raw.githubusercontent.com"
target_url = (
f"https://deploy-preview-1310--docs-frontend-stg.netlify.app/.netlify/functions/fetch-url?url={url}"
rayangler marked this conversation as resolved.
Show resolved Hide resolved
if is_raw_gh_content_url
else url
)

if self.cache_dir is None:
res = requests.get(url)
res = requests.get(target_url)
res.raise_for_status()
return res.content

# Make our user's cache directory if it doesn't exist
filename = urllib.parse.quote(url, safe="")
filename = urllib.parse.quote(target_url, safe="")
self.cache_dir.mkdir(parents=True, exist_ok=True)
inventory_path = self.cache_dir.joinpath(filename)

Expand All @@ -350,7 +358,7 @@ def get(
mktime(mtime.timetuple()), usegmt=True
)

res = requests.get(url, headers=request_headers)
res = requests.get(target_url, headers=request_headers)

res.raise_for_status()
if res.status_code == 304:
Expand Down