Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎉 Cloudflare images #4214

Merged
merged 41 commits into from
Dec 10, 2024
Merged

🎉 Cloudflare images #4214

merged 41 commits into from
Dec 10, 2024

Conversation

ikesau
Copy link
Member

@ikesau ikesau commented Nov 25, 2024

Adds support for Cloudflare Images
Drops support for Gdrive Images

Project checklist

Can be QA'd on staging, or locally with the following steps:

  1. Check this branch out
  2. yarn runDbMigrations
  3. yarn syncCloudflareImages
  4. Set the following env variables:
CLOUDFLARE_IMAGES_ACCOUNT_ID=available in cloudflare, on the main "images" page
CLOUDFLARE_IMAGES_API_KEY=generate your own with read and write permissions
CLOUDFLARE_IMAGES_URL=available in cloudflare, on the main "images" page. looks like "https://imagedelivery.net/owid-account-id/"

Images can now be created, updated, and deleted in the admin, under /admin/images

All environments share the same "prod" cloudflare images space, so it's okay to delete the images that are currently uploaded now, but eventually we'll have to be careful not to delete images in prod from our local & staging environments 😬

TODO

  • A follow-up PR that removes googleId from the images table and type definitions once this PR has proven to be stable

@owidbot
Copy link
Contributor

owidbot commented Nov 25, 2024

Quick links (staging server):

Site Dev Site Preview Admin Wizard Docs

Login: ssh owid@staging-site-cloudflare-images

SVG tester:

Number of differences (default views): 0 ✅
Number of differences (all views): 0 ✅

Edited: 2024-11-29 21:55:45 UTC
Execution time: 1.28 seconds

@ikesau ikesau requested review from larsyencken and rakyi November 26, 2024 23:59
@ikesau ikesau marked this pull request as ready for review November 26, 2024 23:59
Copy link
Contributor

@rakyi rakyi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! I'm happy we are finally doing this.

  • Too bad we didn't talk about this sooner, but I think it would make sense to use the official TS library for Cloudflare API instead of making the raw fetch requests for some more (type) safety and a nicer DX.
  • Will the existing old image URLs keep working at least for a few months, ideally indefinitely? I'm thinking about the atom feeds and newsletters, at least the DI newsletter references the image URL directly. And who knows how many links in the wild point to one of our images. I think it would be worth to spend a few hours on this if necessary.
  • This wasn't too bad, but I'd appreciate it if a feature like this was split into multiple smaller PRs in the future, which would be easier to review.

Makefile Outdated Show resolved Hide resolved
site/README.md Outdated Show resolved Hide resolved
devTools/cloudflareImagesSync/cloudflareImagesSync.ts Outdated Show resolved Hide resolved
devTools/cloudflareImagesSync/cloudflareImagesSync.ts Outdated Show resolved Hide resolved
adminSiteServer/apiRouter.ts Outdated Show resolved Hide resolved
adminSiteClient/GdocsMoreMenu.tsx Outdated Show resolved Hide resolved
adminSiteClient/ImagesIndexPage.tsx Show resolved Hide resolved
adminSiteClient/ImagesIndexPage.tsx Outdated Show resolved Hide resolved
const { admin } = useContext(AdminAppContext)
const [images, setImages] = useState<DbEnrichedImage[]>([])
const [filenameSearchValue, setFilenameSearchValue] = useState("")
const api = useMemo(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an unusual usage of useMemo. Is there a reason why we do this? It doesn't seem like we really need it, and if we do, useCallback would be IMO nicer.

@danyx23
Copy link
Contributor

danyx23 commented Nov 28, 2024

Hey there! Just a drive by comment as I got an email about Martin's review. I'd be in favour of polishing this quite a bit - I think it would be nice if we don't have to touch images for a long time after this :)

Copy link
Contributor

@larsyencken larsyencken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A really nice improvement! 🎉

Aside from small issues I slacked you about, the main piece of feedback is around possible benefits of checksums in the images table.

adminSiteServer/apiRouter.ts Show resolved Hide resolved
adminSiteServer/apiRouter.ts Outdated Show resolved Hide resolved
db/migration/1732994843041-CloudflareImagesAddUserId.ts Outdated Show resolved Hide resolved
db/migration/1731360326761-CloudflareImages.ts Outdated Show resolved Hide resolved
packages/@ourworldindata/types/src/dbTypes/Images.ts Outdated Show resolved Hide resolved
@ikesau ikesau merged commit dc8e075 into master Dec 10, 2024
21 checks passed
@ikesau ikesau deleted the cloudflare-images branch December 10, 2024 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants