Download images or resources locally

[Exagone313]

Summary

Images in feeds are loaded as provided in the feeds. This leads to a potential privacy leak, I've seen feeds using these for analytics. Also, this prevents the use of Content-Security-Policy which reduces the XSS attack surface.

Proposed changes

Add an option to download all images in feeds locally (or, all resources).

Also keep these images for starred articles. Too old images should be cleared if articles are not saved (correct me if I'm wrong, I believe it only keeps starred articles).

Quirks

• Images (or resources) may be big, it may be useful to compute thumbnails, or to do not download these. I believe videos and audios could be downloaded from user action.
• Images may not be actual images, it should be useful to open these with an appropriate extension (gd, imagick...) to check that.
• There are other kind of third-party resources, like iframes.

Additional information

I'm currently using another RSS aggregator but I try to look at alternatives, but I find that feature mandatory.

See also #100 (storage of favicons).

• I have read the CONTRIBUTING.md and followed the provided tips

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[SMillerDev]

Sounds interesting, I look forward to reviewing your pull request.

[Exagone313]

Before starting to look in the code, I've read the steps for installing the app with git (+ running make which runs composer and npm). Is it possible to replace the downloaded build seamlessly without data loss from one version to another (except with potential database migration) ? I don't have data to loose now, but I'll probably migrate to my development version before changes hit a release.

I have no knowledge in Nextcloud (app-related) source code, but I probably can do something.

[Grotax]

I switched between git versions and releases a few times, no problems.
If you change the the migrations you obviously can't go back to an older version.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[Exagone313]

It's not completely dead, I started looking at the code.

My road map:

• Add an administration setting (defaults to false) to enable downloading resources
  • Not yet tested
• Explore the code, establish what to do:
  • get all img[src] when calling Db\Item->setBody, save resources
  • get all img[src] in bodies when calling ->jsonSerialize or ->toAPI, save resources if still unknown, replace bodies with new URIs
• Get resources URL from all feeds
• Find how and where to download these resources
• Edit feed items to use downloaded resources
• Sanitize, resize, etc... downloaded resources

[Exagone313]

I was adding a client-side (per-user) setting, but it would make more sense to have it set globally, by server admin. How could I do this?

[Exagone313]

Nevermind, I didn't see there was already a setting page in admin panel.

[Exagone313]

I saw that HTMLPurifier is used to whitelist just a few iframe URIs (Youtube, Vimeo, VK) and that audio and video tags are forbidden.

So, support for these resources is out of the scope of this feature and may be worked on if this gets completed. I'll only support img tags.