Policy on adding new image formats?

[kornelski]

Is there a documentation of what image formats and what implementations are acceptable for this library?

From discussions in issues/PRs it seems that native-Rust implementations are strongly preferred/required.

What's the bar for adopting new codecs? Currently the library supports (by default!) some niche ones like QOI and farbfeld, and old crufty netpbm. Is the goal to support everything, or is there a bar of usefulness a format has to meet?

Should new implementations be contributed directly to the image crate? Can/should they be released as independent crates first?

---

A related issue: #2363

[fintelia]

At the moment we don't have a written policy, but we probably should. I've also been wondering whether we should have a better story for dealing with additional image formats externally to this crate, so I might start a separate discussion on that point [edit: see #2368].

To summarize the current state of things...

Default vs. optional formats

• Default formats should be written in Rust and compile with MSRV rustc without needing you to install any system libraries or do custom configuration
• So far, non-default formats have only been allowed when there is significant demand for a format, but the previous point cannot be achieved.

De-facto requirements for adding formats

• Has to be an image file format, so for instance, DXT block compression doesn't qualify.
• We have only one decoder for a given format. (This hasn't been without controversy, but has certainly helped our sanity.)
• Format needs to be well-defined and in reasonably widespread use. (Probably the most fuzzy requirement)
• Codec shouldn't have been written using using secret or paywalled information. Also can't have problematic legal restrictions like patents or a non-OSS license. (We don't have legal counsel, so try to be particularly cautious here)
• The decoder implementation needs to be sufficiently complete, or the format important enough to justify a partial implementation.
• (Probably other things I'm forgetting)

Other thoughts

• As image has grown in popularity, the question of supply chain security has started to weigh on me more heavily. We frequently have first-time or comparatively new contributors make PRs to add dependencies on crates they control. Sometimes this is purely innocent, but I can't shake the feeling that this is also exactly what a supply chain attack would look like.
• I've given up on fuzzing this crate for now because so many of our existing decoders don't properly handle untrusted input. But if we add new formats faster than we deal with fuzzing for our existing ones, then it'll be impossible to ever catch up.
• It is substantially more difficult to remove a format than to add one. My personal suspicion is that farbfeld is essentially unused and probably shouldn't have been added, but assessing usage and going through a deprecation cycle hasn't felt worth the time.
• Each new format carries a small but non-trivial maintenance burden forever. Every time we add a new feature like orientation metadata, we have to go through all the formats and handle it appropriately. Any codec is fair game for optimization, bug fix, or refactoring PRs. It adds one more combination to test in CI. etc.
• There's probably dozens of formats from the '80s and '90s that basically amount to bitmap + bespoke RLE compression scheme. None of them are good, many have weird quirks, and sometimes I wish folks would just let them die. Admittedly a few like BMP, TGA and HDR are still common enough that we probably have to keep supporting them.

Since these sorts of decisions can be controversial, I should probably also add that nothing here is an agreement to add or reject any specific format just because it meets the criteria listed.

[kornelski]

If the less-common formats were behind off-by-default feature flags and in separate crates, it'd be possible to see their download numbers and which crates are explicitly enabling them.

[fintelia]

That's an interesting strategy, though I wonder if a bunch of users would enable the formats merely because they were available. Already both Bevy and zune-image specifically chose to support farbfeld, but I'm not convinced that either case was based on any evidence of user interest...

If anything, our having the format may have led other folks to also waste their time dealing with it.

[sorairolake]

If possible, I think it would be better to split all image formats into external crates.