migrate: support migration of FBC to latest preferred FBC

[joelanford]

This commit adds support for migrating FBC to the latest preferred FBC
contents. Note that sqlite and bundle inputs are always rendered using
the latest preferred FBC contents.

The migrate command is updated to now support FBC images and directories
as input (only sqlite was supported prior), such that the written output
will always be migrated.

The render command is updated with a --migrate flag that allows a
caller to opt into migration during rendering.

Under the hood, both of these subcommands use the action.Render struct,
which has a new Migrate boolean field that callers can use to
enable/disable the migration behavior.

Signed-off-by: Joe Lanford [email protected]

---

This is specifically useful right now to convert existing FBCs using olm.bundle.object to use of olm.csv.metadata, which can reduce catalog size significantly. At the time of the PR being opened, it reduces the operatorhub catalog from 98M to 23M (~76% reduction in size)

NOTE: This builds upon #1143, which is necessary to align on the tenet that FBC directory structure can change without changing the validity or meaning of the catalog.

Reviewer Checklist

• Implementation matches the proposed design, or proposal is updated to match implementation
• Sufficient unit test coverage
• Sufficient end-to-end test coverage
• Docs updated or added to /docs
• Commit messages sensible and descriptive

[codecov]

Codecov Report

Attention: 28 lines in your changes are missing coverage. Please review.

Comparison is base (d02b0b6) 53.79% compared to head (22e30c9) 53.83%.
Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1144      +/-   ##
==========================================
+ Coverage   53.79%   53.83%   +0.03%     
==========================================
  Files         108      108              
  Lines       10199    10224      +25     
==========================================
+ Hits         5487     5504      +17     
- Misses       3744     3746       +2     
- Partials      968      974       +6     

Files	Coverage Δ
alpha/action/migrate.go	73.33% <100.00%> (ø)
alpha/property/property.go	75.60% <100.00%> (-3.85%)	⬇️
pkg/api/api_to_model.go	69.47% <0.00%> (ø)
pkg/registry/registry_to_model.go	61.53% <0.00%> (ø)
alpha/declcfg/load.go	77.77% <50.00%> (+3.07%)	⬆️
alpha/action/render.go	67.35% <69.23%> (+0.68%)	⬆️
pkg/cache/json.go	59.18% <70.00%> (+2.15%)	⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: everettraven, joelanford

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [joelanford]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cdjohnson]

@joelanford Thanks for this PR. I think this will be very useful.

I did some testing, and I noticed that you may have inadvertently changed the default for rendering a bundle image. Is this on purpose? I would think that the --migrate option also applied to a bundle image reference.

img="icr.io/cpopen/ibm-mq-operator-bundle@sha256:11639e73211324978bde65fbe35d7f71ee2b7efd96e4d5d1c8158855256c2cae"

opmnew=~/tools/opm/dev/opm
opmold=~/tools/opm/1.26.1/opm
fbcdir=./configs/ibm-mq

# Renders olm.bundle.properties - OK
$opmold render $img > bundle-render-old.yaml

# Renders olm.csv.metadata - ERROR
$opmnew render $img > bundle-render-new.yaml

# Renders olm.csv.metadata - OK
$opmnew render --migrate $img  > bundle-render-migrate-new.yaml

# Renders olm.bundle.properties - OK
$opmold render $fbcdir > catalog-render-old.yaml

# Renders olm.bundle.properties - OK
$opmnew render $fbcdir > catalog-render-new.yaml

# Renders olm.csv.metadata - OK
$opmnew render --migrate $fbcdir > catalog-render-migrate-new.yaml

[joelanford]

# Renders olm.csv.metadata - ERROR
$opmnew render $img > bundle-render-new.yaml

You're saying that this command produces olm.csv.metadata even without --migrate=true? That's the behavior I intended. That has been the case since 1.28+.

The --migrate flag is a no-op when the inputs to render are sqlite DBs and bundles. The reasoning is essentially that opm should be converting sqlite DBs and bundles directly to the latest/best formats, rather than doing the extra work of converting them to the oldest format and then running all of the migrations.

[cdjohnson]

You're saying that this command produces olm.csv.metadata even without --migrate=true? That's the behavior I intended. That has been the case since 1.28+.

Yes, I didn't expect that. I happened to be testing with 1.26.1. I retested with 1.29.0, and you're right, and it's clear in that PR. Just missed it.

[grokspawn]

/lgtm