Chartmuseum component deprecation proposal - effective 2.6.0 or 2.7.0 (see below for full proposed timeline) - feedback welcome #15057
Replies: 35 comments 48 replies
-
|
As long as there is documentation on how to migrate over, commands/paths that changed, etc I don't personally see an issue with it. |
Beta Was this translation helpful? Give feedback.
-
|
And what about support of the ecosystem? A lot of other tools that depend on the registry didn't implement OCI support yet because it's still considered experimental, for example FluxCD (fluxcd.io) which is also a CNCF project |
Beta Was this translation helpful? Give feedback.
-
|
An alternative is to use GitLab for this. See doc. And Charts can be synced with GitLabracadabra. See doc |
Beta Was this translation helpful? Give feedback.
-
|
While I agree that Chartmuseum has quite a few merits and isn't very stable, I also see that Helms OCI Support is also not stable yet. ( concept wise) See for example: https://github.com/helm/community/blob/main/hips/hip-0006.md. It is likely that there are going to be changes on Helm side regarding OCI support, But I don't think this will affect Harbor and the storage of OCI wrapped Helm charts as such. My suggestion would be to deprecate Chartmuseum but allow users to opt in to still use Chartmuseum together with Harbor. In the meantime we can build better Chart support in Harbor. |
Beta Was this translation helpful? Give feedback.
-
|
I agree with your suggestion. until the OCI support is up to spec and stable the current chartmuseum should still be supported/available. |
Beta Was this translation helpful? Give feedback.
-
|
It would be really nice to have this change non-breaking for feed subscribers, although I don't think that's possible, as helm repo URLs have to change for OCI, right? That means all users are forced to update their helm repos. With that in mind - from a user perspective - supporting both for a transition period would be nice. |
Beta Was this translation helpful? Give feedback.
-
|
Some discussions happened in another thread before this discussion launched. Refer to the issue here: #14407 |
Beta Was this translation helpful? Give feedback.
-
|
tl;dr Until OCI support becomes stable and a first-class citizen in the Helm ecosystem, I don't think deprecating Chartmuseum is a good idea. So what is the primary reason for deprecating Chartmuseum? I've seen performance issues being called out, but it feels like a decision to be made by users whether they want to pay that cost or not. (BTW, can someone share some references describing these performance issues?) Are there any other reasons? Maintenance cost could be one, but I would say that this feature is still useful to the community. Anything else? Deprecation could be a valid decision, but performance issues and the fact that OCI is the future don't provide enough basis IMHO. Adding the fact that OCI support is not even stable yet and there are bugs reported every other week, it makes an even weaker argument. I think reaching out to the Helm/Chartmuseum maintainers would make sense at this point to gain some insights about their future plans. |
Beta Was this translation helpful? Give feedback.
-
|
I agree fully with this comment. OCI support in Helm is not great right now, so it's very premature to deprecate Chartmuseum already. |
Beta Was this translation helpful? Give feedback.
-
|
This. We heavily rely on Chartmuseum and don't want to use the experimental OCI feature until it's stable and well supported in the ecosystem. Plus it would be a larger breaking change for us as we'd have to change many build pipelines, deployment processes etc. |
Beta Was this translation helpful? Give feedback.
-
|
Would like to provide a similar feedback as @sagikazarmark, OCI support in Helm itself is still considered Experimental/Alpha, the API is in flux and there are issues constnatly being addressed (which is OK, because it's still on an early path) but more importantly it's also not broadly adopted and supported by various ecosystem tooling, in our case -- fluxcd.io, which is a popular GitOps CD tool (also a CNCF project), they do NOT intend to implement OCI support until it's out of the experimental phase: This is a response I saw today from one of the project (FluxCD) maintainers in their slack. I think the move to OCI is inevitable and I like that Harbor supports it already which allows projects to test it and certify their tooling around it, but it's quite clear that OCI is not ready for mainstream adoption and relying only on the fact that AFAIK, Chartmuseum is an active project and is actively maintained, I also don't see any statement about them closing shop or deprecating it any time soon. |
Beta Was this translation helpful? Give feedback.
-
|
I tend to agree with what has been said by multiple people before : though I understand and agree with the objective in the long terme, the actual deprecation of chartmuseum should only happen a few month after the Helm ecosystem will have fully adapted the OCI standard which is not yet the case yet. |
Beta Was this translation helpful? Give feedback.
-
|
There are a couple of things that surprises me with this. I'm surprised that the deprecation of standard Helm repos are deprecated in a minor release. I think this is a breaking change, since the outwards behavior will be completely different. So this should be indicated with an increment of major version number, i.e. Harbor 3.0. And as already mentioned in this discussion, OCI support in Helm is not considered a stable feature yet, so Helm may change their implementation of it that will cause further breaking changes down the line for Harbor. Considering this, I'm surprised that the Harbor maintainers are pitching the deprecation of Chartmuseum as a reduced maintenance cost, since nobody knows what changes Helm will do to its OCI support. It's not even settled for sure that they will use OCI in the future, even if it sure looks like it right now. I feel this decision from Harbor is premature. As far as I can tell, there haven't been any coordination with the Helm maintainers on this. That also surprises me, since it should be very valuable to Harbor and its users to make sure we are in step with Helm on this particular issue. |
Beta Was this translation helpful? Give feedback.
-
|
The more I think about it, the more certain I feel that this is a breaking change. So this must be Harbor 3.0, not just 2.4. At least if we pretend to follow SemVer. Furthermore, when experimenting with the OCI support, I found an issue. There's now risk for name collisions between Charts and Docker images, which is not terribly nice. It's not very strange to imagine a scenario where the Helm chart Are there any recommended naming schemes from your side to avoid collisions, or is this just left to the users to figure out? |
Beta Was this translation helpful? Give feedback.
-
|
For anyone curious, here's the main PR for maturing the OCI support in Helm: helm/helm#9782 |
Beta Was this translation helpful? Give feedback.
-
|
Harbor 2.4 release notes says "Limit support on Chartmuseum from v2.4.0 release.". What does that exactly mean? |
Beta Was this translation helpful? Give feedback.
-
|
It means that they are marking it for removal, but won't be removing it until a later release. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, we want to release the signal to community that the chartmuseum will be deprecated in future release, and engage user to leverage the OCI compatible chart. In the currently release, we'll not provide deep tech support on Chartmuseum problem. |
Beta Was this translation helpful? Give feedback.
-
|
mind that helm still has no plan to release OCI just yet: |
Beta Was this translation helpful? Give feedback.
-
|
I would also like to vote against the deprecation of chartmuseum on 2.4.x. We are using harbor 2.4.0 (and we want to upgrade to 2.4.1 and later when possible) and use FluxCD and it does not (yet?) support pulling charts from an OCI URL. |
Beta Was this translation helpful? Give feedback.
-
|
This should be no problem as flux will also move to OCI once it becomes GA just like Harbor. |
Beta Was this translation helpful? Give feedback.
-
|
I wasn't aware, and those are good news. But it still seem to be a bit far away from flux to fully (or properly) support OCI , and Harbor is talking about removing chartmuseum support in a very near future.... and that is the issue :-( |
Beta Was this translation helpful? Give feedback.
-
|
Chartmuseum will be removed from Harbor after the OCI support in Helms becomes GA. Deprecate does not mean remove! |
Beta Was this translation helpful? Give feedback.
-
|
Good to ear... still there are other tools that maybe will still need some time after helm goes GA with OCI support (flux, etc)... I know it cannot wait for all of them, nor it should, but just have this in mind. But thanks for answering, it's already very good to know this. |
Beta Was this translation helpful? Give feedback.
-
|
Helm v3.8 changelog states that OCI registry support has graduated out of being an experiment! https://github.com/helm/helm/releases/tag/v3.8.0 🔥 ImportantWe are very well aware that many Harbor users depend on Chartmuseum because of various CI/CD tools and deployment workflows and can not switch just like that. To help a smooth transition, please:
|
Beta Was this translation helpful? Give feedback.
-
|
Hi @atrendafilov - we have voted and implemented that while ago please read the full discussion and voting |
Beta Was this translation helpful? Give feedback.
-
|
@atrendafilov we actually hit this issue this week. We updated our harbor install and all of a sudden our repo links are red with errors in rancher. We had to roll back to a version of the harbor chart that still had Chartmuseum apart of it. Do you know an eta of OCI repo support for rancher? @OrlinVasilev my personal opinion on this "vote" is that you all mostly subjected it around helm support and not anything else that depends on it, like rancher. Focused on users pushing to harbor, but not about systems/infrastructure pulling from it. |
Beta Was this translation helpful? Give feedback.
-
|
@johnwc - I understand your point, but this voting was open for everyone who uses Harbor and leverages services from it. The decision was made on the perception that we cannot ship something that we cannot support within Harbor If you read the discussion you will understand what I mean. We depend on many other projects like Chartmuseum, and if they are not supported and will potentially bring instabilities, and problems we have to mitigate that issue. Also the hole initiative to have Helm Charts bundled in OCI was on fire at the time :) |
Beta Was this translation helpful? Give feedback.
-
|
@OrlinVasilev open to everyone who uses harbor, did you email everyone that uses harbor so they knew about this vote? See how silly that sounds? It was open to everyone who knew about the vote... On fire I agree with, because as of now harbor is pretty much burnt toast to us unless rancher adds support for OCI. Our Kubernetes infrastructure management has higher priority over a container image repository application. We are hoping suse adds support for OCI, otherwise we will have to ditch harbor for something else that allows chart repos. I'm pretty sure we are not alone in this scenario. |
Beta Was this translation helpful? Give feedback.
-
|
@johnwc I'm not here to argue at all and you have a point about the bad communication toward the community, about Rancher and OCI I think that's a must anyway but who am I to say :) maybe @atrendafilov can comment on that! From the Harbor side we can provide support for SUSE and Rancher community! |
Beta Was this translation helpful? Give feedback.
-
|
There is an issue reported in the helm which stated it will not be possible to create a reproducible OCI chart image without a nasty hack. As a result, when we push the same OCI chart many times, Harbor duplicates it and potentially the disk space. |
Beta Was this translation helpful? Give feedback.
-
|
To whom it may concern-
|
Beta Was this translation helpful? Give feedback.
-
|
Will the harbor UI still differentiate between helm charts and images? I disabled chartmuseum and I see the helm charts tab has disappeared. I think that might be a useful feature, especially when a project might have tons of images deployed with just one helm chart (so it doesn't get buried). |
Beta Was this translation helpful? Give feedback.
-
|
It doesn't. What we did in my team is to prefix all charts with This, of course, is just a convention and could be broken if we're not careful. Also, images must not have |
Beta Was this translation helpful? Give feedback.
-
|
Did you create a feature request. IMHO this is a sensible requirement and forward to implement. |
Beta Was this translation helpful? Give feedback.
-
|
I have not. Please go ahead, and link it here! :) |
Beta Was this translation helpful? Give feedback.
-
|
Hi All, Can you provide some news about this:
BR, Damien |
Beta Was this translation helpful? Give feedback.
-
|
2.6 |
Beta Was this translation helpful? Give feedback.
-
|
2.6 |
Beta Was this translation helpful? Give feedback.
-
|
2.6 |
Beta Was this translation helpful? Give feedback.
-
|
Closing the vote early - we have more than 50%+1 votes for 2.6. Vote for deprecation starts immediately and ends at Sunday August 14 1700 UTC or when six votes for +1 (deprecate) or -1 (don't deprecate) by maintainers have been attained. |
Beta Was this translation helpful? Give feedback.
-
|
+1 |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum. |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum |
Beta Was this translation helpful? Give feedback.
-
|
+1 for 2.6 to deprecate Chartmuseum |
Beta Was this translation helpful? Give feedback.
-
|
A majority of Harbor maintainers have voted to deprecate the Chartmuseum feature in the 2.6.0 release, so it is hereby deprecated with the release of Harbor v2.6.0. This means that no fixes or enhancements will be delivered for v2.60. forward for the feature except for fixes to catastrophic failures - bugs which cause Harbor not to work at all -n and to critical security issues - vulnerabilities with CVSS scores of 9.0 or greater. The development team can begin to remove the feature in v2.8.0 (or its equivalent if the major version number changes) or later at their convenience. Users are encouraged to migrate their use of Helm chart storage and retrieval to the supported OCI method as soon as possible. |
Beta Was this translation helpful? Give feedback.
-
|
How that plan impacts the chart replication feature, e.g., from ArtifactHub to Harbor? If I'm not mistaken, this replication path requires ChartMuseum. Will the ArtifactHub support be improved to allow this replication to work with OCI, assuming ArtifactHub supports the distribution of Helm charts as OCI artifacts (does it?)? IMO asking users to set up external periodic jobs to convert Helm charts that came from ArtifactHub to OCI is not a very good experience, and might even introduce other operational issues. Is there anything that can be done to improve upon this? |
Beta Was this translation helpful? Give feedback.
-
|
@danielfm if I'm not wrong but artifacthub.io is like aggregator for charts not a registry, so you cannot replicate from artifacthub but from the original OCI like Harbor or amazon ecr or google artifact registry |
Beta Was this translation helpful? Give feedback.
-
|
Makes sense! However, even very popular charts do not expose an OCI registry that we can use for replicating those into Harbor. At least, not that I could find. Don't the maintainers think that removing Chartmuseum at this point to be a little extreme, considering the chart replication feature (which is one of the reasons we adopted Harbor, and I think there are others in the same place, as this is a very useful feature) relies on Chartmuseum and, as of now, very few charts are distributed via an OCI registry? Apart from Helm supporting OCI, is there any other reason as to why Chartmuseum is being deprecated? What deprecating/removing Chartmuseum now is going to fix or improve, apart from cleaning up some code? Is this worth the downsides brought up by everyone in this thread? IMO Maybe at least there should be a clear/supported migration path in place before Chartmuseum is removed. |
Beta Was this translation helpful? Give feedback.
-
|
@Vad1mo @wy65701436 @qnetter @reasonerjt can you comment on that please "what's the reason behind removing chartmuseum? , I'm missing the background , personally I find it odd as well :) |
Beta Was this translation helpful? Give feedback.
-
|
@danielfm got the answer on the last community meeting: https://youtu.be/ck51xJdxY5I?t=2149 |
Beta Was this translation helpful? Give feedback.
-
|
Awesome! That would solve the issue for us. 👏🏼 We already have some internal charts that are stored in Harbor in OCI format, so other services we use internally (i.e. ArgoCD, Terraform in some cases) already support charts in this format, so the remaining issue would be to continue to replicate external charts in ChartMuseum format into Harbor, and keeping this feature in future Harbor releases would be important IMO. |
Beta Was this translation helpful? Give feedback.
-
|
mind that OCI misses out on some (supposedly) popular features of "traditional" helm chart servers.
|
Beta Was this translation helpful? Give feedback.
-
|
Yes you are right. This puzzled me to see that chart were treated as any other OCI artifacts. I didn't test the For those who are curious about that OCI artifacts thing, I found the following article worth reading: https://dlorenc.medium.com/oci-artifacts-explained-8f4a77945c13 |
Beta Was this translation helpful? Give feedback.
-
|
As stated in helm/helm#9983
So as @horihel said no support for |
Beta Was this translation helpful? Give feedback.
-
|
I have been following this heated discussion for a while, what would be your opinion regarding the compromise of supporting the replication from ChartMuseum? Take a look at the proposal #18098 |
Beta Was this translation helpful? Give feedback.
-
|
@danielfm check this ^^ |
Beta Was this translation helpful? Give feedback.
-
|
Why would you remove support for chartmuseum when OCI is not a full replacement and does not even implement simple features like indexing and search is beyond me. Why not just leave it but disabled by default? So at least in my POV to protect me from a few ms of perf issues, you created a few minutes of jumping around to check the status of the releases, check the versions and update. |
Beta Was this translation helpful? Give feedback.
-
|
Hello @l-maia we appreciate your feedback, but as maintainers and overall responsible for the stability of the project it was voted and decided chartmuseum to be removed! I'm sorry to hear that this may bring some inconvenience to you we are here to support you in your migration please join community meetings if you need help with that! |
Beta Was this translation helpful? Give feedback.
-
|
Thank you but i think i won't be needing help with migration, just thought it is very strange harbor is not suporting helm repo with something like chartmuseum or any other alternative, but then it is using it on goharbor.io. |
Beta Was this translation helpful? Give feedback.
-
|
I understand that the project wants to remove ChartMuseum for some reasons, but can we keep some important functionality. I think the two functions of index.json and search are very important to helm. Can we add these two functions in harbor-core; delete them when there are alternatives in the future. |
Beta Was this translation helpful? Give feedback.
-
|
It seems to have been completely removed now. $ grep _version harbor.yml
_version: 2.8.0
$ ./install.sh --help
Note: Please set hostname and other necessary attributes in harbor.yml first. DO NOT use localhost or 127.0.0.1 for hostname, because Harbor needs to be accessed by external clients.
Please set --with-notary if needs enable Notary in Harbor, and set ui_url_protocol/ssl_cert/ssl_cert_key in harbor.yml bacause notary must run under https.
Please set --with-trivy if needs enable Trivy in Harbor.
Please do NOT set --with-chartmuseum, as chartmusuem has been deprecated and removed.According to the actual test, the |
Beta Was this translation helpful? Give feedback.
and others
-
As Harbor has been an OCI compatible registry and charts can be managed with OCI compatible way. Additionally, the current
Chartmuseumcomponent brings many perf issues. So, Harbor is planning to deprecate it in the coming V2.4 release. The community and maintainers want to hear more feedback about this decision.Beta Was this translation helpful? Give feedback.
All reactions