[Sandbox] KitOps

[gorkem]

Application contact emails

[email protected], [email protected]

Project Summary

Securely store and share AI/ML artifacts as OCI artifacts in a container registry.

Project Description

AI and ML are becoming integral components of cloud-native application architecture. However, the current AI/ML domain faces significant challenges due to fragmented packaging and versioning of not only models, but their associated datasets, codebases, and parameters. These problems are largely solved for cloud native applications thanks to standards like git and OCI that are used across all tools and infrastructures.

Unlike microservices that are loosely coupled, AI models have tight couplings with the datasets that trained and validated them, as well as their weights, parameters, inputs, and outputs. This complexity necessitates an improvement on OCI packaging formats that embraces these dependencies to help organizations manage, secure, and audit AI/ML projects effectively.

KitOps is an open-source packaging and versioning solution for AI/ML projects, designed for enterprise speed, security, and consistency in cloud-native environments. By offering an OCI artifact called ModelKit, KitOps leverages the widely adopted OCI standard to ensure seamless integration with existing tools and infrastructures. This enables organizations to manage, secure, and audit AI/ML projects effectively. KitOps has already been downloaded tens of thousands of times and is in use in private and public sector organizations.

KitOps allows users to:

1. Selectively bundle models, datasets, codebases, and documentation into a single ModelKit package, ensuring reproducibility and version control.
2. Version, sign, and store ModelKits in an OCI-compatible registry, facilitating collaboration and deployment alongside other cloud-native applications.
3. Save resources and time by extracting only the necessary layers for each phase of the project lifecycle, optimizing resource utilization.

Org repo URL (provide if all repos under the org are in scope of the application)

N/A

Project repo URL in scope of application

https://github.com/jozu-ai/kitops

Additional repos in scope of the application

https://github.com/jozu-ai/pykitops
https://github.com/jozu-ai/gh-kit-setup
https://github.com/jozu-ai/daggerverse
https://github.com/jozu-ai/homebrew-kitops

Website URL

https://kitops.ml/

Roadmap

https://github.com/jozu-ai/kitops/blob/main/ROADMAP.md

Roadmap context

The current KitOps roadmap is concentrated on adoption, usability and security features. However, we expect to introduce more items around standardization as part of the CNCF community.

Contributing Guide

https://github.com/jozu-ai/kitops/blob/main/CONTRIBUTING.md

Code of Conduct (CoC)

https://github.com/jozu-ai/kitops/blob/main/CODE-OF-CONDUCT.md

Adopters

No response

Contributing or Sponsoring Org

jozu.com

Maintainers file

https://github.com/jozu-ai/kitops/blob/main/MAINTAINERS.md

IP Policy

• If the project is accepted, I agree the project will follow the CNCF IP Policy

Trademark and accounts

• If the project is accepted, I agree to donate all project trademarks and accounts to the CNCF

Why CNCF?

KitOps is built upon the Open Container Initiative (OCI) standard, which is central to many CNCF projects such as Kubernetes, container runtimes, and registries. By extending OCI principles to the packaging and versioning of AI/ML projects, KitOps directly contributes to the cloud-native ecosystem’s growth and evolution.

Joining the CNCF Sandbox offers the following advantages:

• Promoting Open Standards in AI/ML: As AI becomes an enduring component of cloud-native applications, there’s a critical need for standardized tools and infrastructure. CNCF’s stewardship ensures that KitOps can contribute to, and benefit from, a governance model that prioritizes open standards and interoperability.

• Leveraging CNCF’s Ecosystem: With OCI at its core, KitOps naturally integrates with CNCF projects like Kubernetes, Harbor, and Dragonfly, enabling seamless deployment and management of AI/ML workloads in cloud-native environments.

• Fostering Collaboration and Innovation: Hosting KitOps within CNCF provides a platform for collaboration with a vibrant community of developers, end-users, and vendors. This collaborative environment accelerates innovation and adoption of best practices.

• Addressing Industry Challenges Together: The tight coupling of AI models with datasets and dependencies presents unique challenges. By collaborating within CNCF, KitOps can help develop solutions that address these complexities, benefiting the broader community.

• Alignment with CNCF’s Mission: KitOps shares CNCF’s goal of making cloud-native computing universal and sustainable. By standardizing AI/ML packaging, we aim to reduce fragmentation and promote interoperability across the cloud-native landscape.

Given the central importance of OCI to CNCF projects and the shared vision of promoting open, cloud-native technologies, CNCF is the most logical and natural foundation for hosting this emerging cloud native tooling for AI/ML. We believe that by joining forces with CNCF, KitOps can play a pivotal role in shaping the future of cloud-native AI/ML, driving innovation, and delivering value to organizations worldwide.

Benefit to the Landscape

The current CNCF ecosystem lacks a standard for packaging and versioning AI/ML projects, particularly for seamless integration with CNCF projects like Kubernetes. This absence creates challenges for organizations seeking to deploy AI/ML workloads in cloud-native environments, leading to fragmentation and inefficiencies.

KitOps bridges this critical gap by introducing a clear, open standard for AI/ML project packaging, versioning, and sharing that enhances and extends the existing Open Container Initiative (OCI) standard. By leveraging OCI, KitOps ensures compatibility and interoperability with the tools and infrastructures that organizations already use, promoting a unified approach within the CNCF landscape.

Key benefits KitOps brings to the CNCF ecosystem include:

• Facilitating AI/ML Adoption: KitOps simplifies the integration of AI/ML projects with Kubernetes and other CNCF technologies, smoothing the path for organizations to incorporate AI capabilities into their cloud-native applications.

• Enhancing Interoperability: By building upon the OCI standard, KitOps ensures that AI/ML packaging works harmoniously with existing containerization practices, registries, and deployment workflows. This avoids redundancy and leverages familiar tooling.

• Promoting Open Standards: KitOps provides an open, transparent alternative to proprietary AI/ML packaging solutions. This empowers organizations to adopt open-source AI/ML models and approaches, fostering innovation and collaboration within the community.

• Driving Ecosystem Growth: Introducing a standardized AI/ML packaging solution can stimulate the development of new tools, services, and best practices within the CNCF ecosystem, accelerating the growth of cloud-native AI/ML technologies.

Additionally, as AI becomes increasingly central to cloud-native applications, the need for standardized, open approaches is paramount. KitOps addresses this need by offering a solution that is:

• Open and Extensible: Encouraging community contributions and extensions, fostering a vibrant ecosystem around AI/ML packaging standards.

• Transparent and Auditable: Providing organizations with the ability to manage, secure, and audit their AI/ML projects effectively, enhancing trust and compliance.

• Complementary to Existing CNCF Projects: Enabling seamless integration with projects like Kubernetes, OpenTelemetry, Harbor and others, without introducing conflicts or redundancies.

Cloud Native 'Fit'

KitOps fits well into the Container registry in the CNCF Landscape as it aims to enable and standardize packaging AI/ML projects for OCI registries. I also interact well with other tools in the Cloud Native AI (CNAI) project area, as well as with projects like Kubernetes, CRI-O, containerd, Kserve, and others.

Cloud Native 'Integration'

KitOps depends on OCI registries like Harbor, it complements projects such as Kubernetes, containerd/CRI-O for inference and training of AI/ML workloads.

Cloud Native Overlap

N/A

Similar projects

N/A

Landscape

No

Business Product or Service to Project separation

N/A

Project Domain Technical Review

Not yet

CNCF Contacts

Andrew Block @sabre1041
Chris Aniszczyk @caniszczyk

Additional information

No response

[dims]

@gorkem i like a lot of what i see here. I do have a quesiton about kit dev start (which uses llamafile) , it feels like it does not belong here as a built-in command. It feels like a sample/example application that can be built to show case the capabilities of the rest of the project. Am i reading this right? thanks.

[gorkem]

@dims Thank you so much for taking the time to explore our project!

You’re absolutely right that the dev command serves a different audience. Our goal with integrating kit dev start was to provide application developers with a tool to experiment and build with ModelKits seamlessly. This feature is still an ongoing experiment, and we’re keen to see how the community engages with it.

We understand that the space for local LLM tools is crowded. Our vision for the dev command is to create a development tool for all kind of models including LLMs and making the transition from development to production smoother and more efficient. Depending on the community’s interest and feedback, we can evolve it into a separate tool or even retire it completely.