Kubeflow just applied to join CNCF – what does it mean for you?

Google just announced that they have submitted an application for Kubeflow to become an incubating project in the Cloud Native Computing Foundation (CNCF). It is an initiative supported by the Kubeflow Project Steering group. The request is visible to everyone and it represents a game changer for the rhythm which Kubeflow will develop. It makes community growth a strategic objective and puts Kubeflow on a development fast track.

A brief history of Kubeflow

Kubeflow is an open-source project that started in 2017. Kubeflow 1.0 was released three years later, reaching stable versions for a set of applications. In total, the project gathered more than 200 contributors and has seen more than 20,000 stars on GitHub. Some of the components of the project became mature enough to be treated independently, such as KServe, which is simply  Kubeflow’s serving component.

Over the six releases, multiple organisations contributed to Kubeflow. Charmed Kubeflow is one of the official distributed repositories since its early stage. We also contribute back to the community, reporting bugs, addressing concerns and leading the latest release, Kubeflow 1.7.

What is Kubeflow?

Kubeflow is an open-source end-to-end MLOps platform that transforms machine learning workflows into data science jobs. It addresses key steps of the ML lifecycle such as model development with Kubeflow Notebooks, model serving with KServe, or model training with Pipelines.

Charmed Kubeflow follows the same principles as the upstream project. It is integrated with more ML tools, such as MLFlow for model registry or Seldon Core for model serving.  It is a  production-grade tool, ready to be deployed and used by big teams, simplifying data scientists’ and machine learning engineers’ jobs.

What is CNCF?

Cloud Native Computing Foundation (CNCF) is an open-source, vendor-agnostic hub, dedicated to cloud native computing. As of October 2022, the foundation counts 18 graduated projects and 37 incubating ones. Big names like Kubernetes and Prometheus are among them.

The project started in 2015 and its main objective is to provide support and direction for fast-growing, cloud-native projects. It ensures that technologies are available to the community, fostering their growth and evolution of the ecosystem. It helps projects set clear boundaries, scale and remain platform agnostic. 

There are 4 types of projects: 

• Graduated, which are considered stable, widely adopted and production ready.
• Incubating, which are stable and used by a small number of people.
• Sandbox,   which are experimental and not widely tested.
• Archived, which have reached the end of their cycle.

Read more about CNCF.

Why is it important for Kubeflow?

Kubeflow is a project initiated by Google and over time it suffered from a lot of assumptions. It is a complex tool that includes a lot of components. Even though it garnered a lot of attention over the last few years, the product struggled with a few aspects such as security issues, community growth or process development within the community. 

Kubeflow had some turning points that started hot debates in the community. Headlines like “Is Kubeflow dead?” capture opinions from contributors, who were worried about Kubeflow’s direction and vision. The decision that Google just took to apply to CNCF is not just a reminder of the benefits that the solution has, but also a confirmation that there is a need and desire to keep developing it.

Is Kubeflow lost?

This has been one of the most common questions in the community. The application to CNCF offers hope that the product will soon benefit from guidance and support to grow to fester and fulfil its initial vision.

What’s next?

The pull request is only the first step in the entire process. Kubeflow’s community has to work closely together with Google, CNCF and its Technical Oversight committee to meet the incubation stage requirements. While the process is going to take time, the community will keep working on improving and developing the solution.

If accepted by CNCF, Kubeflow’s code, trademark, website and other collaboration will be transferred to the foundation. Google declared that they will work with CNCF and Kubeflow’s community to update the project’s governance.

Canonical will keep developing and supporting both Charmed Kubeflow and the upstream project.  We are excited to see what the future holds!