Managing multiple software versions

Software projects are often required to maintain multiple versions of the software in parallel in order to support different users of the software with different needs. As far as source code goes, the typical way to maintain multiple versions is by using different Git branches. Using branches in conjunction with Konflux can be somewhat tedious as a separate Component needs to be defined for each branch and a separate Application needs to be defined for each collection of components that need to be tested and released together. In addition multiple other resources such as ImageRepository, IntegrationTestScenario, and ReleasePlan need to be defined for each Component or Application to enable a full CI/CD process.

The Konflux Project Controller seeks to streamline the process of managing multiple versions by introducing the following objects and concepts:

A Project is used to describe a major piece of software that can be worked on by multiple teams over an extended period of time. A project may contain one or more development streams.
A ProjectDevelopmentStream indicates an independent stream of development. A ProjectDevelopmentStream can contain one or more Applications each containing one or more Components.
As described above, starting a new development stream involves creating a large amount of Application, Component, and other resources. The Project Controller helps to streamline that by allowing to create a ProjectDevelopmentStreamTemplate resource that specifies the resources to be created and allows for using variables to customize them. Using a template, many similar development streams can be created quickly.

Using the Project Controller

In the sections below we will describe how to use the custom resources supported by the Project Controller to manage multiple software versions and generate Application, Component, and other resources.

Before you begin

The typical process of onboarding a component to Konflux includes having the Konflux Build Service generate a PR to the component Git repository which includes the Tekton Pipeline-as-Code (PaC) pipeline definition for building that component. As a part of this process various other resources such as the PaC Repository resource and Secret resources for accessing the repository are also created.

When using the Project Controller to create components the process of generating the PR is skipped because in this document we assume the branches being created in the component Git repositories already contain PaC pipeline files. Each Git repository, however, needs to undergo this process at least once, therefore it is recommended to onboard at least once component from each repository from the UI or by other means before attempting to manage components in that repository via the Project Controller.

The rest of this document is going to assume that process is being followed and would, on occasion request that data would be copied from the component that was already on-boarded by other means.

Creating a Project

Create a project resource by applying YAML like the following:

project.yaml

apiVersion: projctl.konflux.dev/v1beta1
kind: Project
metadata:
  name: multi-version-konflux-sample
spec:
  displayName: "Multi-version demonstration sample project"
  description: |
    A sample project to demonstrate how to use the projects API.

Creating a ProjectDevelopmentStreamTemplate

To enable quickly creating multiple development streams, we must create a template for them.

template.yaml

apiVersion: projctl.konflux.dev/v1beta1
kind: ProjectDevelopmentStreamTemplate
metadata:
  name: multi-version-konflux-sample-template
spec:
  project: multi-version-konflux-sample
  variables:
  - name: version
    description: A version number for a new development stream
  - name: versionName
    description: A K8s-compliant name for the version
    defaultValue: "{{hyphenize .version}}"

  resources:
  - apiVersion: appstudio.redhat.com/v1alpha1
    kind: Application
    metadata:
      annotations:
        application.thumbnail: "9"
      name: "multi-version-konflux-sample-{{.versionName}}"
    spec:
      displayName: "multi-version-konflux-sample-{{.versionName}}"

  - apiVersion: appstudio.redhat.com/v1alpha1
    kind: Component
    metadata:
      annotations:
        build.appstudio.openshift.io/pipeline: '{"name":"docker-build","bundle":"latest"}'
        build.appstudio.openshift.io/status: '{"pac":{"state":"enabled","merge-url":"https://github.com/konflux-ci/multi-version-konflux-sample/pull/1","configuration-time":"Wed, 07 Aug 2024 08:59:18 UTC"},"message":"done"}'
      name: multi-version-konflux-sample-{{.versionName}}
    spec:
      application: "multi-version-konflux-sample-{{.versionName}}"
      componentName: "multi-version-konflux-sample-{{.versionName}}"
      source:
        git:
          context: ./
          dockerfileUrl: Dockerfile
          revision: "{{.version}}"
          url: https://github.com/konflux-ci/multi-version-konflux-sample.git

  - apiVersion: appstudio.redhat.com/v1alpha1
    kind: ImageRepository
    metadata:
      name: imagerepository-for-multi-version-konflux-sample-{{.versionName}}
      labels:
        appstudio.redhat.com/application: multi-version-konflux-sample-{{.versionName}}
        appstudio.redhat.com/component: multi-version-konflux-sample-{{.versionName}}
    spec:
      image:
        name: konflux-samples-tenant/multi-version-konflux-sample-{{.versionName}}/multi-version-konflux-sample-{{.versionName}}
        visibility: public
      notifications:
        - config:
            url: 'https://bombino.preprod.api.redhat.com/v1/sbom/quay/push'
          event: repo_push
          method: webhook
          title: SBOM-event-to-Bombino

The resources section for the template may be created by looking at the YAML for existing resources and copying it while removing generated and unnecessary data and adding variable references where needed.

The following kinds of resources are currently supported. Please do not attempt to create other kinds as that would prevent the template from being applied successfully.

apiVersion kind Comments

apiVersion	kind	Comments
appstudio.redhat.com/v1alpha1	Application
appstudio.redhat.com/v1alpha1	Component
appstudio.redhat.com/v1alpha1	ImageRepository	Every component should have an accompanying `ImageRepository` resource
appstudio.redhat.com/v1beta2	IntegrationTestScenario	Note the `v1beta2` API version. Make sure to use the right version when querying the cluster, otherwise important data may be missing from the results you get.
appstudio.redhat.com/v1alpha1	ReleasePlan

appstudio.redhat.com/v1alpha1

Application

appstudio.redhat.com/v1alpha1

Component

appstudio.redhat.com/v1alpha1

ImageRepository

Every component should have an accompanying ImageRepository resource

appstudio.redhat.com/v1beta2

IntegrationTestScenario

Note the v1beta2 API version. Make sure to use the right version when querying the cluster, otherwise important data may be missing from the results you get.

appstudio.redhat.com/v1alpha1

ReleasePlan

Note: When in doubt - the source of truth about what the YAML for your resources should look like is the version you can find on the Konflux clusters. We try to keep the examples here up to date, but with this being a static document there may be a gap between what you see here and what you should configure in your system. It is recommended that you follow the process of creating the template by copying the YAML of the resources you have and cleaning it as described below, and not by copying the sample you see here.

Here are specific examples for how to clean up and use the YAML for certain resource kinds:

For any kind of resource specified in the ProjectDevelopmentStreamTemplate, the namespace, creationTimestamp, generation, resourceVersion, uid, ownerReferences, and finalizers metadata fields should be removed as well as the status section.
For Application resources the metadata.name and spec.displayName fields should contain variable references.
For Component resources:
- The following deprecated annotations should be removed:
  - image.redhat.com/image
- The spec.containerImage field should be removed.
- The following fields should probably contain variable references:
  - spec.application
  - spec.componentName
  - source.git.revision
- The build.appstudio.openshift.io/status annotation is in place to make the UI present the component’s pipeline as customized. Its not mandatory if you can ignore the pipeline status being misrepresented in the UI. Note that this annotation’s value needs to contain a reference to a merged PR that added the PaC pipeline files. There is no harm in having multiple components reference the same PR, as long as the pac.component.appstudio.openshift.io/finalizer finalizer is not added to the component’s finalizers list.
For ImageRepository resources:
- The labels referring to the owning component and application should probably contain variable references.
- To allow for correct ownership configuration between Component and ImageRepository resources, every component that has its built image pushed to the default registry organization that is managed automatically by Konflux needs to have a Dedicated ImageRepository resource. This implies each component has its own container image repository.
- To meet the above requirement the spec.image.name field should contain a variable reference.
For IntegrationTestScenario resources:
- Make sure you query for the v1beta2 version of those resources, and that you specify that version in your template.
- The spec.application field should probably contain variable references.
For ReleasePlan resources:
- The spec.application field should probably contain variable references.

Some notes about using template variables:

You can use the Go text/template syntax to place template variable values into various resource attributes as well as variable default values.
You can use the custom hyphenize template function to create a value suitable for use in resource names.
It’s advisable to quote strings that contain variable references and other template syntax elements to prevent the curly braces from being parsed as JSON embedded into YAML.

Creating a ProjectDevelopmentStream

Once the Project and ProjectDevelopmentStreamTemplate resources are in place, we can create ProjectDevelopmentStream resources.

devstream.yaml

apiVersion: projctl.konflux.dev/v1beta1
kind: ProjectDevelopmentStream
metadata:
  name: multi-version-konflux-sample-v1-0-0
spec:
  project: multi-version-konflux-sample
  template:
    name: multi-version-konflux-sample-template
    values:
    - name: version
      value: "v1.0.0"

Creating this ProjectDevelopmentStream resource will cause the resources specified by the referenced ProjectDevelopmentStreamTemplate resource to get created. Since we’ve used the version template variable in the spec.git.revision field of the component resources, each component version will use a different branch of the component repository.

When you look at your components in the Konflux UI, you may see notifications that a PRs were sent to configure pipelines for them. In addition the source code links in the UI will not work until corresponding branches are actually created in your Git repository. To overcome those issues, create and push a branch with the appropriate name for each new component and then create and merge a PR into it that will cause the push pipeline to run for that branch. For the pipeline to run you need to adjust the Tekton PaC pipeline YAML code as described blow. We recommend that the first PR you send into a branch would include those adjustments.

Branching your component repositories

Beyond creating new Git branches for your components in order to maintain different versions, you must also adjust the .tekton/*.yaml files within those branches in order to make the pipelines run and target the right components.

In particular the following changes must be made each time a new branch is created in each of the pipeline YAML files:

The pipelinesascode.tekton.dev/on-cel-expression annotation should be adjusted to specify and filter by the right branch name. For example, for a pull request pipeline that resides in the v1.0.0 branch the annotation value would be:
```
event == "pull_request" && target_branch == "v1.0.0"
```
For a push pipeline in the same branch the value would be:
```
event == "push" && target_branch == "v1.0.0"
```
The appstudio.openshift.io/application and appstudio.openshift.io/component labels must be adjusted to specify the right Application and Component respectively. Failing to do this will cause builds of the pipeline to be associated with the wrong application or component.
The value for the output-image parameter should be set to match the vale of the spec.image.name field of the ImageRepository resource that corresponds to the component the pipeline would build.

If your template is setup in such a way that the repo branch name would appear in the image name, you can use the target_branch PaC variable like so:
```
  - name: output-image
    value: quay.io/redhat-user-workloads/my-tenant/my-app-{{target_branch}}/my-comp-{{target_branch}}:{{revision}}
```
Please note that the examples in this document do not allow for this setup because the image names contain the hyphenated version value while the branch names contain the unhyphenated value.

Known limitations

The following limitations exist in the current controller implementation and are likely to be resolved in the future.

Resource creation order is important. You must first create Project resources followed by ProjectDevelopmentStreamTemplate resources and only then ProjectDevelopmentStream resources.
If a ProjectDevelopmentStreamTemplate is modified, resources that were already created using that template do not get updated unless either:
- The controller gets restarted
- The ProjectDevelopmentStream resource referring to the template is modified
If a resource created by a template is modified, the configuration is not aligned back with the template unless either:
- The controller gets restarted
- The ProjectDevelopmentStream resource referring to the template is modified
A ProjectDevelopmentStream that isn’t referring a template may be modified to refer to a template. Similarly, the template ProjectDevelopmentStream is referring to may be changed. In both those cases, resources owned by the ProjectDevelopmentStream but not defined by the new template do not get deleted.