Creating a custom integration test

In Konflux, you can create your own integration tests to run on all components of a given application before they are deployed.

Procedure

To create any custom test, complete the following steps:

In your preferred IDE, create a Tekton pipeline in a .yaml file.
Within that pipeline, create tasks, which define the actual steps of the test that Konflux executes against images before deploying them.
Commit the .yaml file to a GitHub repo and add it as an integration test in Konflux.

Procedure with example

To create a custom test that checks that your app serves the text “Hello world!”, complete the following steps:

In your preferred IDE, create a new .yaml file, with a name of your choosing.

Define a new Tekton pipeline. The following example is the beginning of a pipeline that uses curl to check that the app serves the text “Hello world!”.

Example pipeline file:

kind: Pipeline
apiVersion: tekton.dev/v1beta1
metadata:
  name: example-pipeline
spec:
  params:
    - description: 'Snapshot of the application'
      name: SNAPSHOT
      default: '{"components": [{"name":"test-app", "containerImage": "quay.io/example/repo:latest"}]}'
      type: string
  tasks:

In the .pipeline.spec path, declare a new task.

Example task declaration:

tasks:
  - name: task-1
    description: Placeholder task that prints the Snapshot and outputs standard TEST_OUTPUT
    params:
      - name: SNAPSHOT
        value: $(params.SNAPSHOT)
    taskSpec:
      params:
      - name: SNAPSHOT
      results:
      - name: TEST_OUTPUT
        description: Test output
      steps:
      - image: registry.redhat.io/openshift4/ose-cli:latest
        env:
        - name: SNAPSHOT
          value: $(params.SNAPSHOT)
        script: |
          dnf -y install jq

          echo -e "Example test task for the Snapshot:\n ${SNAPSHOT}"
          // Run custom tests for the given Snapshot here
          // After the tests finish, record the overall result in the RESULT variable
          RESULT="SUCCESS"

          // Output the standardized TEST_OUTPUT result in JSON form
          TEST_OUTPUT=$(jq -rc --arg date $(date -u --iso-8601=seconds) --arg RESULT "${RESULT}" --null-input \
            '{result: $RESULT, timestamp: $date, failures: 0, successes: 1, warnings: 0}')
          echo -n "${TEST_OUTPUT}" | tee $(results.TEST_OUTPUT.path)

Save the .yaml file.

If you haven’t already, commit this file to a GitHub repository that Konflux can access.

Complete example file:

kind: Pipeline
apiVersion: tekton.dev/v1beta1
metadata:
  name: example-pipeline
spec:
  params:
    - description: 'Snapshot of the application'
      name: SNAPSHOT
      default: '{"components": [{"name":"test-app", "containerImage": "quay.io/example/repo:latest"}]}'
      type: string
    - description: 'Namespace where the application is running'
      name: NAMESPACE
      default: "default"
      type: string
    - description: 'Expected output'
      name: EXPECTED_OUTPUT
      default: "Hello World!"
      type: string
  tasks:
    - name: task-1
      description: Placeholder task that prints the Snapshot and outputs standard TEST_OUTPUT
      params:
        - name: SNAPSHOT
          value: $(params.SNAPSHOT)
      taskSpec:
        params:
        - name: SNAPSHOT
        results:
        - name: TEST_OUTPUT
          description: Test output
        steps:
        - image: registry.redhat.io/openshift4/ose-cli:latest
          env:
          - name: SNAPSHOT
            value: $(params.SNAPSHOT)
          script: |
            dnf -y install jq
            echo -e "Example test task for the Snapshot:\n ${SNAPSHOT}"
            // Run custom tests for the given Snapshot here
            // After the tests finish, record the overall result in the RESULT variable
            RESULT="SUCCESS"

            // Output the standardized TEST_OUTPUT result in JSON form
            TEST_OUTPUT=$(jq -rc --arg date $(date -u --iso-8601=seconds) --arg RESULT "${RESULT}" --null-input \
              '{result: $RESULT, timestamp: $date, failures: 0, successes: 1, warnings: 0}')
            echo -n "${TEST_OUTPUT}" | tee $(results.TEST_OUTPUT.path)

Add your new custom test as an integration test in Konflux.
1. For additional instructions on adding an integration test, see Adding an integration test.

Data injected into the PipelineRun of the integration test

When you create a custom integration test, Konflux automatically adds certain parameters and labels to the PipelineRun of the integration test. This section explains what those parameters and labels are, and how they can help you.

Parameters:

SNAPSHOT: contains the snapshot of the whole application as a JSON string. This JSON string provides useful information about the test, such as which components Konflux is testing, and what git repository and commit Konflux is using to build those components. For information about snapshot JSON string, see an example snapshot JSON string.

Labels:

appstudio.openshift.io/application: contains the name of the application.
appstudio.openshift.io/component: contains the name of the component.
appstudio.openshift.io/snapshot: contains the name of the snapshot.
test.appstudio.openshift.io/optional: contains the optional flag, which specifies whether or not components must pass the integration test before release.
test.appstudio.openshift.io/scenario: contains the name of the integration test (this label ends with "scenario," because each test is technically a custom resource called an IntegrationTestScenario).

It is also possible to set custom labels or annotations in the build pipelineRun, and those will be copied over to all integration pipelineRuns associated with that build. The labels/annotations have to have the custom.appstudio.openshift.io prefix in order to be copied in this manner.

Utilizing the labels and annotations within the integration pipeline run

It is possible to use the injected metadata within the integration PipelineRun itself in order to influence the testing behavior. This can be done by exposing the pipelineRun labels as environment variables within a Task and then referencing them within the Task logic.

Example of extracting the component name and finding its image within the SNAPSHOT parameter’s JSON data:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: test-metadata
spec:
  params:
    - name: SNAPSHOT
      description: The JSON string of the Snapshot under test
  steps:
    - name: find-component-image
      image: quay.io/redhat-appstudio/konflux-test:stable
      workingDir: /workspace
      env:
        - name: SNAPSHOT
          value: $(params.SNAPSHOT)
        - name: COMPONENT_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.labels['appstudio.openshift.io/component']
      script: |
        #!/bin/sh

        # Extract the component container image from the SNAPSHOT JSON data
        COMPONENT_CONTAINER_IMAGE=$(jq -r --arg component_name "${COMPONENT_NAME}" '.components[] | select(.name == $component_name) | .containerImage' <<< "${SNAPSHOT}")

        # Log the extracted variable
        echo "  COMPONENT_CONTAINER_IMAGE: ${COMPONENT_CONTAINER_IMAGE}"

For more examples of available labels and how they can be used within the integration tests, consult the example test-metadata task as well as the example integration pipeline which uses the information from that task’s results to influence its workflow.

Verification

After adding the integration test to an application, you need to trigger a new build of its components to make Konflux run the integration test. Make a commit to the GitHub repositories of your components to trigger a new build.

For information on other ways to trigger a new build, refer to the Retriggering Integration Tests

When the new build is finished, complete the following steps in the Konflux console:

Go to the Integration tests tab and select the highlighted name of your test.
Go to the Pipeline runs tab of that test and select the most recent run.
On the Details page, see if the test succeeded for that component. Select the other tabs to view more details.
1. If you used our example script, switch to the Logs tab and verify that the test printed “Hello world!”.