component_descriptor Trait
Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
step |
no |
name: component_descriptor
|
dict |
The build step name injected by this trait |
resolve_dependencies |
no |
True |
bool |
Indicates whether or not unresolved component dependencies should be resolved |
upload |
no |
legacy |
UploadMode |
Indicates whether or not to publish component-descriptor during Component-Descriptor
step. For backwards-compatibility reasons, defaults to “legacy”, which is a mode
that depends on pipeline’s context.
Should be configured to |
component_name |
no |
None |
str |
Manually overwrites the component name (which defaults to github repository path) |
callback_env |
no |
{}
|
str |
Specifies additional environment variables passed to .ci/component_descriptor script |
retention_policy |
no |
None |
One of:
|
specifies how (old) component-descriptors and their referenced resources should be handled. This is foremostly intended as an option for automated cleanup for components with frequent (shortlived) releases and/or frequent (shortlived) snapshots. if no retention_policy is defined, no cleanup will be done. retention policy may either be defined “inline” (as a mapping value) or by referencing
a pre-defined policy by name (see |
retention_policies |
no |
- dry_run: false
name: clean-snapshots
rules:
- keep: 64
match: snapshots
name: clean-snapshots
recursive: false
restrict: none
- keep: all
match: releases
name: keep-releases
recursive: false
restrict: none
- dry_run: false
name: clean-snapshots-and-releases
rules:
- keep: 64
match: snapshots
name: clean-snapshots
recursive: false
restrict: none
- keep: 128
match: releases
name: clean-releases
recursive: false
restrict: none
|
List[VersionRetentionPolicies] |
predefined retention policies (see default value). may be referenced via
|
validation_policies |
no |
- ignore-me
|
List[str] |
obsolete |
ctx_repository_base_url |
no |
None |
str |
the component descriptor context repository base_url (for component descriptor v2). If not configured, the CICD-landscape’s default ctx will be used. |
ctx_repository |
no |
None |
str |
the component descriptor context repository cfg name (for component descriptor v2). If not configured, the CICD-landscape’s default ctx will be used. |
ocm_repository |
no |
None |
str |
the Component Descriptor OCM Repository url used for publishing. instead of repository-url, may also be the name of a cfg-element. |
component_labels |
no |
[]
|
List[Label] |
a list of labels to add to the component in the base Component Descriptor |
inputs |
no |
[]
|
List[StepInput] |
inputs to expose to component-descriptor step |
depends |
no |
[]
|
List[str] |
steps that need to be run prior to running component-descriptor-step |
ocm_repository_mappings |
no |
[]
|
List[OcmRepositoryMappingEntry] |
used to explicitly configure where to lookup component descriptors. If given, ocm_repository must be defined. Example: - repository: ocm_repo_url
prefix: github.com/some-org/
- repository: ocm_repo_url
prefix: github.com/another-org/
- repository: another_ocm_repo_url
prefix: github.com/yet-another-org/
Value from ocm_repository is implicitly added as entry. |
upload Enumeration Values
legacy
no-upload
retention_policies[] (VersionRetentionPolicies) Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
name |
yes |
None |
str |
|
rules |
yes |
None |
List[VersionRetentionPolicy] |
|
dry_run |
no |
True |
bool |
retention_policies[].rules (VersionRetentionPolicy) Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
name |
no |
None |
str |
|
keep |
no |
all |
One of:
|
|
match |
no |
|
VersionType |
|
restrict |
no |
|
VersionRestriction |
|
recursive |
no |
False |
bool |
component_labels[] (Label) Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
name |
yes |
None |
str |
|
value |
yes |
None |
str | int | float | bool | dict | list |
inputs[] (StepInput) Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
step_name |
yes |
None |
str |
|
output_name |
no |
None |
str |
|
type |
no |
step |
str |
ocm_repository_mappings[] (OcmRepositoryMappingEntry) Attributes
name |
required? |
default |
type |
explanation |
---|---|---|---|---|
repository |
yes |
None |
str |
|
prefix |
no |
None |
str | None |
Dependencies
This trait requires the following traits to be declared:
If declared, a Component Descriptor
is created during job execution. If their
release Trait trait is also declared, the created component descriptor is added to the
component’s release artifacts (as GitHub release asset).
Example
traits:
component_descriptor: ~
Default Component Descriptor
If no additional customisation is done, a _default_ component descriptor is created with the following data:
component_name (the github repository path, e.g.
github.com/gardener/gardener
)component_version (the
effective version
)
In addition, any container image that is built by the component is added as container image dependencies.
Declaring Additional Dependencies
The component descriptor should contain the full “bill-of-materials” that makes up a component. For many components this will simply be the set of container images built and released by the component (those are added automatically).
To declare additional dependencies, an executable may be placed at .ci/component_descriptor
at the component repository. If such an executable is present, it is called with a defined set
of environment variables and file system layout.
Before calling the .ci/component_descriptor
callback, the default (or base) component descriptor
is generated and offered to the component_descriptor script.
After termination, the component_descriptor script is expected to leave a valid “final”
component_descriptor at a defined file system path. Unresolved component references
are
automatically resolved before uploading as release artifact.
Environment Variables Passed to component_descriptor
name |
explanation |
---|---|
BASE_DEFINITION_PATH |
absolute file path to base component descriptor |
COMPONENT_DESCRIPTOR_PATH |
absolute file path to output final descriptor to |
ADD_DEPENDENCIES_CMD |
CLI cmd args to add dependency to base descriptor (see
|
COMPONENT_NAME |
own component name (e.g. github.com/gardener/vpn) |
COMPONENT_VERSION |
the effective version |
Example
How to declare dependencies towards:
component
github.com/gardener/vpn
in version 1.2.3component
github.com/gardener/dashboard
in version 4.5.6container image
alpine:3.6
# inside .ci/component_descriptor, assuming it is a shell script
${ADD_DEPENDENCIES_CMD} \
--component-dependencies \
'{"name": "github.com/gardener/vpn", "version": "1.2.3"}' \
--component-dependencies \
'{"name": "github.com/gardener/dashboard", "version": "4.5.6"}' \
--container-image-dependencies \
'{"image_reference": "alpine:3.6", "version": "3.6", "name": "alpine"}'
# don't forget to expose the image
cp "${BASE_DEFINITION_PATH}" "${COMPONENT_DESCRIPTOR_PATH}"
Local Development and “component-cli”
Setting up a local development environment for running the component_descriptor
step may be
cumbersome. Therefore, a convenience command is provided from gardener-cicd-cli
python package
that evaluates both .ci/pipeline_definitions
(+ optional branch.cfg
), and calls a
.ci/component_descriptor
callback script, thus creating a similar output as if running in
cicd-pipeline.
Setting up Preliminaries
install python3 as indicated by
gardener-cicd-cli
-package (3.10+)run
pip3 install gardener-cicd-cli
(python-headers + c-compiler-toolchain might be required)optional: install
component-cli
to PATHinstall other runtime-dependencies as needed by local
.ci/component_descriptor
callback script
If repository in question uses branch.cfg
(in special ref refs/meta/ci
), fetch it into local
repository by running: git fetch origin refs/meta/ci:refs/meta/ci
(use different origin as needed).
Note
pass --meta-ci fetch
to tell the command (see below) to fetch refs/meta/ci for you
Rendering Component-Descriptor
Run the following command (available from PATH after installing gardener-cicd-cli
) (chdir into
repository’s working tree):
gardener-ci pipeline component_descriptor
Note
The pipeline component_descriptor
command tries to guess things like component-name, or the
pipeline to use (preferring a pipeline-job that is likely the release-job).
Pass the -h
(or --help
) flag to display online-help. Most heuristics can be overwritten.
Note
Component-Descriptors creating this way will be close to those that will be created by CICD Pipeline Jobs, but not necessarily 100% accurate (for example, image-tag-templates are not evaluated, which may lead to different image-tags in “base-component-descriptors”).
Special-handling for “charts/images.yaml” / deprecating component-cli
component-cli
has been deprecated as of 2023-04-06. component-cli
was tailored as an
opinionated tool considering some special-cases useful for many of Gardener’s repositories in
mind. It’s successor - OCM-CLI might
replace component-cli
, however it will not feature said gardener-specific special-case-handling.
To phase-out component-cli
, with little efforts all relevant commands are
re-implemented as part of CICD-Pipeline-Template as a drop-in-replacement.
Implementation can be found
here.
The default instrumentation of component-cli commands can be found here.
“imagevector add” command / charts/images.yaml contract
Some Gardener-Repositories use a standardised format to declare images to be exposed to both
helm-charts and Component-Descriptors
via a regular file located at charts/images.yaml
below
repository root.
The (deprecated) component-cli
features a command imagevector add
that converts data from such
images.yaml
files to component-descriptors.
images.yaml
is expected to be a YAML document (or multi-document) containing (oci-)image-entries.
Those are stored as a list below an attribute images
. Depending on the defined attributes,
entries are handled differently.
In addition to attributes being absent, or present, there is also a list of “component-prefixes”,
which defaults to eu.gcr.io/gardener-project/gardener
, which influences whether an entry is
considered to be “local” (built by component’s pipeline) or “external” (built by someone else).
Gardener-Components have a name that is by convention the github-repo-url (w/o scheme). If the
sourceRepository
is different from current component name, a component-reference is added.
Example
# current component: github.com/gardener/gardener
# current version: 1.67.0
# github-repo: github.com/gardener/gardener
images:
- name: gardenlet
sourceRepository: github.com/gardener/gardener # same as current component in this example
repository: eu.gcr.io/gardener-project/gardener/gardenlet
Results in:
resources:
- name: gardenlet # from name-attribute
relation: local # from repository's prefix matching eu.gcr.io/gardener-project/gardener
type: ociImage # hard-coded
version: 1.67.0 # from current version
access:
imageReference: eu.gcr.io/gardener-project/gardener/gardenlet:1.67.0 # <repo>:<version>
type: ociRegistry # hard-coded
labels:
- name: imagevector.gardener.cloud/name
value: gardenlet # from name-attribute
- name: imagevector.gardener.cloud/repository
value: eu.gcr.io/gardener-project/gardener/gardenlet # from repository-attribute
- name: imagevector.gardener.cloud/source-repository
value: github.com/gardener/gardener # github-repo
Retention Policies (aka cleaning up old versions)
The retention_policies
attribute can be used to configure automated removal of
component descriptors and referenced resources
(mostly OCI Container Images).
Attention
Removal of component descriptors and referenced resources is _permanent_. There is no backup mechanism in place. Use with care. For example, if multiple component descriptors share reference to the same OCI Artefact (using the same registry, repository, and tag) removal of any of the referencing component descriptors will lead to stale references in other component descriptors.
Cleanup Semantics and Use-Case
If frequently publishing component-descriptors as snapshot-versions (e.g. for each head-update, or for pull-request-validation), thus-produced build artefacts and component descriptors typically are only relevant for a short period of time. In such cases, automated cleanup of snapshot-versions can be configured (see attribute-documentation above).
It is possible to further narrow-down versions to cleanup, by setting the restrict
-attribute
to same-minor
. If thus-configured, cleanup will only be done among component descriptors that
share the same minor version w/ the current component version.
Policy rules are evaluated in the order they are defined. When cleanup is run, all existing versions (in current component descriptor repository) are retrieved, and grouped by defined cleanup rules (each version is added exactly to the first matching rule; if no rule matches, versions are dropped (thus exempted from cleanup)).
Each thus-collected group of versions is ordered, acccording to “relaxed” semver-arithmetics,
from smallest to greatest. Depending on the amount of versions to “keep” (keep
attribute),
starting from smallest, progressing to greatest, versions to be removed are determined. It is
possible that no version is identified as being subject for cleanup.
For each version to be removed the component-descriptor to be removed is fetched and processed:
Sources are ignored.
From declared resources, all resources that are supported for removal are removed.
A resource is considered to be supported for removal if it has been declared of relation: local
(i.e. it was built along w/ the component-descriptor), and if its access-type is supported
by underlying CICD Infrastructure. This is currently limited to OCI Artefacts (including
“multi-arch” Images), and subject to being extended over time. Blobs that are inlined within
component descriptor OCI Artefact will be implicitly along with the component descriptor.
Once all supported resources have been removed, the declaring component descriptor is removed.
For performance reasons, cleanup may be limited to an internally defined amount of versions.