This imports revision aa9b9dc9a9 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael.lisicki@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
KAS should exit with the appropriate exit code, if the command parsed to the for_all_repos
plugin fails to apply on one of the repositories.
Example use-case: This allows for CI pipelines or wrapper scripts to take appropriate action (e.g. fail),
if an issue occured within the KAS for_all_repos command.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
An upcoming change will introduce returning non-zero status on the KAS
command if the for_all_repos command fails for one of the repos defined
in the kas configuration. This caused an issue with the non-git "this"
repo defined in the test.yml.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: adjusted commit log to reflect different commit ordering]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since dabda7617f, all elements of KAS_FILES are already absolute.
Therefore, a single replacement rule is sufficient to translate them to
paths for the container.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This stresses the previously broken case of the incorrectly detected
repository root while including files.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
So far, repository paths for local includes were derived from the path
name of the config file containing the include, rather than using the
actual repository root as specified in the documentation. No one
complained so far, some layers simply adjusted their includes to this
inconsistency which was only discovered during refactorings.
Fix this issue by passing also the repository path along with the config
filename down the recursive _internal_include_handler calls. The
top-level repo path now needs to be retrieved before the creation of
IncludeHandler and passed to it then. This has the side effect of
enabling deviating top-level repo paths, a feature that will be used by
the upcoming changes for a .config.yaml file in KAS_WORK_DIR.
As there are existing users of the old behavior out there, fall back to
it if a local include cannot be found under the correct path and warn if
this succeeds. This allows smooth migration of layers to the right
pattern as they update their kas version.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We will fail differently then, but with a nicer error message (missing
init-build-env script) than
ValueError: Set of coroutines/Futures is empty.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
No one asked for this so far but the result of providing a relative
path, e.g. to KAS_WORK_DIR, is so confusing that it is better to simply
resolve all those vars to absolute paths.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
OpenEmbedded Core is about to depend on lz4c, pzstd, and zstd being on
the host. The Dockerfile was already installing zstd which provides
zstd/pzstd, so also install lz4 for lz4c.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Debian Buster comes with version 2.7.1 of git-lfs which does not support
git+ssh yet, therefore we need a newer version of git-lfs for cloning a
repository which uses LFS.
Signed-off-by: Steffen Hieber <steffen.hieber@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Backport repos have a priority of 100 by default, thus can just be added
with extra pinning. Use explicit release selection on the installation
of qemu-user-static instead, that's simpler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
By default git only fetches references under the refs/heads/ tree,
this patch adds support to kas to enable you to specify references
outside of the ref/heads tree. This is useful as it allows you to
use uncommitted gerrit patchsets, Gitlab merge requests or github
pull requests that live under refs/changes/, refs/merge-requests
and refs/pull as the reference for a repo allowing the use of
in development changes. When a refsepc is defined that starts
with refs/ an additional git fetch operation is preformed on the
repo to explicitly fetch the reference given so it can be checked
out for use.
Signed-off-by: Drew Reed <drew.reed@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since 7b18e5ec3b, which was released with 0.19.0, proxy_config has no
effect anymore. No one seems to have missed it, likely because setting
the proxies via the config file was a mistake anyway. The environment
variables are the proper way.
Declare these config entries officially obsolete and warn if they still
hang around in some legacy config. We will drop them from the schema in
one of the next releases.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Additionally to the variable SSH_PRIVATE_KEY, which can be used to
reference a private key saved as a string within a variable (the previous
documentation falsely stated, that it references a path and has been
adjusted), the new variable SSH_PRIVATE_KEY_FILE can now be used to reference
the path to a file containing the private key.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: Fix logic for detecting set env vars]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If command line includes refer to the same repo, missing_repos was
aggregating those multiple times, causing multiple clone procedures to
be triggered. This generally failed. Therefore, build missing_repos
while checking for duplicates.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
kas-container support for KAS_REPO_REF_DIR was broken. The path provided
by the env variable was the path on the local machine, not in the
container.
Signed-off-by: Rotem Bar <rotemb@hailo.ai>
[Jan: massaged commit log]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This options allows to specify a git credential store file, which is
then mounted into the container and used by kas as a
git-credential-helper.
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: remove debug echo]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to set a git credential helper in the .gitconfig, in order
to handle git authentications. For instance when bitbake recipes require
to clone from git via https.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
when importing nested layer in combination with the top level layer like
meta-security
meta-security/meta-tpm
one can use
layers:
.:
meta-tpm:
in the spec file to import the parent layer.
This leads to the effect that
.
meta-tpm = "HEAD:<hash>"
is shown in the build console and in /etc/build
when using image-buildinfo.bbclass.
To fix that just strip the trailing dot from each layer name.
Add test to avoid future regressions
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
[Jan: use only a single rstrip call, fix style]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
To expand the usability of the standard Docker image, move sudo
enablement from Docker.isar.
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There are situation that the user have a local customized container images
with tools installed that are not provided in the default kas container.
- For a local container the user can tag it and use the existing variables:
docker tag local-container my/container:1.0
export KAS_CONTAINER_IMAGE_PATH=my
export KAS_CONTAINER_IMAGE_NAME=container
export KAS_IMAGE_VERSION=1.0
- This patch improve the user interface by allowing to set a custom image
with just a single environment variable.
export KAS_CONTAINER_IMAGE=local-container
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There are some bitbake commands like `recipetool` that allows to easily
create new or append to existing recipes. This of course only works if
the main repository that is worked with is writeable.
However it is mounted into the container only as read-only.
This patch mounts the repository writeable when the `shell` command is use
and read-only in case of the `build` command.
It also adds `--repo-ro` and `--repo-rw` to allow overwriting the
default behaviour.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Skopeo and Umoci are needed for the new ISAR SDK Container target.
A "manual" backport of Skopeo (and only Skopeo) is needed, since the
repository "buster-backports" doesn't provide it.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
[Jan: add golang-github-containers-{image,common} to buster list]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We often see Issues being used while they maybe should not. Introducing
an issue template might help avoid confusion and inviting contributors
to the list without disabling Issues.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Perform link resolution unconditionally to account for cases when the
default paths are links. Use default values to simplify the assignments.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Otherwise the descriptions of KAS_WORK_DIR and KAS_BUILD_DIR are merged
together when rendering on kas.readthedocs.io.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This variable can override default build path `${KAS_WORK_DIR}/build`.
Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This script is deployed in many copies, give people a way to identify
which one they have cached/installed.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: simplify and use basename for the program]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We want all our deps from debian to make sure we do not get random
versions from pip. So we call pip install in such a way, and test that
kas can include all its deps later.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Let's avoid that any of our dependencies decides to break interfaces
that we use and that users run into surprises. As a safe bet, limit us
to the currently know-to-work major versions.
Closes#34
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
No real issues, but let's silence them to catch future ones. One change
is a nice simplification in fact.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There is one actual change where code gets touched, the rest are just
comments to satisfy shellcheck.
All the lines with the ignores should be reviewed later, there might be
problems behind the findings.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: add one more SC2086 suppression]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Yet another special dance needed to preserve argument boundaries while
moving them around: This trick enables
kas-container shell kas.yml --command "echo it works."
by keeping the command argument separate and injecting it via
appropriate quoting into the argument array.
Closes: #42
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
That gets us in line with the default of bitbake itself. If users still
want to pass it, they will have to do so from now on.
While that continue switch can seem like a sane default for interactive
builds, it can also be pretty wasteful in CI. Where it would not just
eat compute ressources but also peoples time, because it potentially
delays build failures significantly.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Create the kas and kas-isar containers also for arm64. This allows using
them for arm-native builds, e.g. in AWS instances.
We need to drop a couple of packages from arm64 images which do not
exist on that arch. If the kas image is usable for Yocto builds is
untested, though. The Isar image works.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Refactor the workflows so that we can use build-push-action also for
publishing. This is a precondition for switching to buildx, needed for
multi-arch images.
The trick needed for this is using an environment variable to control
whether we should push when building the master branch, initializing it
according to the tag status.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The leakage of grub artifacts from the building Debian version and the
usage its grub tools was fixed in Isar 0.6 already. No one is expected
to use that old version with latest kas containers, and the presence of
these packages also collide with providing non-x86 versions.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
