Commit Graph

506 Commits

Author SHA1 Message Date
Jan Kiszka
28891711c0 Release 2.5 2021-05-26 07:27:23 +02:00
Jan Kiszka
46eabc7be3 kas-container: Add support for checkout and for-all-repos
Allow invoking those two new sub-commands also via the container
interface.

Closes #51
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-20 16:39:39 +02:00
Rotem Bar
0e470dc644 kas-container: Fix mounting of custom KAS_REPO_REF_DIR
kas-container support for KAS_REPO_REF_DIR was broken. The path provided
by the env variable was the path on the local machine, not in the
container.

Signed-off-by: Rotem Bar <rotemb@hailo.ai>
[Jan: massaged commit log]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-10 07:30:17 +02:00
Claudius Heine
7c6e3c3994 kas-container: add --git-credential-store options
This options allows to specify a git credential store file, which is
then mounted into the container and used by kas as a
git-credential-helper.

Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: remove debug echo]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-06 14:09:10 +02:00
Claudius Heine
71e7f5523d kas: add GIT_CREDENTIAL_HELPER environment variable
This allows to set a git credential helper in the .gitconfig, in order
to handle git authentications. For instance when bitbake recipes require
to clone from git via https.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-06 14:08:40 +02:00
Konrad Weihmann
b250c341fb repos: strip dot from layer name
when importing nested layer in combination with the top level layer like

meta-security
meta-security/meta-tpm

one can use

layers:
    .:
    meta-tpm:

in the spec file to import the parent layer.
This leads to the effect that

.
meta-tpm             = "HEAD:<hash>"

is shown in the build console and in /etc/build
when using image-buildinfo.bbclass.
To fix that just strip the trailing dot from each layer name.

Add test to avoid future regressions

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
[Jan: use only a single rstrip call, fix style]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-05 15:43:58 +02:00
Jon Mason
eb63b1873b Dockerfile: move sudo to standard Dockerfile
To expand the usability of the standard Docker image, move sudo
enablement from Docker.isar.

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-27 07:25:59 +02:00
Jose Quaresma
6e82e0d12d kas-container: add support to set a custom container images location
There are situation that the user have a local customized container images
with tools installed that are not provided in the default kas container.

- For a local container the user can tag it and use the existing variables:

 docker tag local-container my/container:1.0
 export KAS_CONTAINER_IMAGE_PATH=my
 export KAS_CONTAINER_IMAGE_NAME=container
 export KAS_IMAGE_VERSION=1.0

- This patch improve the user interface by allowing to set a custom image
 with just a single environment variable.

 export KAS_CONTAINER_IMAGE=local-container

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-16 14:20:58 +02:00
Claudius Heine
06240c23da kas-container: mount /repo as read-write for shell command
There are some bitbake commands like `recipetool` that allows to easily
create new or append to existing recipes. This of course only works if
the main repository that is worked with is writeable.

However it is mounted into the container only as read-only.

This patch mounts the repository writeable when the `shell` command is use
and read-only in case of the `build` command.

It also adds `--repo-ro` and `--repo-rw` to allow overwriting the
default behaviour.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-16 08:04:31 +02:00
Silvano Cirujano Cuesta
6d1b65eb7b Add skopeo and umoci to ISAR image
Skopeo and Umoci are needed for the new ISAR SDK Container target.
A "manual" backport of Skopeo (and only Skopeo) is needed, since the
repository "buster-backports" doesn't provide it.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
[Jan: add golang-github-containers-{image,common} to buster list]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-06 11:00:32 +02:00
Henning Schild
243c51d24d Use issue template to point people to the list
We often see Issues being used while they maybe should not. Introducing
an issue template might help avoid confusion and inviting contributors
to the list without disabling Issues.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-25 19:43:01 +01:00
Jan Kiszka
3f98520ab6 kas-container: Improve KAS_WORK/BUILD_DIR retrieval
Perform link resolution unconditionally to account for cases when the
default paths are links. Use default values to simplify the assignments.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-21 09:17:30 +01:00
Jan Kiszka
03da3f9436 docs: Improve link to container images
We were linking version 2.2 here.

Reported-by: Vivien Didelot <vivien.didelot@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-19 16:24:25 +01:00
Jan Kiszka
6a792c0a3e docs: Tune layout of env var table
Otherwise the descriptions of KAS_WORK_DIR and KAS_BUILD_DIR are merged
together when rendering on kas.readthedocs.io.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-19 08:01:29 +01:00
Peter Hatina
9d1d2adbd5 docs: Fix typo in excluded layers example
Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-12 11:18:24 +01:00
Peter Hatina
f32290d425 Introduce KAS_BUILD_DIR environment variable
This variable can override default build path `${KAS_WORK_DIR}/build`.

Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-10 07:53:09 +01:00
Peter Hatina
24c13a491b Apply patches before doing an environment setup
We may need to patch the environment setup, too.

Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-01 13:58:49 +01:00
Henning Schild
6b52357dc7 kas-container: add an argument to get version information
This script is deployed in many copies, give people a way to identify
which one they have cached/installed.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: simplify and use basename for the program]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-25 21:19:02 +01:00
Jan Kiszka
8fe75298e4 Release 2.4 2021-02-25 19:09:02 +01:00
Henning Schild
8234936902 Dockerfile: do not allow pip to fetch any deps anymore
We want all our deps from debian to make sure we do not get random
versions from pip. So we call pip install in such a way, and test that
kas can include all its deps later.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-24 20:54:06 +01:00
Jan Kiszka
5253d3c1d2 Dockerfile: Pull all Python dependencies from Debian
This provides a clearer and more stable environment.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-24 13:14:29 +01:00
Jan Kiszka
5981664b25 setup: Set upper version limit for dependencies
Let's avoid that any of our dependencies decides to break interfaces
that we use and that users run into surprises. As a safe bet, limit us
to the currently know-to-work major versions.

Closes #34
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-24 09:01:08 +01:00
Jan Kiszka
6c5e77800c kas-container: Simplify docker arguments
--privileged implies all caps, so no need to list some explicitly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-16 10:36:14 +01:00
Jan Kiszka
6f8d5d6c78 checkcode: Address shellcheck warnings
...and stop ignoring them all.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 21:00:08 +01:00
Jan Kiszka
6fcdf85d9a release.sh: Address shellcheck remarks
No real issues, but let's silence them to catch future ones. One change
is a nice simplification in fact.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 20:59:55 +01:00
Henning Schild
111906f804 kas-container: make sure that we pass shellcheck
There is one actual change where code gets touched, the rest are just
comments to satisfy shellcheck.
All the lines with the ignores should be reviewed later, there might be
problems behind the findings.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: add one more SC2086 suppression]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 20:42:49 +01:00
Jan Kiszka
4042ef4859 kas-container: Add support for multi-word --command arguments
Yet another special dance needed to preserve argument boundaries while
moving them around: This trick enables

kas-container shell kas.yml --command "echo it works."

by keeping the command argument separate and injecting it via
appropriate quoting into the argument array.

Closes: #42
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 20:29:56 +01:00
Maximilian Schmidt
15aaf4677b Add support for rootless podman with userns keep-id
Running `./kas-container` as rootless podman container fails with:

```
groupadd: Permission denied.
groupadd: cannot lock /etc/group; try again later.
useradd: Permission denied.
useradd: cannot lock /etc/passwd; try again later.
```

This patch add a check whether the correct USER_ID is already in use.

Co-authored-by: Nicolas Riebesel <nicolas.riebesel@gmx.com>
Signed-off-by: Maximilian Schmidt <maximilian@schmidt.so>
[Jan: fixed indention]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-20 08:16:36 +01:00
Henning Schild
c50b3d21d8 kas: drop bitbakes "-k" from the default args
That gets us in line with the default of bitbake itself. If users still
want to pass it, they will have to do so from now on.

While that continue switch can seem like a sane default for interactive
builds, it can also be pretty wasteful in CI. Where it would not just
eat compute ressources but also peoples time, because it potentially
delays build failures significantly.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-19 18:14:59 +01:00
Jan Kiszka
e8b6a4d89b Add support for multi-arch containers
Create the kas and kas-isar containers also for arm64. This allows using
them for arm-native builds, e.g. in AWS instances.

We need to drop a couple of packages from arm64 images which do not
exist on that arch. If the kas image is usable for Yocto builds is
untested, though. The Isar image works.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-03 18:39:35 +01:00
Jan Kiszka
c92d845639 ci: Push directly from build step
Refactor the workflows so that we can use build-push-action also for
publishing. This is a precondition for switching to buildx, needed for
multi-arch images.

The trick needed for this is using an environment variable to control
whether we should push when building the master branch, initializing it
according to the tag status.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-03 18:39:12 +01:00
Jan Kiszka
2f60b982a7 Dockerfile.isar: Drop grub package
The leakage of grub artifacts from the building Debian version and the
usage its grub tools was fixed in Isar 0.6 already. No one is expected
to use that old version with latest kas containers, and the presence of
these packages also collide with providing non-x86 versions.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-03 18:36:38 +01:00
Jan Kiszka
2f4b89c77f Dockerfile: Fold two related ENV settings together
One layer less.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-02 15:53:06 +01:00
Jan Kiszka
cc159c471f Dockerfile: Make DEBIAN_FRONTEND non-persistent
Only use this setting during non-interactive build, not in the possibly
interactive image.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-02 15:50:17 +01:00
Jan Kiszka
a18c414b81 CONTRIBUTING: Drop reference to non-existing maintainer list
This project does not have this.

Reported-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-12-17 18:13:19 +01:00
Silvano Cirujano Cuesta
0892b37ad6 repos: refactor repo path calculation
Refactored repo path calculation eliminates duplicated code and aligns
the matching logic (with respect to url and path) with the
documentation.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-12-16 18:17:12 +01:00
Silvano Cirujano Cuesta
1f1a21992d repos: fix path if no url, but path given
The `repo` documentation says:
"If the url is missing and the path defined, this entry references the
directory the path points to. [...] In case of a relative path name
`kas_work_dir` is prepended."

But if no `url` is given and a relative path is given in `path`, then
the layer paths being added to `bblayer.conf` are missing the
`kas_work_dir` prefix.

This patch fixes it.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-12-16 18:17:04 +01:00
Jan Kiszka
87af46ff8c Silence "Exception ignored when trying to write to the signal wakeup fd"
Disable warn_on_full_buffer via set_wakeup_fd. This avoids tons of

    Exception ignored when trying to write to the signal wakeup fd:
    BlockingIOError: [Errno 11] Resource temporarily unavailable

that are currently issued after running complex builds. Only works since
Python 3.7, though, so this remains best-effort. Fortunately, the Debian
in our container image is 3.7.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-12-02 12:40:19 +01:00
Jan Kiszka
368cdffc24 Release 2.3.3 2020-11-28 12:36:46 +01:00
Quirin Gylstorff
e905f0002d Dockerfile.isar: Escape special characters
Escape '&' '\&' as sed replace & with the search term.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-23 16:45:07 +01:00
Jan Kiszka
ba439d4200 Release 2.3.2 2020-11-20 12:20:24 +01:00
Jan Kiszka
3d0c9b8299 Add release artifact to .gitignore
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-20 12:05:51 +01:00
Jan Kiszka
7f89747645 scripts/release: Fix KAS_IMAGE_VERSION_DEFAULT update
Remove the leading tab, it's no longer there.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-20 12:03:25 +01:00
Jan Kiszka
f50316e1c9 Release 2.3.1 2020-11-20 11:16:18 +01:00
Jan Kiszka
94f20cc9ad ci: Switch to environment files
Fixes:

"Unable to process command '::set-env name=RELEASE_VERSION::2.3'
successfully.
The `set-env` command is disabled. Please upgrade to using Environment
Files [...]."

To make the next release smoother, hopefully.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-20 11:03:59 +01:00
Jan Kiszka
816dffa898 scripts/release: Update announcement footer
We have the packages on github now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-20 08:39:52 +01:00
Jan Kiszka
7bbccbfb47 Release 2.3 2020-11-20 08:19:32 +01:00
Jan Kiszka
6110863a0c userguide: Fix typo
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-18 19:42:10 +01:00
Jan Kiszka
49e90b2728 CONTRIBUTING: Add newline to fix rendering
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-18 15:15:45 +01:00
Jan Kiszka
b36b2dbedd Rename forallrepos plugin to for_all_repos
Comes closer to how the command is spelled. We can't uses dashes as that
is not compatible with the python syntax.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-17 18:26:12 +01:00