Commit Graph

55 Commits

Author SHA1 Message Date
Claudius Heine
26a7ab8f48 kas-container: add --log-driver=none as default runtime arg
Container engines per default log all container output into journald
(Podman) or a json file (Docker).

A build with kas-container/bitbake containes lots of useless build
output like progress lines, etc.

In case of Podman, where journald is used, this means that all this is
spammed into the journald buffer and possible also send to ttys or even
serial ttys. This might make sense for containers that output only
status information about running services, but for a build process, this
is mostly just spamming and causes important information to be lost.

Since the stdout/stderr output of the container is used for
kas-container ordinarily, the background logging from the container
image can be deactivated.

Therefor add the `--log-driver=none` parameter to all container
runtimes command lines.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-09-19 15:56:23 +02:00
Jan Kiszka
d1834dd6bf Release 3.1 2022-08-05 13:17:58 +02:00
Felix Moessbauer
fa8414b660 remove obsolete schroot mntpoint in kas-container
This patch is an addendum to 54ab356f and removes
the external mount used for the schroot.
This is no longer required as the schroot is now
placed in TMPDIR which is already mounted from the host.

This patch does not break backward compatibility, as neither
an ISAR release nor a KAS release happened while this
code path was active.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-03 13:01:20 +02:00
Henning Schild
71cf5dc17b add NETRC_FILE to allow passing credentials into kas home
Tools like wget and git can read credentials from $HOME/.netrc for
servers that require authentication. Allow users to pass in a .netrc
file into the kas home dir to support i.e. bitbake https fetching with
auth.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: style fix in command-line.rst]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-06-23 17:58:37 +02:00
Florian Bezdeka
2f149509c2 kas-container: Add support for podman >= 4.1
Starting with podman 4.1 the --userns=keep-id flag is no longer ignored
for privileged containers leading to an error when trying to start up
such a container (in our case: for ISAR builds):

  Error: keep-id is only supported in rootless mode

To address that we have to move the --userns=keep-id part to a non-ISAR
specific path.

Reported-by: Wadim Klincov <wadim@klincov.com>
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-06-08 16:45:18 +02:00
Marcus Folkesson
0929101104 kas-container: fix typo in help text
cleanstate -> cleansstate

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-04-06 08:12:04 +02:00
Anders Montonen
7909ee2b18 kas-container: add '--ssh-agent' option
The option mounts the SSH_AUTH_SOCK ssh agent socket, and sets the
environment variable in the container.

Signed-off-by: Anders Montonen <Anders.Montonen@iki.fi>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-03-29 09:25:23 +02:00
Jan Kiszka
680c67d756 Release 3.0.2 2022-03-08 15:52:55 +01:00
Jan Kiszka
229310958b Release 3.0.1 2022-03-08 14:31:59 +01:00
Jan Kiszka
41b52b1263 Release 3.0 2022-03-03 12:42:57 +01:00
Ryan Fairfax
32514c8f4f shell: Add option to keep current environment
When doing interactive development via kas shell it is often desirable
to keep the user's customized configuration. The new --preserve-env
argument has been added to support this scenario via an opt in flag.

This flag is blocked  when not running from a TTY or via kas-container
and kas issues a warning to the user about potential unintended side
effects when invoked.

Signed-off-by: Ryan Fairfax <rfairfax@linux.microsoft.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-02-26 09:20:53 +01:00
Schmidl, Tobias
87ce7f90d6 kas-container: pass http_proxy et.al through sudo
If used in a podman environment, it's important to pass the http proxy
information down through sudo. This does not happen by default. We also
want to pass the entire environment down to avoid missing any other
environment variable.

Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-02-03 07:05:23 +01:00
Uladzimir Bely
54ab356f9f Dockerfile.isar: support of sbuild
Upcoming support of sbuild in Isar requires some additional tools
to be preinstalled.

Also, `builder` user should be in `sbuild` group.

Additionally, to use external volume for schroot overlay because
the 'upper' overlayfs layer of sbuild can't be based on another
overlayfs filesystem that happens in case of using Docker.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-28 12:57:40 +01:00
Jan Kiszka
6abf837eab kas-container: Run enable_isar_mode only once
If both --isar is provide and build_system is set, enable_isar_mode will
be called twice that leads to KAS_CONTAINER_COMMAND gaining two "sudo" -
harmless but unneeded.

Reported-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-27 12:03:03 +01:00
Jan Kiszka
484a3dda97 kas-container: Add cleansstate and cleanall
To complete the set of clean tasks that OE and Isar offer, add
cleansstate to purge SSTATE_DIR and cleanall also delete DL_DIR.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-08 10:35:37 +01:00
Jan Kiszka
833a65124e kas-container: Factor out clean into a separate function
The logic will grow, and this better happens outside of the command line
parser.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-08 10:35:32 +01:00
Jan Kiszka
5297f7d1f3 kas-container: Start init service inside container
This helps reaping zombies if processes do not perform proper cleanups.
Known to stumble is bazel so far, see
https://github.com/bazelbuild/bazel/issues/13823. But as the overhead of
an init service is negligible and problems around this are hard to
debug, we better add this option by default.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-08 08:30:10 +01:00
Florian Bezdeka
0f5401be19 kas-container: podman: Remove --pid=host
We were using the host PID namespace as workaround for problems related
to binfmt and its missing namespace support. As it turns out after
running a bunch of tests this is no longer necessary.

This patch "reverts" 6b025e4910 ("kas-docker: Podman: Fixing isar builds
failing with exec format errors").

Test matrix:

kas layer               podman 3.4.4        podman 3.0.1       podman 3.4.4
                        on Fedora 35        on Debian 11       on Debian 11
                                                          (podman from testing)
xenomai-images
 ISAR_CROSS_COMPILE = 1     OK                 OK                 OK
 ISAR_CROSS_COMPILE = 0     OK                 OK                 OK

iot-2000                    OK                 FAIL [1]           FAIL [1]

[1] The iot-2000 layer is not ISAR based, so we do not run in privileged
mode for such builds which seems to make a difference when using the
--userns=keep-id argument. As it works on Fedora and the error message
indicates "creating of systemd unit failed" it might by systemd related.

podman run --rm -t -i --userns=keep-id debian:buster-slim
Fedora: OK
Debian: Fail
  Error: OCI runtime error: error creating systemd unit
  `libpod-<snip>.scope`: got `failed`

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-01-08 08:30:03 +01:00
Jan Kiszka
75d1a5cce4 Release 2.6.3 2021-11-29 18:19:12 +01:00
Quirin Gylstorff
2c0486d846 kas-container: Add check for KAS_REPO_REF_DIR
Check if KAS_REPO_REF_DIR exists to avoid an error during
the execution of readlink.

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-11-06 10:57:15 +01:00
Quirin Gylstorff
9b9bb00c56 kas-container: create KAS_WORK_DIR if it not exists
If KAS_WORK_DIR is set in the executing shell
and does not exists kas-container fails silently
during readlink -f on KAS_BUILD_DIR if the default
"${KAS_WORK_DIR}/build" is used.

Create KAS_WORK_DIR to ensure the subsequent
code execution.

This can be tested by setting KAS_WORK_DIR, e.g.

```
KAS_WORK_DIR="$(pwd)/kas_work" kas-container ...
```

Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-11-06 10:57:15 +01:00
Jan Kiszka
3ad03d1590 Release 2.6.2 2021-11-02 13:01:21 +01:00
Jan Kiszka
0b394f0bb7 Release 2.6.1 2021-10-22 20:22:20 +02:00
Jan Kiszka
a81ee2c2b1 Release 2.6 2021-10-22 17:40:17 +02:00
Jan Kiszka
da62e0bfbd kas-container: Add support for menu plugin
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.

Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.

Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-09 15:00:05 +02:00
Jan Kiszka
5fb9067894 Fall back to .config.yaml if no configuration file given
Make the configuration file on the command line optional and fall back
to trying to open the generated .config.yaml instead. This allows the
sequence

kas menu -> save & exit
kas build
kas shell
kas ...

and also makes rebuilding the self-configured image simpler.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-09 15:00:05 +02:00
Jan Kiszka
1271320de3 kas-container: Enter with /repo as current dir
Set KAS_WORK_DIR instead. This will be needed for running build/shell
with the implicit kas configuration file .config.yaml.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-09 15:00:05 +02:00
Jan Kiszka
6eda3ba48a kas-container: Simplify translation of file argument list
Since dabda7617f, all elements of KAS_FILES are already absolute.
Therefore, a single replacement rule is sufficient to translate them to
paths for the container.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-24 19:19:36 +02:00
Jan Kiszka
ed19ad0059 kas-container: Fix indention
No functional changes.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-07-19 07:57:32 +02:00
Jan Kiszka
8b3ff93bae kas-container: Update and improve help output
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-06-29 13:45:40 +02:00
Jan Kiszka
28891711c0 Release 2.5 2021-05-26 07:27:23 +02:00
Jan Kiszka
46eabc7be3 kas-container: Add support for checkout and for-all-repos
Allow invoking those two new sub-commands also via the container
interface.

Closes #51
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-20 16:39:39 +02:00
Rotem Bar
0e470dc644 kas-container: Fix mounting of custom KAS_REPO_REF_DIR
kas-container support for KAS_REPO_REF_DIR was broken. The path provided
by the env variable was the path on the local machine, not in the
container.

Signed-off-by: Rotem Bar <rotemb@hailo.ai>
[Jan: massaged commit log]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-10 07:30:17 +02:00
Claudius Heine
7c6e3c3994 kas-container: add --git-credential-store options
This options allows to specify a git credential store file, which is
then mounted into the container and used by kas as a
git-credential-helper.

Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: remove debug echo]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-06 14:09:10 +02:00
Jose Quaresma
6e82e0d12d kas-container: add support to set a custom container images location
There are situation that the user have a local customized container images
with tools installed that are not provided in the default kas container.

- For a local container the user can tag it and use the existing variables:

 docker tag local-container my/container:1.0
 export KAS_CONTAINER_IMAGE_PATH=my
 export KAS_CONTAINER_IMAGE_NAME=container
 export KAS_IMAGE_VERSION=1.0

- This patch improve the user interface by allowing to set a custom image
 with just a single environment variable.

 export KAS_CONTAINER_IMAGE=local-container

Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-16 14:20:58 +02:00
Claudius Heine
06240c23da kas-container: mount /repo as read-write for shell command
There are some bitbake commands like `recipetool` that allows to easily
create new or append to existing recipes. This of course only works if
the main repository that is worked with is writeable.

However it is mounted into the container only as read-only.

This patch mounts the repository writeable when the `shell` command is use
and read-only in case of the `build` command.

It also adds `--repo-ro` and `--repo-rw` to allow overwriting the
default behaviour.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-16 08:04:31 +02:00
Jan Kiszka
3f98520ab6 kas-container: Improve KAS_WORK/BUILD_DIR retrieval
Perform link resolution unconditionally to account for cases when the
default paths are links. Use default values to simplify the assignments.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-21 09:17:30 +01:00
Peter Hatina
f32290d425 Introduce KAS_BUILD_DIR environment variable
This variable can override default build path `${KAS_WORK_DIR}/build`.

Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-10 07:53:09 +01:00
Henning Schild
6b52357dc7 kas-container: add an argument to get version information
This script is deployed in many copies, give people a way to identify
which one they have cached/installed.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: simplify and use basename for the program]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-25 21:19:02 +01:00
Jan Kiszka
8fe75298e4 Release 2.4 2021-02-25 19:09:02 +01:00
Jan Kiszka
6c5e77800c kas-container: Simplify docker arguments
--privileged implies all caps, so no need to list some explicitly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-16 10:36:14 +01:00
Henning Schild
111906f804 kas-container: make sure that we pass shellcheck
There is one actual change where code gets touched, the rest are just
comments to satisfy shellcheck.
All the lines with the ignores should be reviewed later, there might be
problems behind the findings.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: add one more SC2086 suppression]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 20:42:49 +01:00
Jan Kiszka
4042ef4859 kas-container: Add support for multi-word --command arguments
Yet another special dance needed to preserve argument boundaries while
moving them around: This trick enables

kas-container shell kas.yml --command "echo it works."

by keeping the command argument separate and injecting it via
appropriate quoting into the argument array.

Closes: #42
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-29 20:29:56 +01:00
Jan Kiszka
368cdffc24 Release 2.3.3 2020-11-28 12:36:46 +01:00
Jan Kiszka
ba439d4200 Release 2.3.2 2020-11-20 12:20:24 +01:00
Jan Kiszka
6a8abf277c Add build_system property to config file
This allows to pre-select the build system, specifically avoiding
confusion when kas-container is accidentally not called with --isar for
an isar config. For that, build_system needs to be defined in the
lop-level config file passed to kas-container.

Theoretically, this also allows to combine layers which have both
oe-init-build-env and isar-init-build-env scripts.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-16 11:53:25 +01:00
Claudius Heine
36ab0549ce kas-container: allow the -d/-v of kas-container to set -d on kas
Currently kas-container did not allow to set the `-d` parameter of kas,
this patch changes that and allows to set it via the `-d` or `-v`
parameter of kas-container.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-10-22 08:40:37 +02:00
Claudius Heine
b719fc9f6a kas-container: rename -v to -d and deprecate -v
kas itself has a `-d` parameter that allows to print debug information,
while kas-container has the `-v` parameter. Since the `-d` parameter of
kas was first, rename the `-v` parameter of kas-container to fit kas.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-10-22 08:40:25 +02:00
Jan Kiszka
3403ba4923 kas-container: Drop container image prefix
The current default causes spurious pulls even when the image is already
locally available, at least with podman 2.0.x. Looking through podman
how-tos, none mentions that this prefix is required, and the
image_default_transport in containers.con is generally "docker://". So
it's safe to drop it for better default behavior.

If a custom local setup deviate, users can still override
KAS_CONTAINER_IMAGE_PATH, prepending what is desired.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-10-18 23:21:20 +02:00
Claudius Heine
7e22208849 kas-container: Fix help output
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: further massaging]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-10-02 08:44:04 +02:00