As we no longer create the builder user at runtime, placing data into
/etc/skel at runtime is semantically not correct anymore. Instead, we
bind mount host paths below /var/kas/userdata. By that, we now place
the data into a directory which is fully handled by us.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
With the added auto-caching logic of the referenced repos in 1c2c859,
kas-container has to mount this directory in read-write mode.
Otherwise, initial clones will fail.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The user environment may lack /usr/sbin while certain podman
configuration will need, e.g., iptables for the setup. This can cause
Error: plugin type="bridge" failed (add): cni plugin bridge failed: failed to locate iptables: exec: "iptables": executable file not found in $PATH
Resolved that by appending /usr/sbin to the PATH in privileged podman
mode.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Already create the builder user/group during container image build and
only align the IDs in the entrypoint if started with a non-zero USER_ID.
The primary gain is code simplification because this removes some
dynamics from the entrypoint.
As this refactoring avoids that gitlab-ci runners start the container as
root, it was also supposed to resolve the mismatch between the owner of
the checked-out repo and builder user. Unfortunately, this does not work
yet, and the reason is still unclear.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The current detection mechanism assumes that if the docker command
is available the engine behind is also docker. In fact nowadays a
lot of people use /usr/bin/docker as an alias for podman. In that
case the script expects a docker engine and misses to use
podman-specific settings.
To fix this, run "docker -v" and check if the output indicates a
podman engine or a real docker engine. Otherwise error out as the
enginge is not supported.
Signed-off-by: Frieder Schrempf <frieder.schrempf@kontron.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch adds support to use the dump plugin in kas-container.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The kas-container trace / debug infrastructure has to write to stderr to
not interfer with kas output to stdout. Note, that the kas debug
messages already write to stderr. By that, the kas-container script can
be used in pipes similar to a direct invocation of kas.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch forwards the DISTRO_APT_PREMIRRORS environment variable into
the container when using the kas-container script. This variable is
commonly used to externally set mirrors for debian apt, e.g. in a CI
environment.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch forwards environment variables to control the concurrency and
parallelism of the build to bitbake.
With that, the user can reduce the concurrency in case of running short
on memory. Further, by reducing the concurrency on very large and NUMA
machines, the build sometimes can be significantly speed up.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Container engines per default log all container output into journald
(Podman) or a json file (Docker).
A build with kas-container/bitbake containes lots of useless build
output like progress lines, etc.
In case of Podman, where journald is used, this means that all this is
spammed into the journald buffer and possible also send to ttys or even
serial ttys. This might make sense for containers that output only
status information about running services, but for a build process, this
is mostly just spamming and causes important information to be lost.
Since the stdout/stderr output of the container is used for
kas-container ordinarily, the background logging from the container
image can be deactivated.
Therefor add the `--log-driver=none` parameter to all container
runtimes command lines.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch is an addendum to 54ab356f and removes
the external mount used for the schroot.
This is no longer required as the schroot is now
placed in TMPDIR which is already mounted from the host.
This patch does not break backward compatibility, as neither
an ISAR release nor a KAS release happened while this
code path was active.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Tools like wget and git can read credentials from $HOME/.netrc for
servers that require authentication. Allow users to pass in a .netrc
file into the kas home dir to support i.e. bitbake https fetching with
auth.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: style fix in command-line.rst]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Starting with podman 4.1 the --userns=keep-id flag is no longer ignored
for privileged containers leading to an error when trying to start up
such a container (in our case: for ISAR builds):
Error: keep-id is only supported in rootless mode
To address that we have to move the --userns=keep-id part to a non-ISAR
specific path.
Reported-by: Wadim Klincov <wadim@klincov.com>
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The option mounts the SSH_AUTH_SOCK ssh agent socket, and sets the
environment variable in the container.
Signed-off-by: Anders Montonen <Anders.Montonen@iki.fi>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
When doing interactive development via kas shell it is often desirable
to keep the user's customized configuration. The new --preserve-env
argument has been added to support this scenario via an opt in flag.
This flag is blocked when not running from a TTY or via kas-container
and kas issues a warning to the user about potential unintended side
effects when invoked.
Signed-off-by: Ryan Fairfax <rfairfax@linux.microsoft.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If used in a podman environment, it's important to pass the http proxy
information down through sudo. This does not happen by default. We also
want to pass the entire environment down to avoid missing any other
environment variable.
Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Upcoming support of sbuild in Isar requires some additional tools
to be preinstalled.
Also, `builder` user should be in `sbuild` group.
Additionally, to use external volume for schroot overlay because
the 'upper' overlayfs layer of sbuild can't be based on another
overlayfs filesystem that happens in case of using Docker.
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If both --isar is provide and build_system is set, enable_isar_mode will
be called twice that leads to KAS_CONTAINER_COMMAND gaining two "sudo" -
harmless but unneeded.
Reported-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
To complete the set of clean tasks that OE and Isar offer, add
cleansstate to purge SSTATE_DIR and cleanall also delete DL_DIR.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This helps reaping zombies if processes do not perform proper cleanups.
Known to stumble is bazel so far, see
https://github.com/bazelbuild/bazel/issues/13823. But as the overhead of
an init service is negligible and problems around this are hard to
debug, we better add this option by default.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We were using the host PID namespace as workaround for problems related
to binfmt and its missing namespace support. As it turns out after
running a bunch of tests this is no longer necessary.
This patch "reverts" 6b025e4910 ("kas-docker: Podman: Fixing isar builds
failing with exec format errors").
Test matrix:
kas layer podman 3.4.4 podman 3.0.1 podman 3.4.4
on Fedora 35 on Debian 11 on Debian 11
(podman from testing)
xenomai-images
ISAR_CROSS_COMPILE = 1 OK OK OK
ISAR_CROSS_COMPILE = 0 OK OK OK
iot-2000 OK FAIL [1] FAIL [1]
[1] The iot-2000 layer is not ISAR based, so we do not run in privileged
mode for such builds which seems to make a difference when using the
--userns=keep-id argument. As it works on Fedora and the error message
indicates "creating of systemd unit failed" it might by systemd related.
podman run --rm -t -i --userns=keep-id debian:buster-slim
Fedora: OK
Debian: Fail
Error: OCI runtime error: error creating systemd unit
`libpod-<snip>.scope`: got `failed`
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Check if KAS_REPO_REF_DIR exists to avoid an error during
the execution of readlink.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If KAS_WORK_DIR is set in the executing shell
and does not exists kas-container fails silently
during readlink -f on KAS_BUILD_DIR if the default
"${KAS_WORK_DIR}/build" is used.
Create KAS_WORK_DIR to ensure the subsequent
code execution.
This can be tested by setting KAS_WORK_DIR, e.g.
```
KAS_WORK_DIR="$(pwd)/kas_work" kas-container ...
```
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.
Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.
Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make the configuration file on the command line optional and fall back
to trying to open the generated .config.yaml instead. This allows the
sequence
kas menu -> save & exit
kas build
kas shell
kas ...
and also makes rebuilding the self-configured image simpler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Set KAS_WORK_DIR instead. This will be needed for running build/shell
with the implicit kas configuration file .config.yaml.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since dabda7617f, all elements of KAS_FILES are already absolute.
Therefore, a single replacement rule is sufficient to translate them to
paths for the container.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
kas-container support for KAS_REPO_REF_DIR was broken. The path provided
by the env variable was the path on the local machine, not in the
container.
Signed-off-by: Rotem Bar <rotemb@hailo.ai>
[Jan: massaged commit log]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This options allows to specify a git credential store file, which is
then mounted into the container and used by kas as a
git-credential-helper.
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: remove debug echo]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There are situation that the user have a local customized container images
with tools installed that are not provided in the default kas container.
- For a local container the user can tag it and use the existing variables:
docker tag local-container my/container:1.0
export KAS_CONTAINER_IMAGE_PATH=my
export KAS_CONTAINER_IMAGE_NAME=container
export KAS_IMAGE_VERSION=1.0
- This patch improve the user interface by allowing to set a custom image
with just a single environment variable.
export KAS_CONTAINER_IMAGE=local-container
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
