Only local repositories may be configured without refspec.
Raise an error if a repository url is given but refspec is not.
Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
How relative paths in local file includes are handles is more complex
than this one line makes out to be.
This change clarifies the documentation here.
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: dropped duplicate blank at beginning of sentence]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If used in a podman environment, it's important to pass the http proxy
information down through sudo. This does not happen by default. We also
want to pass the entire environment down to avoid missing any other
environment variable.
Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch adds the pigz package to enable parallel compression
and decompression support for the sstate artifacts.
This significantly speeds up compression of large artifacts on
build systems with many cores.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Upcoming support of sbuild in Isar requires some additional tools
to be preinstalled.
Also, `builder` user should be in `sbuild` group.
Additionally, to use external volume for schroot overlay because
the 'upper' overlayfs layer of sbuild can't be based on another
overlayfs filesystem that happens in case of using Docker.
Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If both --isar is provide and build_system is set, enable_isar_mode will
be called twice that leads to KAS_CONTAINER_COMMAND gaining two "sudo" -
harmless but unneeded.
Reported-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Break up the build into two stages, performing basic tests with poky and
isar on the two kas container types before uploading them. The tests
consist of building zlib-native with poky and cowsay for x86 bullseye
with Isar.
Note that the ordering "build kas", "test kas", "push kas",
"build kas-isar" is important because "build kas-isar" will not use the
locally built and imported kas:next image and rather pull the previous
one from the registry.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
To complete the set of clean tasks that OE and Isar offer, add
cleansstate to purge SSTATE_DIR and cleanall also delete DL_DIR.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This helps reaping zombies if processes do not perform proper cleanups.
Known to stumble is bazel so far, see
https://github.com/bazelbuild/bazel/issues/13823. But as the overhead of
an init service is negligible and problems around this are hard to
debug, we better add this option by default.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We were using the host PID namespace as workaround for problems related
to binfmt and its missing namespace support. As it turns out after
running a bunch of tests this is no longer necessary.
This patch "reverts" 6b025e4910 ("kas-docker: Podman: Fixing isar builds
failing with exec format errors").
Test matrix:
kas layer podman 3.4.4 podman 3.0.1 podman 3.4.4
on Fedora 35 on Debian 11 on Debian 11
(podman from testing)
xenomai-images
ISAR_CROSS_COMPILE = 1 OK OK OK
ISAR_CROSS_COMPILE = 0 OK OK OK
iot-2000 OK FAIL [1] FAIL [1]
[1] The iot-2000 layer is not ISAR based, so we do not run in privileged
mode for such builds which seems to make a difference when using the
--userns=keep-id argument. As it works on Fedora and the error message
indicates "creating of systemd unit failed" it might by systemd related.
podman run --rm -t -i --userns=keep-id debian:buster-slim
Fedora: OK
Debian: Fail
Error: OCI runtime error: error creating systemd unit
`libpod-<snip>.scope`: got `failed`
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This is added to avoid confusion in the creation of downstream layers.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: fixed over-long line]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Currently the containers are based on buster, but use buster-backports,
and (in case if the Isar container) even plain bullseye with pinning as
additional deb soure.
This changes that to use bullseye only.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Permits installing via wheel, even if this dependency is not yet
installed.
Reported-by: Marius Kriegerowski <marius.kriegerowski@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Adding `.` was not intuitively for me, therefore an explicit comment on
how to add both the repo and a subdir might be useful.
Signed-off-by: Tobias Schmidl <tobias@schmidl.dev>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
kas wrongly assumes so far that setting $HOME will also make ssh use the
kas provided $HOME/.ssh/ folder. But ssh will pick up the homedir from
/etc/passwd instead. This could cause kas to overwrite the users
~/.ssh/config when using SSH_PRIVATE_KEY*. We can try to cure ssh config
isolation, but that may cause surprises for users so far silently
relying on it.
For now, as a stable fix, avoid to cause damage to ~/.ssh/config in
cases where this is likely not desired, namely when there is already
config file. Warn if that file does not contain the generated content
from a previous run.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Use the more powerful configuration file .readthedocs.yaml to ensure
that all requirements are installed. This now includes also a distro
package, python-newt, something that cannot be installed via a
requirements.txt.
Fixes the doc build for 2.6.x which become broken with the addition of
the menu plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Check if KAS_REPO_REF_DIR exists to avoid an error during
the execution of readlink.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If KAS_WORK_DIR is set in the executing shell
and does not exists kas-container fails silently
during readlink -f on KAS_BUILD_DIR if the default
"${KAS_WORK_DIR}/build" is used.
Create KAS_WORK_DIR to ensure the subsequent
code execution.
This can be tested by setting KAS_WORK_DIR, e.g.
```
KAS_WORK_DIR="$(pwd)/kas_work" kas-container ...
```
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The github CI environment causes the copied /kas folder to become
world-writable. This is at least undesired, so fix up the permissions.
This is done the classic way, i.e. via a separate chmod, rather than
using the new and not yet widely available "COPY --chmod".
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Moving this file to /usr/local/bin broke user configurations that had
/usr/bin/ hard-coded, and it also broke inside the kas sandbox which
does not have /usr/local/bin in its PATH. So move things back to the
original location.
Do that by setting a link to the /kas/contrib folder, rather than
copying the file once more.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This prevents adding package_data to the source distribution, which we
have to.
Reported by: Ralf Beier <rbeier57@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The `this` repo is skipped in the test for the `for-all-repos` command.
Adapt to checking for KAS_REPO_URL being empty instead of comparing the
KAS_REPO_NAME. This way the test will fail if the implementation
diverges from the documentation:
> KAS_REPO_URL: The URL from which this repository was cloned, or an
> empty string if no remote URL was given in the config file.
Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
repo.url will point the repo.path in that case, but we want the user to
see an empty KAS_REPO_URL, as also documented.
Reported-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Irrespective of build system (OE / Yocto / Isar), they all have
BBPATH = "${TOPDIR}"
BBFILES ?= ""
in their bblayers_conf_header fragments. This could be dropped if kas
wrote those standard assignments already. So let's do that, weakly so
that those could still be overwritten via bblayers_conf_header.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This checks basic features of kas menu, some option selection, yaml
including, rebuilding and target selection.
The UI is operated by mocking smack's GridFormHelp.runOnce and some
result evaluation functions. Injecting keys from the keyboard was not
easily possibly (if at all), thus this approach. The menu rendering is
not shown due to pytest.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.
Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.
Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make the configuration file on the command line optional and fall back
to trying to open the generated .config.yaml instead. This allows the
sequence
kas menu -> save & exit
kas build
kas shell
kas ...
and also makes rebuilding the self-configured image simpler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Set KAS_WORK_DIR instead. This will be needed for running build/shell
with the implicit kas configuration file .config.yaml.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This uses the kconfiglib python module to implement simple interactive
configuration for complex layers with many kas options and even variable
input. See the menu.py documentation header for details on the Kconfig
format and its translation to kas.
Rather than using the limited menuconfig implementation of kconfiglib,
this comes with an own, python-newt based version. The permits smooth
integration in the workflow, e.g. to offer an option that directly
triggers a build after completing a configuration. And it comes with a
nicer layout.
Unfortunately, python-newt can't be fulfilled via pip. But it is
available as proper package via common distros. So warn if the package
is missing.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to define plugins which do not need have any of common args.
Move setup_parser_common_args into libkas for this as it now becomes a
library function.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This avoids pulling in anything unexpected and then running it as part
of the test and later on image build process.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Avoids that succeeding tests may stumble over this. So far not an issue.
Drop a forgotten debug print at this chance.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
- Easier to reference as schema for IDEs
- Can be added to schemastore.org
Signed-off-by: ciarancourtney <ciaran.courtney@activeenergy.ie>
[Jan: rebased, merged in schema installation]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since 2077900b4e, the existence of a repo key is enforced by the
config schema. So we can drop the redundant check in the include
handler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Commit 2077900b4e was wrong in claiming that the documentation already
demanded repo-relative paths for local includes. It only did that for
dictionary-based include entries. Aligning both to repo-relative paths
remain a value, but we definitely have to update the documentation now.
While at it, also update the code comments accordingly. That also still
talked about the repo key being option, something that was never the
case in practice.
Reported-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Using ssh to authenticate against repositories to check out is enabled.
When switching to https and authentication via the git credential helper
it is necessary to first setup the home directory before checking out the
repositories.
Signed-off-by: Ralf Anton Beier <ralf_beier@me.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
When packaging kas for OpenSUSE, rpmlint threw the following warnings:
RPMLINT report:
===============
kas.noarch: W: non-executable-script /usr/lib/python3.6/site-packages/kas/__main__.py 644 /usr/bin/env python3
kas.noarch: W: non-executable-script /usr/lib/python3.6/site-packages/kas/kas.py 644 /usr/bin/env python3
This text file contains a shebang or is located in a path dedicated for
executables, but lacks the executable bits and cannot thus be executed. If
the file is meant to be an executable script, add the executable bits,
otherwise remove the shebang or move the file elsewhere.
2 packages and 0 specfiles checked; 0 errors, 2 warnings.
The Python files inside the kas module don't need shebang lines as they
are not executed directly so we can just remove them.
Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This imports revision aa9b9dc9a9 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael.lisicki@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
KAS should exit with the appropriate exit code, if the command parsed to the for_all_repos
plugin fails to apply on one of the repositories.
Example use-case: This allows for CI pipelines or wrapper scripts to take appropriate action (e.g. fail),
if an issue occured within the KAS for_all_repos command.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
