repo.url will point the repo.path in that case, but we want the user to
see an empty KAS_REPO_URL, as also documented.
Reported-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Irrespective of build system (OE / Yocto / Isar), they all have
BBPATH = "${TOPDIR}"
BBFILES ?= ""
in their bblayers_conf_header fragments. This could be dropped if kas
wrote those standard assignments already. So let's do that, weakly so
that those could still be overwritten via bblayers_conf_header.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This checks basic features of kas menu, some option selection, yaml
including, rebuilding and target selection.
The UI is operated by mocking smack's GridFormHelp.runOnce and some
result evaluation functions. Injecting keys from the keyboard was not
easily possibly (if at all), thus this approach. The menu rendering is
not shown due to pytest.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.
Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.
Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make the configuration file on the command line optional and fall back
to trying to open the generated .config.yaml instead. This allows the
sequence
kas menu -> save & exit
kas build
kas shell
kas ...
and also makes rebuilding the self-configured image simpler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Set KAS_WORK_DIR instead. This will be needed for running build/shell
with the implicit kas configuration file .config.yaml.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This uses the kconfiglib python module to implement simple interactive
configuration for complex layers with many kas options and even variable
input. See the menu.py documentation header for details on the Kconfig
format and its translation to kas.
Rather than using the limited menuconfig implementation of kconfiglib,
this comes with an own, python-newt based version. The permits smooth
integration in the workflow, e.g. to offer an option that directly
triggers a build after completing a configuration. And it comes with a
nicer layout.
Unfortunately, python-newt can't be fulfilled via pip. But it is
available as proper package via common distros. So warn if the package
is missing.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to define plugins which do not need have any of common args.
Move setup_parser_common_args into libkas for this as it now becomes a
library function.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This avoids pulling in anything unexpected and then running it as part
of the test and later on image build process.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Avoids that succeeding tests may stumble over this. So far not an issue.
Drop a forgotten debug print at this chance.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
- Easier to reference as schema for IDEs
- Can be added to schemastore.org
Signed-off-by: ciarancourtney <ciaran.courtney@activeenergy.ie>
[Jan: rebased, merged in schema installation]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since 2077900b4e, the existence of a repo key is enforced by the
config schema. So we can drop the redundant check in the include
handler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Commit 2077900b4e was wrong in claiming that the documentation already
demanded repo-relative paths for local includes. It only did that for
dictionary-based include entries. Aligning both to repo-relative paths
remain a value, but we definitely have to update the documentation now.
While at it, also update the code comments accordingly. That also still
talked about the repo key being option, something that was never the
case in practice.
Reported-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Using ssh to authenticate against repositories to check out is enabled.
When switching to https and authentication via the git credential helper
it is necessary to first setup the home directory before checking out the
repositories.
Signed-off-by: Ralf Anton Beier <ralf_beier@me.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
When packaging kas for OpenSUSE, rpmlint threw the following warnings:
RPMLINT report:
===============
kas.noarch: W: non-executable-script /usr/lib/python3.6/site-packages/kas/__main__.py 644 /usr/bin/env python3
kas.noarch: W: non-executable-script /usr/lib/python3.6/site-packages/kas/kas.py 644 /usr/bin/env python3
This text file contains a shebang or is located in a path dedicated for
executables, but lacks the executable bits and cannot thus be executed. If
the file is meant to be an executable script, add the executable bits,
otherwise remove the shebang or move the file elsewhere.
2 packages and 0 specfiles checked; 0 errors, 2 warnings.
The Python files inside the kas module don't need shebang lines as they
are not executed directly so we can just remove them.
Signed-off-by: Paul Barker <paul@pbarker.dev>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This imports revision aa9b9dc9a9 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael.lisicki@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
KAS should exit with the appropriate exit code, if the command parsed to the for_all_repos
plugin fails to apply on one of the repositories.
Example use-case: This allows for CI pipelines or wrapper scripts to take appropriate action (e.g. fail),
if an issue occured within the KAS for_all_repos command.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
An upcoming change will introduce returning non-zero status on the KAS
command if the for_all_repos command fails for one of the repos defined
in the kas configuration. This caused an issue with the non-git "this"
repo defined in the test.yml.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: adjusted commit log to reflect different commit ordering]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since dabda7617f, all elements of KAS_FILES are already absolute.
Therefore, a single replacement rule is sufficient to translate them to
paths for the container.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This stresses the previously broken case of the incorrectly detected
repository root while including files.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
So far, repository paths for local includes were derived from the path
name of the config file containing the include, rather than using the
actual repository root as specified in the documentation. No one
complained so far, some layers simply adjusted their includes to this
inconsistency which was only discovered during refactorings.
Fix this issue by passing also the repository path along with the config
filename down the recursive _internal_include_handler calls. The
top-level repo path now needs to be retrieved before the creation of
IncludeHandler and passed to it then. This has the side effect of
enabling deviating top-level repo paths, a feature that will be used by
the upcoming changes for a .config.yaml file in KAS_WORK_DIR.
As there are existing users of the old behavior out there, fall back to
it if a local include cannot be found under the correct path and warn if
this succeeds. This allows smooth migration of layers to the right
pattern as they update their kas version.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We will fail differently then, but with a nicer error message (missing
init-build-env script) than
ValueError: Set of coroutines/Futures is empty.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
No one asked for this so far but the result of providing a relative
path, e.g. to KAS_WORK_DIR, is so confusing that it is better to simply
resolve all those vars to absolute paths.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
OpenEmbedded Core is about to depend on lz4c, pzstd, and zstd being on
the host. The Dockerfile was already installing zstd which provides
zstd/pzstd, so also install lz4 for lz4c.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Debian Buster comes with version 2.7.1 of git-lfs which does not support
git+ssh yet, therefore we need a newer version of git-lfs for cloning a
repository which uses LFS.
Signed-off-by: Steffen Hieber <steffen.hieber@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Backport repos have a priority of 100 by default, thus can just be added
with extra pinning. Use explicit release selection on the installation
of qemu-user-static instead, that's simpler.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
By default git only fetches references under the refs/heads/ tree,
this patch adds support to kas to enable you to specify references
outside of the ref/heads tree. This is useful as it allows you to
use uncommitted gerrit patchsets, Gitlab merge requests or github
pull requests that live under refs/changes/, refs/merge-requests
and refs/pull as the reference for a repo allowing the use of
in development changes. When a refsepc is defined that starts
with refs/ an additional git fetch operation is preformed on the
repo to explicitly fetch the reference given so it can be checked
out for use.
Signed-off-by: Drew Reed <drew.reed@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since 7b18e5ec3b, which was released with 0.19.0, proxy_config has no
effect anymore. No one seems to have missed it, likely because setting
the proxies via the config file was a mistake anyway. The environment
variables are the proper way.
Declare these config entries officially obsolete and warn if they still
hang around in some legacy config. We will drop them from the schema in
one of the next releases.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Additionally to the variable SSH_PRIVATE_KEY, which can be used to
reference a private key saved as a string within a variable (the previous
documentation falsely stated, that it references a path and has been
adjusted), the new variable SSH_PRIVATE_KEY_FILE can now be used to reference
the path to a file containing the private key.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: Fix logic for detecting set env vars]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If command line includes refer to the same repo, missing_repos was
aggregating those multiple times, causing multiple clone procedures to
be triggered. This generally failed. Therefore, build missing_repos
while checking for duplicates.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
kas-container support for KAS_REPO_REF_DIR was broken. The path provided
by the env variable was the path on the local machine, not in the
container.
Signed-off-by: Rotem Bar <rotemb@hailo.ai>
[Jan: massaged commit log]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This options allows to specify a git credential store file, which is
then mounted into the container and used by kas as a
git-credential-helper.
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: remove debug echo]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to set a git credential helper in the .gitconfig, in order
to handle git authentications. For instance when bitbake recipes require
to clone from git via https.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
when importing nested layer in combination with the top level layer like
meta-security
meta-security/meta-tpm
one can use
layers:
.:
meta-tpm:
in the spec file to import the parent layer.
This leads to the effect that
.
meta-tpm = "HEAD:<hash>"
is shown in the build console and in /etc/build
when using image-buildinfo.bbclass.
To fix that just strip the trailing dot from each layer name.
Add test to avoid future regressions
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
[Jan: use only a single rstrip call, fix style]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
To expand the usability of the standard Docker image, move sudo
enablement from Docker.isar.
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There are situation that the user have a local customized container images
with tools installed that are not provided in the default kas container.
- For a local container the user can tag it and use the existing variables:
docker tag local-container my/container:1.0
export KAS_CONTAINER_IMAGE_PATH=my
export KAS_CONTAINER_IMAGE_NAME=container
export KAS_IMAGE_VERSION=1.0
- This patch improve the user interface by allowing to set a custom image
with just a single environment variable.
export KAS_CONTAINER_IMAGE=local-container
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There are some bitbake commands like `recipetool` that allows to easily
create new or append to existing recipes. This of course only works if
the main repository that is worked with is writeable.
However it is mounted into the container only as read-only.
This patch mounts the repository writeable when the `shell` command is use
and read-only in case of the `build` command.
It also adds `--repo-ro` and `--repo-rw` to allow overwriting the
default behaviour.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Skopeo and Umoci are needed for the new ISAR SDK Container target.
A "manual" backport of Skopeo (and only Skopeo) is needed, since the
repository "buster-backports" doesn't provide it.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
[Jan: add golang-github-containers-{image,common} to buster list]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We often see Issues being used while they maybe should not. Introducing
an issue template might help avoid confusion and inviting contributors
to the list without disabling Issues.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Perform link resolution unconditionally to account for cases when the
default paths are links. Use default values to simplify the assignments.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Otherwise the descriptions of KAS_WORK_DIR and KAS_BUILD_DIR are merged
together when rendering on kas.readthedocs.io.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
