Until now all environment variables in the kas config's
env section require a default value.
In some cases we want to rely on the weak assignment
of bitbake instead of relying on the default value of the
variable in the kas config.
This means if we assign the 'None' value or nothing to the variable
in the kas config then it is only added to BB_ENV_EXTRAWHITE or
BB_ENV_PASSTHROUGH_ADDITIONS without setting it as environment
variable.
Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
[Jan: fixed overlong line]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch forwards environment variables to control the concurrency and
parallelism of the build to bitbake.
With that, the user can reduce the concurrency in case of running short
on memory. Further, by reducing the concurrency on very large and NUMA
machines, the build sometimes can be significantly speed up.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Tools like wget and git can read credentials from $HOME/.netrc for
servers that require authentication. Allow users to pass in a .netrc
file into the kas home dir to support i.e. bitbake https fetching with
auth.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
[Jan: style fix in command-line.rst]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Gitlab CI predefine many variables in its environment, among them the
sever hostname and a token that can be used to authenticate with the
server. If we find these variables in the environment add the
credentials to .netrc which in turn allow git and other tools to
access resources found on the CI server.
Signed-off-by: Alban Bedel <alban.bedel@aerq.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make "null" an allowed type for the `url` and `path` properties of a
repository. This allows "deleting" them in an override kas-file to e.g.
point a repository at a local directory instead of a git remote:
"Original" file:
repos:
meta-foo:
url: https://example.org/foo.git
refspec: badc0ffee
Override:
repos:
meta-foo:
url:
path: /path/to/local/foo
Without the explicit "nulling" of the URL, kas would start messing with
the repo in /path/to/local/foo (changing remote URL, switching to a
different rev, etc.). With this change, we can instead force kas to use
the recipes in that directory untouched.
Signed-off-by: Harald Seiler <hws@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
How relative paths in local file includes are handles is more complex
than this one line makes out to be.
This change clarifies the documentation here.
Signed-off-by: Claudius Heine <ch@denx.de>
[Jan: dropped duplicate blank at beginning of sentence]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This is added to avoid confusion in the creation of downstream layers.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
[Jan: fixed over-long line]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Adding `.` was not intuitively for me, therefore an explicit comment on
how to add both the repo and a subdir might be useful.
Signed-off-by: Tobias Schmidl <tobias@schmidl.dev>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Use the more powerful configuration file .readthedocs.yaml to ensure
that all requirements are installed. This now includes also a distro
package, python-newt, something that cannot be installed via a
requirements.txt.
Fixes the doc build for 2.6.x which become broken with the addition of
the menu plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Irrespective of build system (OE / Yocto / Isar), they all have
BBPATH = "${TOPDIR}"
BBFILES ?= ""
in their bblayers_conf_header fragments. This could be dropped if kas
wrote those standard assignments already. So let's do that, weakly so
that those could still be overwritten via bblayers_conf_header.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This uses the kconfiglib python module to implement simple interactive
configuration for complex layers with many kas options and even variable
input. See the menu.py documentation header for details on the Kconfig
format and its translation to kas.
Rather than using the limited menuconfig implementation of kconfiglib,
this comes with an own, python-newt based version. The permits smooth
integration in the workflow, e.g. to offer an option that directly
triggers a build after completing a configuration. And it comes with a
nicer layout.
Unfortunately, python-newt can't be fulfilled via pip. But it is
available as proper package via common distros. So warn if the package
is missing.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Commit 2077900b4e was wrong in claiming that the documentation already
demanded repo-relative paths for local includes. It only did that for
dictionary-based include entries. Aligning both to repo-relative paths
remain a value, but we definitely have to update the documentation now.
While at it, also update the code comments accordingly. That also still
talked about the repo key being option, something that was never the
case in practice.
Reported-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Since 7b18e5ec3b, which was released with 0.19.0, proxy_config has no
effect anymore. No one seems to have missed it, likely because setting
the proxies via the config file was a mistake anyway. The environment
variables are the proper way.
Declare these config entries officially obsolete and warn if they still
hang around in some legacy config. We will drop them from the schema in
one of the next releases.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Additionally to the variable SSH_PRIVATE_KEY, which can be used to
reference a private key saved as a string within a variable (the previous
documentation falsely stated, that it references a path and has been
adjusted), the new variable SSH_PRIVATE_KEY_FILE can now be used to reference
the path to a file containing the private key.
Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: Fix logic for detecting set env vars]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to set a git credential helper in the .gitconfig, in order
to handle git authentications. For instance when bitbake recipes require
to clone from git via https.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Otherwise the descriptions of KAS_WORK_DIR and KAS_BUILD_DIR are merged
together when rendering on kas.readthedocs.io.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This variable can override default build path `${KAS_WORK_DIR}/build`.
Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Comes closer to how the command is spelled. We can't uses dashes as that
is not compatible with the python syntax.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to pre-select the build system, specifically avoiding
confusion when kas-container is accidentally not called with --isar for
an isar config. For that, build_system needs to be defined in the
lop-level config file passed to kas-container.
Theoretically, this also allows to combine layers which have both
oe-init-build-env and isar-init-build-env scripts.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This plugin checks out all necessary repositories and sets up the build
directory as specificed in the configuration file.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This plugin runs a specified command in all checked out repositories.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The docstrings for the build and shell plugins are extended so that they
clearly describe each plugin and give brief examples of their use.
Also fix a typo in the build command help message.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The introduction page does not include installation instructions.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This change makes the user guide page more readable and results in a
command line usage page which is easy to navigate as all sub-commands
can be found in the left-hand outline panel.
Also fix the layout of the environment variables table by using a
replacement.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
* Add plugins module to the class reference documentation.
* Drop documentation of member classes and functions within plugins as
the these aren't intended to be called directly outside of the plugin
class itself.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
All images are now also available on github's new container registry.
Refer to that as default, both in kas-docker as well as in the user
manual. The latter can benefit from some refreshment at this chance.
Automatic deployment via CI will be switched separately.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Also update config file version to 9 due to this format change.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
[Jan: style fix]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Python 2 is dead, let's not encourage people to add support for it.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This informaition is out-of-sync with the upstream supported Yocto
Project versions and gives the impression that kas is unmaintained.
kas should work with any recent Yocto version.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Bitbake provides a S3 fetcher (`lib/bb/fetch2/s3.py`), however the
`AWS_CONFIG_FILE` and `AWS_SHARED_CREDENTIALS_FILE` variables are not
transfered to the environment of the fetcher command (`aws` in this
case) in the `runfetchcmd` function (`lib/bb/fetch2/__init__.py`).
That means the location of these files need to be the default one, so
copying it in KAS to the new HOME directory is necessary.
This patch implements the copying of those files if the
`AWS_CONFIG_FILE` and `AWS_SHARED_CREDENTIALS_FILE` are set.
Per default there variables are not available in the bitbake
environment. To have them available there a `env` entry in the kas
configuration will be necessary.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
'qemu' is not a valid machine. So, let's use the default one defined in
poky -> 'qemux86-64' [1]
Fixes:
ERROR: OE-core's config sanity checker detected a potential misconfiguration.
Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
Following is the list of potential problems / advisories:
MACHINE=qemu is invalid. Please set a valid MACHINE in your local.conf, environment or other configuration file.
[1] - https://git.yoctoproject.org/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample#n37
Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Analogously to bitbake's PREMIRRORS, this allows to define alternative
sources for repo URLs specified in kas files.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
pylint has been a great tool to get the initial code base into shape
but since then we mostely false postives. All those warnings needed to
be annonated.
This allong wouldn't be reason enough to remove it. The main problem
is that the pylint version on gitlab is updated in the back and
suddenly we get new warnings which are bogus.
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
This extends the command line syntax for specifying configuration files.
You can now combine files by concatenating them, separated by colons:
kas build base.yml:board.yml:feature.yml
The motivation for this feature is to avoid having to write tons of
configuration files that perform this combinations statically via
includes.
In order to avoid complications and prevent that users shoot themselves
too easily into their feet, we deny the case of distributing the
configuration files over multiple repositories. Either all files
specified on the command line come from the same repo, or they are all
local (without versioning control).
Based on idea by Claudius Heine.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
pycodestyle v2.4.0 checks for W606 (async deprecated), but
we can't disable it on source code level with '# noqa'. Let's disable
the check globaly. We will still catch it with pylint.
By disabling the W606 we need also to list the W503 which was
previously active.
While at it fix following things:
./docs/conf.py:252:6: E121 continuation line under-indented for hanging indent
./docs/conf.py:257:6: E126 continuation line over-indented for hanging indent
./kas/build.py:123:17: W504 line break after binary operator
./kas/configschema.py:48:25: E123 closing bracket does not match indentation of opening bracket's line
./kas/configschema.py:49:21: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:82:13: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:121:13: E123 closing bracket does not match indentation of opening bracket's line
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
With this commit its now possible to patch 3rd party repos before bitbake is started.
Example:
This is our repo:
.
├── kas.yml
├── first-patch.patch
└── quilt-patches
├── second-patch.patch
├── third-patch.patch
└── series
Content of kas.yml:
header:
version: 8
repos:
my:
third-party:
url: "git://example.com/third-party.git"
refspec: "35adf4...34"
patches:
01-first:
repo: my
path: "first-patch.patch"
02-second:
repo: my
path: "quilt-patches"
Currently only 'git' repositories can be patched.
Signed-off-by: Claudius Heine <ch@denx.de>
If a repo path is relative, append the kas work dir to it.
Background: The path ends up in bblayers.conf
Without this change the following example will fail:
--------------------------------------------------------
header:
version: 4
machine: qemux86
repos:
poky:
url: "https://git.yoctoproject.org/git/poky"
refspec: "rocko"
path: "thirdparty/poky"
layers:
meta:
meta-poky:
--------------------------------------------------------
Signed-off-by: Georg Lutz <georg@georglutz.de>
This will be used to carry the type of repo we are talking about. At the
moment it defaults to "git".
Signed-off-by: Henning Schild <henning.schild@siemens.com>
pip warns us with:
"Running pip install with root privileges is generally not a good
idea. Try `pip3 install --user` instead."
So update the documentation to install kas as user.
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
