Refactored repo path calculation eliminates duplicated code and aligns
the matching logic (with respect to url and path) with the
documentation.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The `repo` documentation says:
"If the url is missing and the path defined, this entry references the
directory the path points to. [...] In case of a relative path name
`kas_work_dir` is prepended."
But if no `url` is given and a relative path is given in `path`, then
the layer paths being added to `bblayer.conf` are missing the
`kas_work_dir` prefix.
This patch fixes it.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Disable warn_on_full_buffer via set_wakeup_fd. This avoids tons of
Exception ignored when trying to write to the signal wakeup fd:
BlockingIOError: [Errno 11] Resource temporarily unavailable
that are currently issued after running complex builds. Only works since
Python 3.7, though, so this remains best-effort. Fortunately, the Debian
in our container image is 3.7.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Escape '&' '\&' as sed replace & with the search term.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Fixes:
"Unable to process command '::set-env name=RELEASE_VERSION::2.3'
successfully.
The `set-env` command is disabled. Please upgrade to using Environment
Files [...]."
To make the next release smoother, hopefully.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Comes closer to how the command is spelled. We can't uses dashes as that
is not compatible with the python syntax.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The environment for executing the command in each repository is extended
to include the variables KAS_REPO_NAME, KAS_REPO_PATH, KAS_REPO_URL and
KAS_REPO_REFSPEC. This allows more complex actions to be automated via
this command.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The Repo.factory method determines the name of each repo from either a
'name' property from the key used for this repo within the repos
dictionary. The Repo object should be initialised with this name instead
of dropping it and falling back to the basename of the path.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows to pre-select the build system, specifically avoiding
confusion when kas-container is accidentally not called with --isar for
an isar config. For that, build_system needs to be defined in the
lop-level config file passed to kas-container.
Theoretically, this also allows to combine layers which have both
oe-init-build-env and isar-init-build-env scripts.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The init script is a bit noisy, so replace it with a direct call of
update-binfmts. That tool unfortunately fails to report missing kernel
support via a proper error code. Catch that by doing an extra check on
the mounted filesystem.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This plugin checks out all necessary repositories and sets up the build
directory as specificed in the configuration file.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This plugin runs a specified command in all checked out repositories.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The list of steps to be executed within a macro is split into setup
commands, main commands and cleanup commands to ensure that new steps
are added to the appropriate place in the execution order by plugins.
By default, the setup and cleanup commands are set to the default list
of steps which the build and shell plugins have in common. It's expected
that most future plugins will also share these common setup and cleanup
steps but we do provide arguments to the Macro initialiser to skip these
if customisation is needed.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This makes the body of the Shell.run function closely match Build.run so
that we can factor out common code.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This variable was only set in the shell plugin to disable the normal
behaviour of the FinishSetupRepos step. We can achieve the same result
by not adding the FinishSetupRepos step when we wish to keep the config
in the build environment unchanged.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
"Line break occurred before a binary operator (W503)" goes against PEP8
and has been superseded by W504.
See https://www.flake8rules.com/rules/W503.html.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
As we still support 3.5..3.7, we need to catch the case that
asyncio.get_running_loop and asyncio.all_tasks are not yet available.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Only tasks should be provided to it, no longer coroutines. Python 3.9
starts to complain about the deprecated pattern.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The target argument is only used when initializing the build plugin so
we can make it optional. This allows us to simplify the call to Config()
in the shell plugin and in other future plugins.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This fixes the missing sub-command entries in the User Guide section of
the docs by allowing the commands and arguments introduced by the
plugins to be seen by sphinx when building the docs.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The docstrings for the build and shell plugins are extended so that they
clearly describe each plugin and give brief examples of their use.
Also fix a typo in the build command help message.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The introduction page does not include installation instructions.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This change makes the user guide page more readable and results in a
command line usage page which is easy to navigate as all sub-commands
can be found in the left-hand outline panel.
Also fix the layout of the environment variables table by using a
replacement.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
* Add plugins module to the class reference documentation.
* Drop documentation of member classes and functions within plugins as
the these aren't intended to be called directly outside of the plugin
class itself.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
According to git-rev-parse(1) "-q" only does something in combination
with "--verify". Skip it, we do not need "--verify" because HEAD is
exactly one arg.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
It would be nice to have zstd frontend installed in the docker
environment. Since this is a relatively new and trending compression
algorithm, some packages might use it as part of their SRC's.
Also debian buster's tar has incorporated zstd support from upstream
GNU Tar version 1.31, starting from 1.30+dfsg-3. Therefore '.tar.zst'
extension files would be recognized and treated correctly by the tar
utility during unpack, etc.
Signed-off-by: Mert Kirpici <mert.kirpici@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Collecting all declared multiconfigs in a set rather than a list
avoids duplicates in the resulting string when multiple targets
with the same multiconfig are declared.
Signed-off-by: Mert Kirpici <mert.kirpici@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Currently kas-container did not allow to set the `-d` parameter of kas,
this patch changes that and allows to set it via the `-d` or `-v`
parameter of kas-container.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
kas itself has a `-d` parameter that allows to print debug information,
while kas-container has the `-v` parameter. Since the `-d` parameter of
kas was first, rename the `-v` parameter of kas-container to fit kas.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This change moves all plugin handling code into the kas.plugins module.
New accessor functions `plugins.get(name)` and `plugins.all()` are
provided to wrap the plugins dictionary so that the kas main function
doesn't need to worry about how this is accessed. Plugins are loaded at
runtime rather than at parse time by calling `plugins.load()` which
gives us an improved ability to handle errors.
The `@kasplugin` decorator is removed as it modified and attribute on
the kasplugin function itself when a plugin module was loaded. Importing
a module should not result in changes to a variable in a different
module as it leads to an initialization code flow which is difficult to
reason about. Instead, plugin modules should now list the plugins which
they introduce in a `__KAS_PLUGINS__` list which will be walked at
runtime by `plugins.load()`.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This support for external plugins is undocumented and results in
differences between how internal and external plugins are loaded. It
will shortly be replaced by more integrated support for dynamically
finding and loading plugins.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This change groups the plugin modules together under kas.plugins and
moves the @kasplugin decorator to this submodule. This will ensure the
codebase stays organised as more plugins are added.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
By storing plugins in a dictionary indexed by plugin name rather than a
list, we can simplify plugin lookup and remove the undocumented need for
a plugin's run() method to return True when it has matched the given
command.
The command will be rejected by the argument parser if it does not match
one of the plugin names so we do not need to handle failure to lookup
the plugin in the dictionary.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The current default causes spurious pulls even when the image is already
locally available, at least with podman 2.0.x. Looking through podman
how-tos, none mentions that this prefix is required, and the
image_default_transport in containers.con is generally "docker://". So
it's safe to drop it for better default behavior.
If a custom local setup deviate, users can still override
KAS_CONTAINER_IMAGE_PATH, prepending what is desired.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
No reason to run this with "fail=False" because we won't ignore errors
anyway. The current setting rather leads to the stderr output being
swallowed.
Reported-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
