When a configuration file depends on a include from a repo, which is
defined in another included file, a 'KeyError' occures.
This fixes this, so that only known repos are tried to be fetched.
Signed-off-by: Claudius Heine <ch@denx.de>
This exception is always thrown with both a message and a file name.
Combine them to a single string, instead of throwing tuples.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This prevents overwriting next builds with master output, and vice
versa. Users can now decide if they want more or less bleeding edge.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Yeah, that was "too simple to possibly fail". Consequently, the release
revealed two issues: The script only looked for annotated tags but they
are not used so far. Moreover, the TRAVIS_BRANCH name of a tag is that
tag and neither "master" or "next".
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
ConfigStatic does not respect the --target overwrite so far. Fix this
by pulling the default definition into ConfigPython (ConfigStatic
already does that via the Config base class) and set the target provided
via the command line in the static config after parsing all files.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
There is a nasty problem with legacy aufs: wic tries to find out the
block size of the filesystem that holds the partition images, but aufs
does not seem to implement this properly, returning 0 at least on Debian
Jessie. That makes wic become upset and through a division-by-zero
exception soon after.
Catch this case by warning the user about the inappropriate docker
setup during container start.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
As suggested by Claudius Heine, reorder the blocks so that we can add
the pylint exception to the if half.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The `load_config` function in the includehandler already checks for the
file extension, so checking it before that, is redundant.
Signed-off-by: Claudius Heine <ch@denx.de>
The application should not be stopped from this code, removed `sys.exit`
and added exceptions to improve error handling.
Also added some more checks that were missing.
Signed-off-by: Claudius Heine <ch@denx.de>
In case of ConfigStatic, proxy settings only come from the host
environment. We need them early, prior to fetching the first missing
repo that is pulled via an include.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
For me 'distro.id()' returned 'ubuntu' and thats not equal to 'Ubuntu'.
This patch converts the distro id to lowercase before comparing it.
Signed-off-by: Claudius Heine <ch@denx.de>
This seems to be the cleanest and simplest way of properly terminating
kas while one or more sub-processes are running: just ignore the signal.
The sub-processes will get it as well, and they will terminate
themselves properly. kas will get an error then and shut down. Problem
solved.
Forwarding the signal didn't work reliably anyway because we couldn't
count on kas being a process group leader in all cases.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This behavior got lost through 0bdd7a8d52. Specifically, errors during
git clone no longer shut down kas. Fix this by reporting back an error
code from _repo_fetch_async and terminate in repos_fetch if it's
non-zero.
Drop the return value of repos_fetch - no caller expects any.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make the header sections in local.conf and bblayers.conf easier
identifiable by naming them and writing them in chunks, separated by
blank lines. This can help with debugging the resulting configuration.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Another, hopefully better attempt: Ignore the TERM signal in the sending
process so that we can properly clean up all the futures before actually
closing. This also avoids a "Terminated" message even on regular kas
endings.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
That only creates exceptions and complaints if more routines are running
or we are already in the exiting process. Rather leave the coroutine and
catch the error on the waiting end.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Don't add a newline to stderr dumps on failing commands, the logger does
that already. And don't print success messages if the fetch command
actually failed.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Isar requires a number of extra packages compared to a Yocto build. It
also needs a newer e2fsprogs version which can be taken from backports.
Last but not least, its keystone multistrap contains a nasty bug [1]
under in jessie so that we need to pull the updated version from
stretch.
The value of having this image still based on jessie is that it can be
used for both purposes: Yocto (2.1/2.2) and Isar builds. We push it
along the kas core image as kas-isar during the CI run.
Isar still requires some raised privileges. Therefore, the image has to
be started with the additional switches "--cap-add=SYS_ADMIN
--cap-add=MKNOD --device $(/sbin/losetup -f)".
[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=774698
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
/usr/share/locale/locale.alias is a symlink to /etc/locale.alias and
needed by locale-gen. /usr/share/locale/locale.alias gets normally
installed by locale package but is filtered in jessie-slim, so re-add
the symlink.
Signed-off-by: Mustafa Yücel <mustafa.yuecel@siemens.com>
Since recursion is recursion and we always have to go deeper, putting a
link to the documentation in the documentation is essential.
Signed-off-by: Claudius Heine <ch@denx.de>
While the guides are available and renderd on github, there are many
things, that gar generated by sphinx, missing.
This patch replaces those links to one that points to the readthedocs
site.
Signed-off-by: Claudius Heine <ch@denx.de>
- removed markdown support for documentation, because of import
difficulties.
- converted `intro.md` to restructured text `intro.rst`
- added requirements_rtd.txt that lists the required packages for rtd to
build the documentation.
Signed-off-by: Claudius Heine <ch@denx.de>
In order to include sphinx generated documentation, restructured text
has to be used. Added some sphinx generated content.
Signed-off-by: Claudius Heine <ch@denx.de>
Restructured text provide more options to extend the documentation by
including sphinx generated content.
Also added reference documentation for the static configuration format.
Signed-off-by: Claudius Heine <ch@denx.de>
The readthedocs style for sphinx has an issue where it does not auto
wrap text in table cells. This results in very long cells with a
vertical scrollbar.
This adds a css style sheet to fix this wrapping issue.
Signed-off-by: Claudius Heine <ch@denx.de>
This commit adds initial files for the Sphinx documentation system and
expanded checkcode.sh to include `doc8` as documentation checker. Also
added this checker to the dependencies for travis CI.
Signed-off-by: Claudius Heine <ch@denx.de>
The current .gitignore is very specific to the project in the current
form and developing process and not commented.
This patch uses a more generic and commented .gitignore for python base
projects that should fit all future needs.
Signed-off-by: Claudius Heine <ch@denx.de>
Documentation for the 'build' and 'shell' command line were missing.
Also fixed kas command line documentation mentioning 'ebs-yocto'.
Signed-off-by: Claudius Heine <ch@denx.de>
With Sphinx it is possible to create the command line documentation
automatically if there is a function that returns just the command line
parser.
Currently the creation of the argument parser is rather entangled with
the rest of kas. This patches seperates this.
Signed-off-by: Claudius Heine <ch@denx.de>
When updating the previously used version of pylint 1.6.5 to 1.7.2
some new issues where found.
This patch fixes these issues:
************* Module kas.config
E:402, 0: Bad option value 'redefined-variable-type' (bad-option-value)
************* Module kas.includehandler
E: 33, 0: No name 'version' in module 'distutils' (no-name-in-module)
E: 33, 0: Unable to import 'distutils.version' (import-error)
R:239,12: Unnecessary "else" after "return" (no-else-return)
************* Module kas.libkas
C:214, 7: Do not use `len(SEQUENCE)` as condition value
(len-as-condition)
************* Module kas.repos
R: 54,12: Unnecessary "else" after "return" (no-else-return)
Signed-off-by: Claudius Heine <ch@denx.de>
Changing even simple settings like target or machine requires the
creation of additional configuration files. This does not scale well.
This patch allows the target, machine and distro to be overwritten by
environment variables (called KAS_TARGET, KAS_MACHINE and KAS_DISTRO
respectively).
It also fixes how the environment variables for the proxy settings are
handled. Currently the settings in the config files overwrite the proxy
settings from the environment, but since that would be inconsistent this
patch switches that around. With this patch the environment overwrites
the proxy settings in the configuration file.
Signed-off-by: Claudius Heine <ch@denx.de>
Building on top of run_cmd_async, this reworks repo_fetch to a
repository list fetcher repos_fetch that runs those operations in
parallel.
The two users, ReposFetch and ConfigStatic, are converted to exploit
this parallelization.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Make it more straightforward by removing complete variable and
terminating the loop on the condition - no more missing repos -
directly. Rename variables to clarify what they reference.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Enhance _stream_subprocess to run_cmd_async, a co-routine variant of
run_cmd that the caller can use to parallelize command execution.
run_cmd becomes a simple wrapper that waits for the async variant to
complete.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Instead of writing "commit", issue a more telling log message if some
repo already contains the required hash.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
