Commit Graph

189 Commits

Author SHA1 Message Date
Jan Kiszka
c7cc57c42f build: Run bitbake in caller shell if available
When kas is invoked in a shell, pass this property to bitbake, enabling
its fancy UI mode.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-09-03 10:34:44 +02:00
Jan Kiszka
bb103365d2 libkas: Remove unused shell mode from run_cmd
No users, and it even seems broken /wrt universal_newlines.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-09-03 10:18:14 +02:00
Jan Kiszka
d4a615bb0b Command line based configuration file merge
This extends the command line syntax for specifying configuration files.
You can now combine files by concatenating them, separated by colons:

kas build base.yml:board.yml:feature.yml

The motivation for this feature is to avoid having to write tons of
configuration files that perform this combinations statically via
includes.

In order to avoid complications and prevent that users shoot themselves
too easily into their feet, we deny the case of distributing the
configuration files over multiple repositories. Either all files
specified on the command line come from the same repo, or they are all
local (without versioning control).

Based on idea by Claudius Heine.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:54:32 +02:00
Jan Kiszka
4a1fba912e repos: Add fallback parameter to get_root_path()
This allows the caller to differentiate between "no repo found" and
"result points to repo". Default behavior remain unchanged (return
provided path itself if no repo is found).

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:54:25 +02:00
Jan Kiszka
b349f86457 repos: Remove ctx parameter from repo functions and their callers
We can use the global context instead.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:54:18 +02:00
Jan Kiszka
7937c1963e libkas: Remove parameters from get_build_environ()
All of them can be retrieved from the global context.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:54:02 +02:00
Jan Kiszka
448107781e libkas: Remove config/context parameters from ssh functions
We can obtain the global context now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:53:48 +02:00
Jan Kiszka
52fdc4f4ae config: Remove context field
No longer used.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:53:18 +02:00
Jan Kiszka
4acde53b94 Repo: Rework factory() interface
Rather than handing down a complete config, only pass what is really
needed: a repository fallback path. This can already be determined by
the caller.

Inside Repo.factory(), we can retrieve the global context now and can
stop relying on a reference stored in the config.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:53:08 +02:00
Jan Kiszka
0c66b24a12 repos: Remove environ parameter from get_root_path()
This is just the default, namely the environment from the global
context.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:52:07 +02:00
Jan Kiszka
86dc584284 run_cmd: Default to environment from global context
If env is not provided, use the global context environment. Then drop
all every passing of that parameter.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:51:34 +02:00
Jan Kiszka
f5cea27269 Move Config creation out of Context constructor
This helps to avoid circular dependencies. It also allows the Config
constructor to make use of the global context.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:50:51 +02:00
Jan Kiszka
594fa97692 context: Provide global instance
This will avoid the increasingly ugly pushing around of a context
instance as parameter to various methods/functions. Everyone can now
simply call get_context() after build or shell called
create_global_context().

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:49:42 +02:00
Jan Kiszka
3fccd1ad3e Move get_distro_id_base from config to context
This is where it is used, only.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:48:59 +02:00
Jan Kiszka
b10b59e330 context: Stop keeping config_filename, bitbake_target/task
No one uses those properties, and we have them in the config anyway.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:48:40 +02:00
Jan Kiszka
efbac41590 shell: Unify import pattern
We do not spell out kas. elsewhere, only here so far. Unify that.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:48:15 +02:00
Jan Kiszka
5c28272d73 includehandler: Fold GlobalIncludes into IncludeHandler class
No user for this abstraction in sight.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:48:08 +02:00
Jan Kiszka
76e7339fdb repos: Fix false positives when detecting dirty repos
This fixes a regression of 5b85fba0: diff-index is porcelain and seems
to lack some explicit refresh prior to being called. Therefore, it can
incorrectly detect files as being modified although a "git diff" will
not - and will actually also update the status so that the next
diff-index would be fine again.

Fix those false positives while still detecting also cached changes by
using git status -s instead.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:45:15 +02:00
Jan Kiszka
3744ff329a config: Account for multiple ID_LIKE words in os-release
The ID_LIKE tag that get_distro_id_base also queries may contain
multiple, space-separated values. Opensuse Leap 15.0 is exploiting that,
e.g. ("suse opensuse").

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-23 16:53:25 +02:00
Andreas Reichel
7b18e5ec3b Refactor repo checkout
If a kas-file inside a repo includes another kas-file form a repo
which is not checked-out, and this 2nd repo can only be checked-out
with an SSH-key, kas fails. This is, because the constructor of
the Config class used to fetch missing repos already before the
SSH-Agent was setup.

This patch refactors the way in which kas checks-out repositories. This
also required the Config class to be split into Context and Config,
where the new Context is the build-Context, i.e. containing the
environment of commands executed by kas and the new Config is the Config
of kas itself, i.e. containing the repo dictionary.

This way it is possible to initialize the context needed for SSH setup
independently of the kas configuration.

The commands ReposFetch and ReposCheckout are refactored into a
SetupRepos command. This command parses the include files and
successively checks-out needed repos and updates the config as long
as repos are missing. The logic is taken directly from the constructor
of the former Config class (with minor adaptations).

Further refactoring will be needed to clean up the conceptual
programming patterns (i.e. to keep the Macro-Style consistent),
and to re-add the macro pattern regarding the above ReposFetch.

Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
2018-08-23 16:29:56 +02:00
Daniel Wagner
06f0c20f21 libkas: Disable not-an-itable false positive
pylint has a bug:

https://github.com/PyCQA/pylint/issues/996

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:27:21 +02:00
Daniel Wagner
d6110a175c repos: Fix unnecessary "else" after "return"
pylint reports:

kas/repos.py:53:8: R1705: Unnecessary "else" after "return" (no-else-return)
kas/repos.py:264:12: R1705: Unnecessary "else" after "return" (no-else-return)

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:27:15 +02:00
Daniel Wagner
bc7aa5c694 includehandler: Silence too-many-arguments pyling warning
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:26:47 +02:00
Daniel Wagner
1b804f937a includehandler: Remove useless object inheritance
pylint reports:

kas/includehandler.py:105:0: R0205: Class 'IncludeHandler' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:26:47 +02:00
Daniel Wagner
de243b28b8 Release 0.18.0 2018-05-17 13:12:10 +02:00
Daniel Wagner
c1e5c4f379 build: Fix all style reports by pycodestyle v2.4.0
pycodestyle v2.4.0 checks for W606 (async deprecated), but
we can't disable it on source code level with '# noqa'. Let's disable
the check globaly. We will still catch it with pylint.

By disabling the W606 we need also to list the W503 which was
previously active.

While at it fix following things:

./docs/conf.py:252:6: E121 continuation line under-indented for hanging indent
./docs/conf.py:257:6: E126 continuation line over-indented for hanging indent
./kas/build.py:123:17: W504 line break after binary operator
./kas/configschema.py:48:25: E123 closing bracket does not match indentation of opening bracket's line
./kas/configschema.py:49:21: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:82:13: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:121:13: E123 closing bracket does not match indentation of opening bracket's line

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-05-17 13:00:26 +02:00
Claudius Heine
6afd92eabc kas/config: Make 'get_repo_dict' private
Since 'get_repo_dict' does much more than just querying available
repositories, this patch makes it private.  The same data is now
available via the 'repo_dict' property. This also means that
'get_repos' is now faster, because it accesses cached data, instead of
calling 'get_repo_dict' that generates this data.

This patch also allows 'get_repos' and 'repo_dict' to be used even in the
'Repo' class. 'get_repo_dict' creates new 'Repo' instances and thus
invalidates the current instance.

This patch is in preparation of the 'patch support'-feature, that
depends on this patch.

Signed-off-by: Claudius Heine <ch@denx.de>
2018-04-04 13:43:37 +02:00
Abdur Rehman
5b85fba0df repos: use git diff-index to check if repo is dirty
is_dirty_cmd() uses `git diff --shortstat` to see if the repo can be
safely checked out. This only checks for the modified files and does
not take the cached(staged) files into account.

There can be cases where the cached changes could be overwritten by a
checkout. In this case, the git checkout command aborts with an error
message telling the user to commit or stash their changes.

Use `git diff-index HEAD --shortstat` to determine if there are any
changes in the tracked files, whether cached or not.

https://git-scm.com/docs/git-diff-index

Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
2018-04-03 11:39:37 +02:00
Daniel Wagner
4be43c7bf6 Revert "kas/config: Make 'get_repo_dict' private"
This reverts commit b9032ec025.

The commit breaks inclusion chains of kas files.  In my example a
first kas-file includes a second one. The second one is supposed to
add repos but kas does not fetch those.

Reported-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-03-26 14:49:06 +02:00
Claudius Heine
40e528065f kas/libcmd: Instead of copying the .gitconfig, create a new dummy one
Copying the .gitconfig failes if the current user doesn't have one,
since the commits that are done within kas, will not be pushed or
merged anywhere, we can just create a minimal .gitconfig in the temp
home directory of kas.

Signed-off-by: Claudius Heine <ch@denx.de>
2018-03-21 17:16:04 +01:00
Claudius Heine
e8851a5fb3 implements patch support for repos
With this commit its now possible to patch 3rd party repos before bitbake is started.

Example:

This is our repo:
  .
  ├── kas.yml
  ├── first-patch.patch
  └── quilt-patches
      ├── second-patch.patch
      ├── third-patch.patch
      └── series

Content of kas.yml:
  header:
    version: 8

  repos:
    my:
    third-party:
      url: "git://example.com/third-party.git"
      refspec: "35adf4...34"
      patches:
        01-first:
          repo: my
          path: "first-patch.patch"
        02-second:
          repo: my
          path: "quilt-patches"

Currently only 'git' repositories can be patched.

Signed-off-by: Claudius Heine <ch@denx.de>
2018-03-09 09:59:07 +01:00
Claudius Heine
b9032ec025 kas/config: Make 'get_repo_dict' private
Since 'get_repo_dict' does much more than just querying available
repositories, this patch makes it private.  The same data is now
available via the 'repo_dict' property. This also means that
'get_repos' is now faster, because it accesses cached data, instead of
calling 'get_repo_dict' that generates this data.

Signed-off-by: Claudius Heine <ch@denx.de>
2018-03-09 09:55:43 +01:00
Daniel Wagner
1e73112a9a Release 0.17.0 2018-02-08 15:49:56 +01:00
Jan Kiszka
cda778acb5 libcmds: Write MACHINE and DISTRO as weak defaults
Otherwise we cannot overwrite them in mulitconfig files. It's not
expected that anything else in the layer config files sets those
variable, thus shadows our setting in local.conf.

Something changed in latest bitbake to reveal this issue. We first see
it in Isar since bitbake moved to f8126aaf7741.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-02-05 21:07:28 +01:00
Jan Kiszka
ca6b3c05c9 libkas: Fail run_cmd silently if command is not found
If we only try-execute a command (fail=False), do not bail out with an
exception when starting that command already fails. Catch typical
exceptions and return the corresponding error code along with the
exception string.

This fixes get_root_path in case hg is not installed.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-01-25 19:11:43 +01:00
Georg Lutz
7fa75d94c0 repos: Fix relative path for repos
If a repo path is relative, append the kas work dir to it.

Background: The path ends up in bblayers.conf

Without this change the following example will fail:

--------------------------------------------------------
header:
     version: 4
machine: qemux86
repos:
     poky:
         url: "https://git.yoctoproject.org/git/poky"
         refspec: "rocko"
         path: "thirdparty/poky"
         layers:
             meta:
             meta-poky:
--------------------------------------------------------

Signed-off-by: Georg Lutz <georg@georglutz.de>
2018-01-25 18:54:55 +01:00
Daniel Wagner
41dc0bef59 Release 0.16.1 2018-01-10 13:06:15 +01:00
Daniel Wagner
c6ce77a4c9 Release 0.16.0 2018-01-10 11:29:20 +01:00
Daniel Wagner
03caf33162 shell: Add missing imports for CleanupSSHAgent and SetupSSHAgent
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-09 14:26:47 +01:00
Henning Schild
013722fd28 repos: add support for Mercurial Repos
This commit adds support for cloning mercurial repos using the type
string 'hg'.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
c47b511d34 repos: factor git-cmds out of a common Repo implementation
This creates an interface that could also be used to support other
version control systems.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
6a85296836 repos: move more of the complexity from config to Repo.factory
This commit just moved some code around, to a place where it seems to
fit better.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
0c7db599fd repos: introduce a new repo conf parameter "type"
This will be used to carry the type of repo we are talking about. At the
moment it defaults to "git".

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
9da82dba13 config: call Repo.factory only in one place
This patch simplifies the code a bit and does not change any semantics.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
3594ad50a0 repos: introduce a factory to abstract away from Repo implementation
The user just calls that factory and gets something that is a Repo, in
fact just a GitRepo for now.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
b5b2766b24 repos: also move version control call out of config.py
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
941fa38a2d repos: move repository functions into repos.py
This commit just moves core around and does not contain any logical
changes.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
dc8d15c3be git: rename "git_operation_disabled" to "operations_disabled"
Prepare for other version control systems and remove "git" from the
names.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Daniel Wagner
9210fe57b4 shell: Defer SetupEnviron step
If we start on a green field (only kas is available) we can't setup
the environment directly. So in this case the order is SetupProxy,
ReposFetch, ReposCheckout and then SetupEnvironment. Only if the
--keep-config-unchanged flag is provided directly go from SetupProxy
to SetupEnvironment.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-08 16:55:44 +01:00
Daniel Wagner
415e0bf114 shell: Honor SSH_PRIVATE_KEY environment variable
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-08 16:55:44 +01:00
Jan Kiszka
b42aafb97d shell: Drop default from --target
Forgotten by 49cb9b7a7b: The default is now set in
get_bitbake_targets.

This fixes spurious overwrites of the target defined in the config file.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-01-02 14:30:29 +01:00
Daniel Wagner
8a87f09c70 repos: Raise exception if attribute is unknown in __getattr__
pylint reports

R: 52, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)

According the Python documentation we should raise AttributeError
if the key is unknown.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-02 14:26:46 +01:00
Daniel Wagner
90315d47bf includehandler: Fix string formating issue
pylint is reporting

W: 76, 8: Exception arguments suggest string formatting might be intended (raising-format-tuple)

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-02 14:26:46 +01:00
Henning Schild
ce02354627 git: allow an empty refspec even when using version control
If no refspec was given just take the default we get after clone.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 14:26:36 +01:00
Henning Schild
62cf4fb0eb libkas: include another space when extending BB_ENV_EXTRAWHITE
the way the env variable got extended broke the last entry in it and the
first entry in the added string because the two strings where not space
separated

i.e.
  "... no_proxySSTATE_DIR ..."
now you shoud get
  "... no_proxy SSTATE_DIR ..."

Seems that problem was in kas since day one.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Henning Schild
4140961b11 libcmds: create directory before open in case it is not there
This could happen if you use kas to checkout repos that do not contain
the directory. i.e. repos that are not actually bb layers

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Henning Schild
19e518cd25 config: support "gentoo" as a known distro
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Daniel Wagner
3c960140fd Release 0.15.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-12-06 16:47:28 +01:00
Claudius Heine
95015045db implementation of environment variable passthrough
With this its possible to create custom environment variables within the kas
configuration file, that are passed through to the build environment:

    env:
      VARIABLE_NAME: "default variable value"

Those variables can be overwritten by variables of the environment kas is
started in:

    $ VARIABLE_NAME="overwritten variable value" kas shell ...

Those variables are also added to the 'BB_ENV_EXTRAWHITE' variable list to make
them available to bitbake.

This is useful for instance storing sensitive information, that should not be
part of the configuration files or bitbake recipes.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-24 10:19:12 +01:00
Daniel Wagner
7aff26d105 libkas: Add /sbin and /usr/sbin to PATH
/sbin and /usr/sbin might contain tools Yocto depens on, e.g. 'ip'.

"""
The following required tools (as specified by HOSTTOOLS) appear to be
unavailable in PATH, please install them in order to proceed:

	ip
"""

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-11-20 14:12:28 +01:00
Cedric Hombourger
06b05a35b6 config: support major distro variants
Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
2017-11-09 09:26:43 +01:00
Claudius Heine
fa1575790f multiconfig: implement initial support
This patch adds multiconfig support to kas by defining the
"BBMULTICONFIG" variable in the "local.conf" file.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-08 09:42:56 +01:00
Daniel Wagner
f24b341745 Release 0.14.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-11-02 16:02:33 +01:00
Claudius Heine
aa79374f7e config: debug message for list of missing repo names
Its useful to know the missing repos that are required to
create the full configuration as they are detected.

This might help to find problems within the configuration and kas code
in the future.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-19 17:15:26 +02:00
Claudius Heine
3eff938a54 kas: added version output when starting kas
The used version of kas is useful to have in resulting build logs.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-19 17:15:26 +02:00
Claudius Heine
5d7fae7094 configschema: Added copyright and license variables
Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 13:49:09 +02:00
Claudius Heine
41fc9c8261 __version__: Added note about updating docs/format-changelog.rst
Since its easy to forget to update the documentation when changing the
configuration file format version, this patch adds a small note to
remind the developer about it.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 13:49:06 +02:00
Claudius Heine
4940d4d580 includehandler: remove possible duplicates from the missing_repos list
I was observed that in some more evolved configurations the build
stopped because kas wanted to clone a git repository multiple times
concurrently. This failed.

This patch should prevent these issues by removing all possible
duplicates from the missing_repos list prior to returning it to the
configuration.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 10:04:06 +02:00
Claudius Heine
845d212f02 Added 'ftp_proxy' environment variable support
Some bitbake recipes use FTP to fetch objects. This patch adds support
for the ftp_proxy variable to kas. Is now usable in the same manner as
the 'http_proxy' variable

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-16 11:03:07 +02:00
Claudius Heine
f140d7905c config: includehandler: remove other checks that are no longer needed
Using the jsonschema module makes some checks in the includehandler
redundant and can be removed. This patch does this.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:19 +02:00
Claudius Heine
75fa095aa5 config: Added schema validation for configuration files
Currently many of the error messages regarding the configuration file
are not really helpful.

To improve reporting this patch adds validation of the configuration
files using the jsonschema module.

Example:

  Forgetting ':' at the end of a layer definition, like this:

    repos:
      meta-iot2000:
        layers:
          meta-iot2000-example
                              ^

  Exception before:

    TypeError: string indices must be integers

  Exception after:

    Validation Error:
     'meta-iot2000-example' is not of type 'object'

     Failed validating 'type' in schema['properties']['repos']['additionalProperties']['properties']['layers']:
         {'additionalProperties': {'oneOf': [{'type': 'null'},
                                             {'type': 'integer'},
                                             {'type': 'boolean'},
                                             {'type': 'string'}]},
          'type': 'object'}

     On instance['repos']['meta-iot2000']['layers']:
         'meta-iot2000-example'

This patch adds 'jsonschema' as an additional dependency.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:10 +02:00
Claudius Heine
ab8e4cd375 config: Implemented multi-target support
With this patch support for building multiple targets in parallel with
bitbake is now supported.

Changes:

  - Its now possible to state a list of targets under the target key in
    the configuration file.

    Example:

        target:
          - product-image
          - product-update-image

  - Its now possible to define multiple targets in the kas command line.

    Example:

        $ kas build --target product-image --target product-update-image \
          kas.yml

  - Its now possible to define multiple targets via the environment:

    Example:

        $ export KAS_TARGET="product-image product-update-image"
        $ kas build kas.yml

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:00 +02:00
Claudius Heine
f7d2e5ae20 config: added debug output of complete configuration
It might be useful to see the merged configuration in case of debugging.
This patch adds it to the debug output log level.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:10:47 +02:00
Daniel Wagner
15e6fb0a81 Release 0.13.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-10-10 09:13:50 +02:00
Jan Kiszka
4f76e47090 config: Belatedly increment file version to account for task field
Captain Hindsight: We should have never committed 13a87cb8c7 ("Allow
to define task in config and environment") without forwarding the file
version. Without this, no config file writer can tell kas versions
apart that support task control from those that don't.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-10-10 09:12:20 +02:00
Daniel Wagner
d1968a7f84 Release 0.12.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-10-09 17:53:40 +02:00
Daniel Wagner
133f1be38a libcmds: Remove unused pre and post hooks
9e136e532d ("config: Remove dynamic configuration variant")
removed the posibility to add pre and post hooks. Remove the remaining
bits from libcmds.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-09-27 10:49:35 +02:00
Jan Kiszka
bfd7770b81 config: Fold ConfigStatic into Config
Now that we no longer have two different configuration variants, we can
remove the abstract class and make Config the primary interface.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-09-27 10:48:59 +02:00
Jan Kiszka
9e136e532d config: Remove dynamic configuration variant
We no longer see an urging use case for it. All field scenarios can
perfectly be modeled with the static format.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-09-27 10:48:35 +02:00
Jan Kiszka
13a87cb8c7 Allow to define task in config and environment
Based on original patch by Sascha Weisenberger.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-09-25 09:33:43 +02:00
Henning Schild
d06bdc4781 libkas: do not use absolute paths when executing programs
Using absolute paths breaks on systems that place the binaries in
another directory and it defeats local overlays with PATH.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2017-09-11 10:19:09 +02:00
Jan Kiszka
affd1184e7 entry: Consistently call python3
This may not be needed in practice, but it is cleaner to align all
shebangs.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-08-31 22:31:41 +02:00
Claudius Heine
8745aabb62 kas/shell: add 'keep-config-unchanged' switch
This patch adds a command line argument to the shell plugin, that allows to
enter the shell without changing the bitbake configuration.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-08-04 09:53:50 +02:00
Claudius Heine
11f047a408 kas/shell: Prepare complete bitbake configuration before entering shell
Before this patch, starting the shell in a clean environment, does not
checkout all repository or configures all layers correctly.

This patch adds the missing checkout and configuration steps to the
shell plugin.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-08-01 09:35:34 +02:00
Jan Kiszka
2d6b1ce6a0 libkas: Detect and report multiple init scripts
Having more than one them is a sign of serious problems in the
configuration. Better catch that early and fail loudly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-28 17:29:54 +02:00
Jan Kiszka
eb284afb8e libkas: Detect error of init script
Do not swallow errors of the init script. Rather leave our wrapper
immediately and let run_cmd report the splash and terminate kas.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-28 17:11:54 +02:00
Daniel Wagner
39dba6fe75 Release 0.11.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-07-24 17:17:36 +02:00
Jan Kiszka
bb853cec02 config: Write proper info message about root of URL-less repos
Instead of flooding the log with misleading error messages, always print
a note which path is used as root for repositories that have no URLs.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-21 15:34:20 +02:00
Claudius Heine
96034e2d9a config: Allow in-tree repos not to be in a git repo.
If the repo, where the configuration file resides in (in-tree
configuration) is not a git repository, take the directory where the
configuration file is located at as the path of this repo instead of failing.

This patch does not need to raise the configuration file version,
because it does not depend on the configuration file.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-20 16:08:34 +02:00
Jan Kiszka
100c7e047f libkas: Pass git proxy related environment variables
This allows to define GIT_PROXY_COMMAND="oe-git-proxy" outside of kas
and pass this to the build. Now we can pull from native git repos when
there is only a http proxy available.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 16:08:34 +02:00
Jan Kiszka
c9326cc1ed Write deterministic local.conf and bblayers.conf
Sort the header sections by their keys and the layers by their names.
This will re-enable writing deterministic config files, and bitbake can
use its cache again, accelerating rebuild startup times a lot.

While at it, indent the layers in bblayers.conf nicely.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 15:32:06 +02:00
Jan Kiszka
ef0ef7c60c kas: Also print config format version on --version
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 15:31:34 +02:00
Jan Kiszka
2a618686f1 Switch to separate config file version
Use a separately incremented integer to track the configuration file
format version. We start with 2 due to the change that 23c3a951f6
introduced. 1 is declared to be equivalent to the original '0.10'.

The separate versioning has the advantage of being able to increment it
already during the development cycle, and using it with config files
that test/exploit the new format.

Using an integer has the advantages of a) differentiating it clearly
from the now independent kas version and b) simplifying the version
parsing. We can now also remove the string type restriction.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 15:31:23 +02:00
Claudius Heine
23c3a951f6 config: ConfigStatic should only try to fetch missing known repos
When a configuration file depends on a include from a repo, which is
defined in another included file, a 'KeyError' occures.

This fixes this, so that only known repos are tried to be fetched.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-12 10:22:15 +02:00
Jan Kiszka
f3e8d9ffd8 includehandler: Format LoadConfigException message nicely
This exception is always thrown with both a message and a file name.
Combine them to a single string, instead of throwing tuples.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-12 10:22:15 +02:00
Daniel Wagner
6bc7715f15 Release 0.10.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-07-10 12:27:26 +02:00
Jan Kiszka
49cb9b7a7b Fix overwriting of target via command line with static configs
ConfigStatic does not respect the --target overwrite so far. Fix this
by pulling the default definition into ConfigPython (ConfigStatic
already does that via the Config base class) and set the target provided
via the command line in the static config after parsing all files.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-08 22:08:47 +02:00
Jan Kiszka
8e3862112c libkas: Fix false positive of pylint
As suggested by Claudius Heine, reorder the blocks so that we can add
the pylint exception to the if half.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-07 16:57:21 +02:00
Claudius Heine
f10718043e config: Removes file extension check for ConfigStatic
The `load_config` function in the includehandler already checks for the
file extension, so checking it before that, is redundant.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-07 16:57:21 +02:00
Jan Kiszka
f58d44f190 libkas: Properly format stderr dump of failing commands
Inject the linefeed that we removed in f0a18868c9 and add an
introduction line.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-07 16:57:21 +02:00