Commit Graph

666 Commits

Author SHA1 Message Date
Daniel Wagner
41dc0bef59 Release 0.16.1 2018-01-10 13:06:15 +01:00
Daniel Wagner
afbcd0018b scripts: Add release script
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-10 13:04:18 +01:00
Daniel Wagner
c6ce77a4c9 Release 0.16.0 2018-01-10 11:29:20 +01:00
Daniel Wagner
03caf33162 shell: Add missing imports for CleanupSSHAgent and SetupSSHAgent
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-09 14:26:47 +01:00
Henning Schild
db25611977 docs: update docs accordning to previous changes
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
ec69bcb0c2 docker: add mercurial to our toolbox
With kas now supporting mercuial as well, add that tool to the docker
containers.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
013722fd28 repos: add support for Mercurial Repos
This commit adds support for cloning mercurial repos using the type
string 'hg'.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
c47b511d34 repos: factor git-cmds out of a common Repo implementation
This creates an interface that could also be used to support other
version control systems.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
6a85296836 repos: move more of the complexity from config to Repo.factory
This commit just moved some code around, to a place where it seems to
fit better.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
0c7db599fd repos: introduce a new repo conf parameter "type"
This will be used to carry the type of repo we are talking about. At the
moment it defaults to "git".

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
9da82dba13 config: call Repo.factory only in one place
This patch simplifies the code a bit and does not change any semantics.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
3594ad50a0 repos: introduce a factory to abstract away from Repo implementation
The user just calls that factory and gets something that is a Repo, in
fact just a GitRepo for now.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
b5b2766b24 repos: also move version control call out of config.py
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
941fa38a2d repos: move repository functions into repos.py
This commit just moves core around and does not contain any logical
changes.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Henning Schild
dc8d15c3be git: rename "git_operation_disabled" to "operations_disabled"
Prepare for other version control systems and remove "git" from the
names.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Daniel Wagner
9210fe57b4 shell: Defer SetupEnviron step
If we start on a green field (only kas is available) we can't setup
the environment directly. So in this case the order is SetupProxy,
ReposFetch, ReposCheckout and then SetupEnvironment. Only if the
--keep-config-unchanged flag is provided directly go from SetupProxy
to SetupEnvironment.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-08 16:55:44 +01:00
Daniel Wagner
415e0bf114 shell: Honor SSH_PRIVATE_KEY environment variable
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-08 16:55:44 +01:00
Daniel Wagner
bef5aa80b3 docs: Install kas as user
pip warns us with:

"Running pip install with root privileges is generally not a good
idea. Try `pip3 install --user` instead."

So update the documentation to install kas as user.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-08 16:55:44 +01:00
Jan Kiszka
b42aafb97d shell: Drop default from --target
Forgotten by 49cb9b7a7ba9: The default is now set in
get_bitbake_targets.

This fixes spurious overwrites of the target defined in the config file.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-01-02 14:30:29 +01:00
Daniel Wagner
8a87f09c70 repos: Raise exception if attribute is unknown in __getattr__
pylint reports

R: 52, 4: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)

According the Python documentation we should raise AttributeError
if the key is unknown.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-02 14:26:46 +01:00
Daniel Wagner
90315d47bf includehandler: Fix string formating issue
pylint is reporting

W: 76, 8: Exception arguments suggest string formatting might be intended (raising-format-tuple)

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-01-02 14:26:46 +01:00
Henning Schild
ce02354627 git: allow an empty refspec even when using version control
If no refspec was given just take the default we get after clone.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 14:26:36 +01:00
Henning Schild
62cf4fb0eb libkas: include another space when extending BB_ENV_EXTRAWHITE
the way the env variable got extended broke the last entry in it and the
first entry in the added string because the two strings where not space
separated

i.e.
  "... no_proxySSTATE_DIR ..."
now you shoud get
  "... no_proxy SSTATE_DIR ..."

Seems that problem was in kas since day one.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Henning Schild
4140961b11 libcmds: create directory before open in case it is not there
This could happen if you use kas to checkout repos that do not contain
the directory. i.e. repos that are not actually bb layers

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Henning Schild
19e518cd25 config: support "gentoo" as a known distro
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-02 12:59:30 +01:00
Daniel Wagner
3c960140fd Release 0.15.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-12-06 16:47:28 +01:00
Claudius Heine
95015045db implementation of environment variable passthrough
With this its possible to create custom environment variables within the kas
configuration file, that are passed through to the build environment:

    env:
      VARIABLE_NAME: "default variable value"

Those variables can be overwritten by variables of the environment kas is
started in:

    $ VARIABLE_NAME="overwritten variable value" kas shell ...

Those variables are also added to the 'BB_ENV_EXTRAWHITE' variable list to make
them available to bitbake.

This is useful for instance storing sensitive information, that should not be
part of the configuration files or bitbake recipes.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-24 10:19:12 +01:00
Jan Kiszka
cd076101f4 docker: Avoid error messages on container restarts
When the user restarts an already existing kas container, errors are
thrown because of existing folders or users. One example is the usage
of gitlab-ci runners.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-11-22 12:38:07 +01:00
Daniel Wagner
7aff26d105 libkas: Add /sbin and /usr/sbin to PATH
/sbin and /usr/sbin might contain tools Yocto depens on, e.g. 'ip'.

"""
The following required tools (as specified by HOSTTOOLS) appear to be
unavailable in PATH, please install them in order to proceed:

	ip
"""

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-11-20 14:12:28 +01:00
Claudius Heine
5fac90611f setup: added python 3.6 line
Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-10 09:21:42 +01:00
Claudius Heine
6d6e7ff2e7 setup: added python version requirement
Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-10 09:21:37 +01:00
Cedric Hombourger
06b05a35b6 config: support major distro variants
Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
2017-11-09 09:26:43 +01:00
Claudius Heine
8119dff5f0 scripts: pep8 is renamed to pycodestyle
Fix it to use the new name.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-08 09:46:00 +01:00
Claudius Heine
fa1575790f multiconfig: implement initial support
This patch adds multiconfig support to kas by defining the
"BBMULTICONFIG" variable in the "local.conf" file.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-11-08 09:42:56 +01:00
[ext] Henning Schild
9370bd6267 docker: isar: configure sudo to keep proxy env variables
Keep proxy related env variables when running commands with sudo.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2017-11-02 16:16:34 +01:00
Daniel Wagner
f24b341745 Release 0.14.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-11-02 16:02:33 +01:00
Claudius Heine
aa79374f7e config: debug message for list of missing repo names
Its useful to know the missing repos that are required to
create the full configuration as they are detected.

This might help to find problems within the configuration and kas code
in the future.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-19 17:15:26 +02:00
Claudius Heine
3eff938a54 kas: added version output when starting kas
The used version of kas is useful to have in resulting build logs.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-19 17:15:26 +02:00
Claudius Heine
5d7fae7094 configschema: Added copyright and license variables
Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 13:49:09 +02:00
Claudius Heine
41fc9c8261 __version__: Added note about updating docs/format-changelog.rst
Since its easy to forget to update the documentation when changing the
configuration file format version, this patch adds a small note to
remind the developer about it.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 13:49:06 +02:00
Claudius Heine
4940d4d580 includehandler: remove possible duplicates from the missing_repos list
I was observed that in some more evolved configurations the build
stopped because kas wanted to clone a git repository multiple times
concurrently. This failed.

This patch should prevent these issues by removing all possible
duplicates from the missing_repos list prior to returning it to the
configuration.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-18 10:04:06 +02:00
Claudius Heine
9753b561d2 docs/format-changelog.rst: Updated to include the recent changes
This patch updated the 'docs/format-changelog.rst' file to include
information about the configuration file version 3 and 4.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-16 11:05:16 +02:00
Claudius Heine
c1bfeefa04 docs/devguide: Added the 'no_proxy' parameter to the description
In environments where proxies are used, there are usually some addresses
that should be reached without proxy. So it makes sense to document
that.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-16 11:03:20 +02:00
Claudius Heine
845d212f02 Added 'ftp_proxy' environment variable support
Some bitbake recipes use FTP to fetch objects. This patch adds support
for the ftp_proxy variable to kas. Is now usable in the same manner as
the 'http_proxy' variable

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-16 11:03:07 +02:00
Claudius Heine
f140d7905c config: includehandler: remove other checks that are no longer needed
Using the jsonschema module makes some checks in the includehandler
redundant and can be removed. This patch does this.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:19 +02:00
Claudius Heine
75fa095aa5 config: Added schema validation for configuration files
Currently many of the error messages regarding the configuration file
are not really helpful.

To improve reporting this patch adds validation of the configuration
files using the jsonschema module.

Example:

  Forgetting ':' at the end of a layer definition, like this:

    repos:
      meta-iot2000:
        layers:
          meta-iot2000-example
                              ^

  Exception before:

    TypeError: string indices must be integers

  Exception after:

    Validation Error:
     'meta-iot2000-example' is not of type 'object'

     Failed validating 'type' in schema['properties']['repos']['additionalProperties']['properties']['layers']:
         {'additionalProperties': {'oneOf': [{'type': 'null'},
                                             {'type': 'integer'},
                                             {'type': 'boolean'},
                                             {'type': 'string'}]},
          'type': 'object'}

     On instance['repos']['meta-iot2000']['layers']:
         'meta-iot2000-example'

This patch adds 'jsonschema' as an additional dependency.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:10 +02:00
Claudius Heine
ab8e4cd375 config: Implemented multi-target support
With this patch support for building multiple targets in parallel with
bitbake is now supported.

Changes:

  - Its now possible to state a list of targets under the target key in
    the configuration file.

    Example:

        target:
          - product-image
          - product-update-image

  - Its now possible to define multiple targets in the kas command line.

    Example:

        $ kas build --target product-image --target product-update-image \
          kas.yml

  - Its now possible to define multiple targets via the environment:

    Example:

        $ export KAS_TARGET="product-image product-update-image"
        $ kas build kas.yml

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:11:00 +02:00
Claudius Heine
f7d2e5ae20 config: added debug output of complete configuration
It might be useful to see the merged configuration in case of debugging.
This patch adds it to the debug output log level.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-10-13 14:10:47 +02:00
Daniel Wagner
15e6fb0a81 Release 0.13.0
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-10-10 09:13:50 +02:00
Jan Kiszka
4f76e47090 config: Belatedly increment file version to account for task field
Captain Hindsight: We should have never committed 13a87cb8c7 ("Allow
to define task in config and environment") without forwarding the file
version. Without this, no config file writer can tell kas versions
apart that support task control from those that don't.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-10-10 09:12:20 +02:00