Commit Graph

18 Commits

Author SHA1 Message Date
Claudius Heine
23c3a951f6 config: ConfigStatic should only try to fetch missing known repos
When a configuration file depends on a include from a repo, which is
defined in another included file, a 'KeyError' occures.

This fixes this, so that only known repos are tried to be fetched.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-12 10:22:15 +02:00
Jan Kiszka
49cb9b7a7b Fix overwriting of target via command line with static configs
ConfigStatic does not respect the --target overwrite so far. Fix this
by pulling the default definition into ConfigPython (ConfigStatic
already does that via the Config base class) and set the target provided
via the command line in the static config after parsing all files.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-08 22:08:47 +02:00
Claudius Heine
f10718043e config: Removes file extension check for ConfigStatic
The `load_config` function in the includehandler already checks for the
file extension, so checking it before that, is redundant.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-07 16:57:21 +02:00
Jan Kiszka
2be4007113 config: Initialize proxy environment prior to fetching includes
In case of ConfigStatic, proxy settings only come from the host
environment. We need them early, prior to fetching the first missing
repo that is pulled via an include.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-06 09:10:11 +02:00
Claudius Heine
6f37db72d4 Compare the distro_id in lowercase
For me 'distro.id()' returned 'ubuntu' and thats not equal to 'Ubuntu'.
This patch converts the distro id to lowercase before comparing it.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-07-06 09:10:09 +02:00
Jan Kiszka
9f4af22ac6 config: Write out config headers in sections
Make the header sections in local.conf and bblayers.conf easier
identifiable by naming them and writing them in chunks, separated by
blank lines. This can help with debugging the resulting configuration.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-06 09:05:04 +02:00
Claudius Heine
546b51f450 Fixed issues that occured with pylint 1.7.2
When updating the previously used version of pylint 1.6.5 to 1.7.2
some new issues where found.

This patch fixes these issues:

************* Module kas.config
E:402, 0: Bad option value 'redefined-variable-type' (bad-option-value)
************* Module kas.includehandler
E: 33, 0: No name 'version' in module 'distutils' (no-name-in-module)
E: 33, 0: Unable to import 'distutils.version' (import-error)
R:239,12: Unnecessary "else" after "return" (no-else-return)
************* Module kas.libkas
C:214, 7: Do not use `len(SEQUENCE)` as condition value
(len-as-condition)
************* Module kas.repos
R: 54,12: Unnecessary "else" after "return" (no-else-return)

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-28 15:10:14 +02:00
Claudius Heine
8ba1ce52e8 Allow environment variables to overwrite proxy, target, machine, distro
Changing even simple settings like target or machine requires the
creation of additional configuration files. This does not scale well.

This patch allows the target, machine and distro to be overwritten by
environment variables (called KAS_TARGET, KAS_MACHINE and KAS_DISTRO
respectively).

It also fixes how the environment variables for the proxy settings are
handled. Currently the settings in the config files overwrite the proxy
settings from the environment, but since that would be inconsistent this
patch switches that around. With this patch the environment overwrites
the proxy settings in the configuration file.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-28 15:10:08 +02:00
Jan Kiszka
0bdd7a8d52 libkas: Rework repo_fetch to parallelized repos_fetch
Building on top of run_cmd_async, this reworks repo_fetch to a
repository list fetcher repos_fetch that runs those operations in
parallel.

The two users, ReposFetch and ConfigStatic, are converted to exploit
this parallelization.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-06-28 14:51:29 +02:00
Jan Kiszka
19ee6edcb1 config: Clean up ConfigStatic constructor
Make it more straightforward by removing complete variable and
terminating the loop on the condition - no more missing repos -
directly. Rename variables to clarify what they reference.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-06-28 14:51:29 +02:00
Jan Kiszka
9a300046e6 config: Detect OpenSUSE via distro.id()
distro.id() reports OpenSUSE differently than platform.get_distro_id().
Account for this.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-06-28 11:06:35 +02:00
Claudius Heine
17691d2756 Refactored fetching and checking out of missing repos in kas.config
The current implementation did some more hackish solution to work
around some of kas infrastructure, without changing to much.

This patch cleans this up and therefor remove the more obsure part
of the include mechansim.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-22 12:33:00 +02:00
Claudius Heine
34dd07e76e added initial implementation of a include handler
Splitting configuration files into multiple files and implementing an
include mechanism allows to handle configurations more flexible and
allowing following scenarios:

 - including kas configuration file from external meta layer into
   own project
 - splitting many different similar configurations into multiple files
   with only a couple of common files

To include file its necessary to add a 'includes' entry into the kas
configuration. To include files from the same repo, do it like this:

-----
includes:
  - ../base/base_include.yml
-----

The path is relative to the current configuration file. If they start
with a path seperator ("/") they are absolute.

To include files from a different repository, do it like this:

-----
includes:
  - file: bsps/my_machine_include.yml
    repo: collected_machines

repos:
  collected_machines:
    url: https://url.to.git.repo/
    revspec: master
-----

You have to make sure that the repository definition is available
in your current file, or in any include that is available at this
point.

Yaml ("*.yml") and Json ("*.json") files are supported.

Included in this patch are also some changes to the configuration
file structure. Here is an overview:

The value of the 'repos' key is a dictionary that maps a repo identifier
to a dictionary that contains 5 entries:

- url: The url to the repository. If that is missing, no git operations
are used
- refspec: The git refspec of the repository that is used. If that is
missing the latest commit of the default branch is used.
- name: The is defines under which name the repository is stored. If
that is missing the repository identifier is used
- path: The path where the repository is stored. If no url is given and
a path is missing, the repository is referencing the repository under
which the configuration file is stored. If no url is given and a path is
specified, the repository is referencing a directory where layers might
be stored. If an url is specified, path can be used to overwrite the
default (kas_work_dir + repo.name) checkout directory.
- layers: This contains a dictionary of layers, that should be included
into the bblayers.conf. The keys are paths relative to the repository
and the values can be used to exclude layers if they are one of
"excluded", "disabled", "false", "0", "n", or "no". Also boolean values
are accepted. Any other value, including "None" means that this layer is
included into the bblayers.conf. If "layers" is missing or empty, the
repository itself is included into the bblayers. If this is specified, the
repository itself is not included into the bblayers.conf.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 17:30:51 +02:00
Claudius Heine
c950de46f9 Renamed sublayers back to layers
It was decided that 'sublayers' is to confusing, because this term is
not used in the bitbake/openembedded context.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 17:30:51 +02:00
Claudius Heine
33a21c8d0d run pylint3 and fixed report
This commit adds a pylint configuration and fixed all found issues.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 17:30:51 +02:00
Jan Kiszka
6bc8e08459 Fix pep8 reports
Zero findings after this commit. We should try to keep that level.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-06-21 17:30:51 +02:00
Jan Kiszka
c65657cac6 Remove state tracking, use KAS_WORK_DIR env variable
Config.has_changed didn't work reliably so far, so the state cache was
only allowing to associate a work directory with configuration file.
However this was rather unintuitive, specifically when moving the work
dir to a different home - nothing told the user about the persistent
association.

We therefore agreed to remove the state cache for now. It can be
reintroduced at any time once a consistent usage model exists.

To establish a relationship to a work directory inside a shell session,
kas now evaluates the KAS_WORK_DIR environment variable.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-06-19 12:54:24 +02:00
Daniel Wagner
daf0abab5e Initial public release
This is the first public release of kas.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2017-06-14 15:21:19 +02:00