PTU5KAS/kas/includehandler.py

291 lines
11 KiB
Python
Raw Normal View History

added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
# kas - setup tool for bitbake based projects
#
# Copyright (c) Siemens AG, 2017-2021
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
"""
This module implements how includes of configuration files are handled in
kas.
"""
import os
from pathlib import Path
from collections import OrderedDict
from collections.abc import Mapping
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
import functools
import logging
from jsonschema.validators import Draft4Validator
from . import __file_version__, __compatible_file_version__
from . import CONFIGSCHEMA
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
__license__ = 'MIT'
__copyright__ = 'Copyright (c) Siemens AG, 2017-2021'
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
class LoadConfigException(Exception):
"""
Class for exceptions that appear while loading a configuration file.
"""
def __init__(self, message, filename):
super().__init__('{}: {}'.format(message, filename))
def load_config(filename):
"""
Load the configuration file and test if version is supported.
"""
(_, ext) = os.path.splitext(filename)
config = None
if ext == '.json':
import json
with open(filename, 'rb') as fds:
config = json.load(fds)
elif ext in ['.yml', '.yaml']:
import yaml
with open(filename, 'rb') as fds:
config = yaml.safe_load(fds)
else:
raise LoadConfigException('Config file extension not recognized',
filename)
validator = Draft4Validator(CONFIGSCHEMA)
validation_error = False
for error in validator.iter_errors(config):
validation_error = True
logging.error('Config file validation Error:\n%s', error)
if validation_error:
raise LoadConfigException('Error(s) occured while validating the '
'config file', filename)
try:
version_value = int(config['header']['version'])
except ValueError:
# Be compatible: version string '0.10' is equivalent to file version 1
# This check is already done in the config schema so here just set the
# right version
version_value = 1
if version_value < __compatible_file_version__ or \
version_value > __file_version__:
raise LoadConfigException('This version of kas is compatible with '
'version {} to {}, file has version {}'
.format(__compatible_file_version__,
__file_version__, version_value),
filename)
if config.get('proxy_config'):
logging.warning('Obsolete ''proxy_config'' detected. '
'This has no effect and will be rejected soon.')
return config
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
class IncludeException(Exception):
"""
Class for exceptions that appear in the include mechanism.
"""
pass
class IncludeHandler:
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
"""
Implements a handler where every configuration file should
contain a dictionary as the base type with and 'includes'
key containing a list of includes.
The includes can be specified in two ways: as a string
containing the path, relative to the repository root from the
current file, or as a dictionary. The dictionary must have a
'file' key containing the path to the include file and a 'repo'
key containing the key of the repository. The path is interpreted
relative to the repository root path.
The includes are read and merged from the deepest level upwards.
In case ignore_lock is false, kas checks if a file <file>.lock.<ext>
exists next to the first entry in top_files. This file is then appended
to the list of top_files.
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
"""
def __init__(self, top_files, top_repo_path, use_lock=True):
self.top_files = top_files
self.top_repo_path = top_repo_path
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
if use_lock:
lockfile = self.get_lockfile()
if Path(lockfile).exists():
logging.debug('includehandler: append lockfile %s', lockfile)
self.top_files.append(lockfile)
def get_lockfile(self):
file = Path(self.top_files[0])
return file.parent / (file.stem + '.lock' + file.suffix)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
def get_config(self, repos=None):
"""
Parameters:
repos -- A dictionary that maps repo names to directory paths
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
Returns:
(config, repos)
config -- A dictionary containing the configuration
repos -- A list of missing repo names that are needed \
to create a complete configuration
"""
repos = repos or {}
def _internal_include_handler(filename, repo_path):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
"""
Recursively loads include files and finds missing repos.
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
Includes are done in the following way:
topfile.yml:
-------
header:
includes:
- include1.yml
- repo: repo1
file: include-repo1.yml
- repo: repo2
file: include-repo2.yml
- include3.yml
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
-------
Includes are merged in in this order:
['include1.yml', 'include2.yml', 'include-repo1.yml',
'include-repo2.yml', 'include-repo2.yml', 'topfile.yml']
On conflict the latter includes overwrite previous ones and
the current file overwrites every include. (evaluation depth first
and from top to bottom)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
"""
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
missing_repos = []
configs = []
current_config = load_config(filename)
if not isinstance(current_config, Mapping):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
raise IncludeException('Configuration file does not contain a '
'dictionary as base type')
header = current_config.get('header', {})
for include in header.get('includes', []):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
if isinstance(include, str):
includefile = ''
if include.startswith(os.path.pathsep):
includefile = include
else:
includefile = os.path.abspath(
os.path.join(repo_path, include))
if not os.path.exists(includefile):
alternate = os.path.abspath(
os.path.join(os.path.dirname(filename),
include))
if os.path.exists(alternate):
logging.warning(
'Falling back to file-relative addressing '
'of local include "%s"', include)
logging.warning(
'Update your layer to repo-relative '
'addressing to avoid this warning')
includefile = alternate
(cfg, rep) = _internal_include_handler(includefile,
repo_path)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
configs.extend(cfg)
missing_repos.extend(rep)
elif isinstance(include, Mapping):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
includerepo = include.get('repo', None)
includedir = repos.get(includerepo, None)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
if includedir is not None:
try:
includefile = include['file']
except KeyError:
raise IncludeException(
'"file" is not specified: {}'
.format(include))
abs_includedir = os.path.abspath(includedir)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
(cfg, rep) = _internal_include_handler(
os.path.join(abs_includedir, includefile),
abs_includedir)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
configs.extend(cfg)
missing_repos.extend(rep)
else:
missing_repos.append(includerepo)
configs.append((filename, current_config))
# Remove all possible duplicates in missing_repos
missing_repos = list(OrderedDict.fromkeys(missing_repos))
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
return (configs, missing_repos)
def _internal_dict_merge(dest, upd, recursive_merge=True):
"""
Merges upd recursively into a copy of dest as OrderedDict
If recursive_merge is False, it will use the classic dict.update,
otherwise it will fall back on a manual merge (helpful for non-dict
types like FunctionWrapper)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
"""
if (not isinstance(dest, Mapping)) \
or (not isinstance(upd, Mapping)):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
raise IncludeException('Cannot merge using non-dict')
dest = OrderedDict(dest)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
updkeys = list(upd.keys())
if not set(list(dest.keys())) & set(updkeys):
recursive_merge = False
if recursive_merge:
for key in updkeys:
val = upd[key]
try:
dest_subkey = dest.get(key, None)
except AttributeError:
dest_subkey = None
if isinstance(dest_subkey, Mapping) \
and isinstance(val, Mapping):
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
ret = _internal_dict_merge(dest_subkey, val)
dest[key] = ret
else:
dest[key] = upd[key]
return dest
try:
for k in upd:
dest[k] = upd[k]
except AttributeError:
# this mapping is not a dict
for k in upd:
dest[k] = upd[k]
return dest
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
configs = []
missing_repos = []
for configfile in self.top_files:
cfgs, reps = _internal_include_handler(configfile,
self.top_repo_path)
configs.extend(cfgs)
for repo in reps:
if repo not in missing_repos:
missing_repos.append(repo)
added initial implementation of a include handler Splitting configuration files into multiple files and implementing an include mechanism allows to handle configurations more flexible and allowing following scenarios: - including kas configuration file from external meta layer into own project - splitting many different similar configurations into multiple files with only a couple of common files To include file its necessary to add a 'includes' entry into the kas configuration. To include files from the same repo, do it like this: ----- includes: - ../base/base_include.yml ----- The path is relative to the current configuration file. If they start with a path seperator ("/") they are absolute. To include files from a different repository, do it like this: ----- includes: - file: bsps/my_machine_include.yml repo: collected_machines repos: collected_machines: url: https://url.to.git.repo/ revspec: master ----- You have to make sure that the repository definition is available in your current file, or in any include that is available at this point. Yaml ("*.yml") and Json ("*.json") files are supported. Included in this patch are also some changes to the configuration file structure. Here is an overview: The value of the 'repos' key is a dictionary that maps a repo identifier to a dictionary that contains 5 entries: - url: The url to the repository. If that is missing, no git operations are used - refspec: The git refspec of the repository that is used. If that is missing the latest commit of the default branch is used. - name: The is defines under which name the repository is stored. If that is missing the repository identifier is used - path: The path where the repository is stored. If no url is given and a path is missing, the repository is referencing the repository under which the configuration file is stored. If no url is given and a path is specified, the repository is referencing a directory where layers might be stored. If an url is specified, path can be used to overwrite the default (kas_work_dir + repo.name) checkout directory. - layers: This contains a dictionary of layers, that should be included into the bblayers.conf. The keys are paths relative to the repository and the values can be used to exclude layers if they are one of "excluded", "disabled", "false", "0", "n", or "no". Also boolean values are accepted. Any other value, including "None" means that this layer is included into the bblayers.conf. If "layers" is missing or empty, the repository itself is included into the bblayers. If this is specified, the repository itself is not included into the bblayers.conf. Signed-off-by: Claudius Heine <ch@denx.de>
2017-06-21 13:32:58 +02:00
config = functools.reduce(_internal_dict_merge,
map(lambda x: x[1], configs))
return config, missing_repos