There is no good reason to only warn about an incorrect patch entry or
a missing patch file. This just causes CI systems to succeed with a
build - or fail at unexpected places.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Analogously to bitbake's PREMIRRORS, this allows to define alternative
sources for repo URLs specified in kas files.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
PEP8:
"""
To better support introspection, modules should explicitly declare the
names in their public API using the __all__ attribute.
"""
flake8 reports
./kas/kas.py:51:1: F401 '.build' imported but unused
./kas/kas.py:52:1: F401 '.shell' imported but unused
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
PEP8:
"""
To better support introspection, modules should explicitly declare the
names in their public API using the __all__ attribute.
"""
reported by flake8:
./kas/__init__.py:26:1: F401 '.__version__.__version__' imported but unused
./kas/__init__.py:27:1: F401 '.__version__.__file_version__' imported but unused
./kas/__init__.py:27:1: F401 '.__version__.__compatible_file_version__' imported but unused
./kas/__init__.py:28:1: F401 '.configschema.CONFIGSCHEMA' imported but unused
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
pylint has been a great tool to get the initial code base into shape
but since then we mostely false postives. All those warnings needed to
be annonated.
This allong wouldn't be reason enough to remove it. The main problem
is that the pylint version on gitlab is updated in the back and
suddenly we get new warnings which are bogus.
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
They were ignored so far is the config file was specifying a target or a
task, respectively. But the proper priority is (in descending order):
1. command line options
2. environment (KAS_TARGET, KAS_TASK)
3. kas config
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
In 3.7, async became a reserved word. So we can no longer use it
directly for pre-3.5 compatibility.
At this chance, extract the common compat part into a helper function.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allow to use the host-side SSH agent when cloning protected
repositories via kas. We already do this for the build environment.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Reviewed-by: Lukasz Majewski <lukma@denx.de>
Divide the SetupRepos command into three parts: init, step and finish
Build a macro-based loop with the step part of the function.
Rewrite this in build.py and shell.py
Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
To keep the macro style of the program flow and explicitely state the
command sequence with Macro.add, we need a mechanism that repeats
several steps of the Macro depending on a given condition. This will be
used by the repo checkout logic.
Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
This extends the command line syntax for specifying configuration files.
You can now combine files by concatenating them, separated by colons:
kas build base.yml:board.yml:feature.yml
The motivation for this feature is to avoid having to write tons of
configuration files that perform this combinations statically via
includes.
In order to avoid complications and prevent that users shoot themselves
too easily into their feet, we deny the case of distributing the
configuration files over multiple repositories. Either all files
specified on the command line come from the same repo, or they are all
local (without versioning control).
Based on idea by Claudius Heine.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This allows the caller to differentiate between "no repo found" and
"result points to repo". Default behavior remain unchanged (return
provided path itself if no repo is found).
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Rather than handing down a complete config, only pass what is really
needed: a repository fallback path. This can already be determined by
the caller.
Inside Repo.factory(), we can retrieve the global context now and can
stop relying on a reference stored in the config.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If env is not provided, use the global context environment. Then drop
all every passing of that parameter.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This helps to avoid circular dependencies. It also allows the Config
constructor to make use of the global context.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This will avoid the increasingly ugly pushing around of a context
instance as parameter to various methods/functions. Everyone can now
simply call get_context() after build or shell called
create_global_context().
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This fixes a regression of 5b85fba0: diff-index is porcelain and seems
to lack some explicit refresh prior to being called. Therefore, it can
incorrectly detect files as being modified although a "git diff" will
not - and will actually also update the status so that the next
diff-index would be fine again.
Fix those false positives while still detecting also cached changes by
using git status -s instead.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The ID_LIKE tag that get_distro_id_base also queries may contain
multiple, space-separated values. Opensuse Leap 15.0 is exploiting that,
e.g. ("suse opensuse").
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
If a kas-file inside a repo includes another kas-file form a repo
which is not checked-out, and this 2nd repo can only be checked-out
with an SSH-key, kas fails. This is, because the constructor of
the Config class used to fetch missing repos already before the
SSH-Agent was setup.
This patch refactors the way in which kas checks-out repositories. This
also required the Config class to be split into Context and Config,
where the new Context is the build-Context, i.e. containing the
environment of commands executed by kas and the new Config is the Config
of kas itself, i.e. containing the repo dictionary.
This way it is possible to initialize the context needed for SSH setup
independently of the kas configuration.
The commands ReposFetch and ReposCheckout are refactored into a
SetupRepos command. This command parses the include files and
successively checks-out needed repos and updates the config as long
as repos are missing. The logic is taken directly from the constructor
of the former Config class (with minor adaptations).
Further refactoring will be needed to clean up the conceptual
programming patterns (i.e. to keep the Macro-Style consistent),
and to re-add the macro pattern regarding the above ReposFetch.
Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
pylint reports:
kas/includehandler.py:105:0: R0205: Class 'IncludeHandler' inherits from object, can be safely removed from bases in python3 (useless-object-inheritance)
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
pycodestyle v2.4.0 checks for W606 (async deprecated), but
we can't disable it on source code level with '# noqa'. Let's disable
the check globaly. We will still catch it with pylint.
By disabling the W606 we need also to list the W503 which was
previously active.
While at it fix following things:
./docs/conf.py:252:6: E121 continuation line under-indented for hanging indent
./docs/conf.py:257:6: E126 continuation line over-indented for hanging indent
./kas/build.py:123:17: W504 line break after binary operator
./kas/configschema.py:48:25: E123 closing bracket does not match indentation of opening bracket's line
./kas/configschema.py:49:21: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:82:13: E123 closing bracket does not match indentation of opening bracket's line
./tests/test_includehandler.py:121:13: E123 closing bracket does not match indentation of opening bracket's line
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Since 'get_repo_dict' does much more than just querying available
repositories, this patch makes it private. The same data is now
available via the 'repo_dict' property. This also means that
'get_repos' is now faster, because it accesses cached data, instead of
calling 'get_repo_dict' that generates this data.
This patch also allows 'get_repos' and 'repo_dict' to be used even in the
'Repo' class. 'get_repo_dict' creates new 'Repo' instances and thus
invalidates the current instance.
This patch is in preparation of the 'patch support'-feature, that
depends on this patch.
Signed-off-by: Claudius Heine <ch@denx.de>
is_dirty_cmd() uses `git diff --shortstat` to see if the repo can be
safely checked out. This only checks for the modified files and does
not take the cached(staged) files into account.
There can be cases where the cached changes could be overwritten by a
checkout. In this case, the git checkout command aborts with an error
message telling the user to commit or stash their changes.
Use `git diff-index HEAD --shortstat` to determine if there are any
changes in the tracked files, whether cached or not.
https://git-scm.com/docs/git-diff-index
Signed-off-by: Abdur Rehman <abdur_rehman@mentor.com>
This reverts commit b9032ec025.
The commit breaks inclusion chains of kas files. In my example a
first kas-file includes a second one. The second one is supposed to
add repos but kas does not fetch those.
Reported-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Copying the .gitconfig failes if the current user doesn't have one,
since the commits that are done within kas, will not be pushed or
merged anywhere, we can just create a minimal .gitconfig in the temp
home directory of kas.
Signed-off-by: Claudius Heine <ch@denx.de>
With this commit its now possible to patch 3rd party repos before bitbake is started.
Example:
This is our repo:
.
├── kas.yml
├── first-patch.patch
└── quilt-patches
├── second-patch.patch
├── third-patch.patch
└── series
Content of kas.yml:
header:
version: 8
repos:
my:
third-party:
url: "git://example.com/third-party.git"
refspec: "35adf4...34"
patches:
01-first:
repo: my
path: "first-patch.patch"
02-second:
repo: my
path: "quilt-patches"
Currently only 'git' repositories can be patched.
Signed-off-by: Claudius Heine <ch@denx.de>
Since 'get_repo_dict' does much more than just querying available
repositories, this patch makes it private. The same data is now
available via the 'repo_dict' property. This also means that
'get_repos' is now faster, because it accesses cached data, instead of
calling 'get_repo_dict' that generates this data.
Signed-off-by: Claudius Heine <ch@denx.de>
