Commit Graph

54 Commits

Author SHA1 Message Date
Jan Kiszka
241d03f159 Switch overrides and dump plugin to new commit/branch schema
Lock files are now using the commit key, rather than legacy refspec, and
the plugin writes out flattened configs that are in line with the input
configs (only write refspec if input repo was using refspec).

This also requires slight adjustments of the related test case. Enhance
the dump test at this chance to actually validate the written commit ID
against the expected one.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-30 17:32:03 +02:00
Jan Kiszka
f1f358bed5 repos: git: Always replicate refs/ into local branches on checkout
If a refspec addresses an upstream branch which commit happens to be
local already, checkout will not properly replicate that into a local
branch. Extend contains_refspec_cmd so that it does not falsely report
the remotes/origin/ ref as locally available and rather triggers its
fetching.

The related test case was papering over this issue so far. Adjust it
accordingly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-30 13:36:26 +02:00
Felix Moessbauer
e274886ab0 test combination of kas menu and KAS_WORK_DIR
This patch adds two tests of the kas menu plugin in combination
with KAS_WORK_DIR.

Reported-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-30 11:08:49 +02:00
Felix Moessbauer
ea0baa06d0 refactor menu test to ease extending it
This patch refactors the menu test by moving the monkey patching out
into a fixture. When changing the cwd, we now use the monkeypatch
tooling. By that, the following tests are not affected by a failure in a
previous one, which leads to a cleaner test report.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-30 10:49:55 +02:00
Felix Moessbauer
6e3a8d9118 repos: add tests for common errors
This patch adds tests for common errors when working with patches.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-19 06:49:57 +02:00
Felix Moessbauer
ecd670e9ae fix: collect exceptions on task errors
When async tasks fail, all exceptions need to be collected to not get
subsequent exceptions about invalid future states. This is achieved by
gathering the task results, instead of just waiting for them. By
gathering the results, also user-requested cancellation (e.g. via
ctrl-c) works without throwing tons of additional exceptions.
Since ac437308 we more likely run into that case, which unvealed the bug.

By properly handling the exception, a TaskResultError is returned
instead of the underlying CommandExecError. This change is reflected in
the corresponding unit test.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-19 06:36:31 +02:00
Felix Moessbauer
d2ecff4243 test handling of command exec errors
This adds a test for common command exec errors when checking out
repositories. The test checks if the correct error is thrown
(CommandExecError) instead of failing silently or with an
unspecific exception.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-10 09:02:53 +02:00
Felix Moessbauer
222f07de69 refactor: port all sys.exit over to kas exceptions
This patch replaces all direct invocations of sys.exit outside of the
main invocation to KasUserError based exceptions. By that, only one
method for returning is used and return codes can be handled
consistently. In addition, this makes it possible to handle specific
errors differently.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-05 21:56:32 +02:00
Felix Moessbauer
bb2148016b do not shadow variables in command tests
As part of the tests of the dump plugins a variable was shadowed. By
that, one branch of the test was not executed.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-05-04 09:03:22 +02:00
Felix Moessbauer
e2bf6d74df add test for lockfile support
This commit adds a test that check the creation, effectiveness and
update of a lockfile. Testing this functionality via the dump plugin is
sufficient, as the plugin directly uses the checkout workflow.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
[Jan: fix over-long lines and removed assert brackets]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-04-23 14:05:04 +02:00
Jan Kiszka
78084c05bb tests: Clean up remaining temporary directory setups
This completes what 6b1df5f425 started.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-02-12 13:55:23 +01:00
Jan Kiszka
6b1df5f425 tests: Avoid useless directory creations
The pattern

mkdir test_commands
rm -rf test_commands

makes no sense. If there were test_commands before, mkdir would have
failed. Simply build the path and copy the source content in.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-02-05 21:18:35 +01:00
Felix Moessbauer
1c2c859a4a add test for KAS_REPO_REF_DIR logic
This patch adds a test for cloning with KAS_REPO_REF_DIR.
It explicitly tests the case that two repos with a single upstream URI
are added and also tests mercurial.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
[Jan: ensure Python 3.5 compatibility]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-01-05 20:03:25 +01:00
Felix Moessbauer
88a2b3d47b add test for resolve-env option in dump plugin
This patch adds a test for the --resolve-env option in the dump plugin.
Both cases resolve and non-resolve are tested, each for yaml and json
output format.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-12-13 14:41:00 +01:00
Felix Moessbauer
33250866ec test dump plugin support for resolved refs
This patch extends the test of the dump plugin.
In addition, we test if relative refspecs are expanded and if the
generated output can be used as input to kas again.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-12-13 14:39:51 +01:00
Felix Moessbauer
bfd6bd6195 add test of dump plugin
This patch adds a basic test of the dump plugin. We check if the configuration
is flattened and external references are included.
In addition, we check if no other files are referenced anymore.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-12-13 14:39:03 +01:00
Benedikt Niedermayr
4795c21fb6 tests: Add test case for environment variables
These additional test cases check if the env section
can:

- export variables with their default value
- add variables with 'None' assigned only to
  BB_ENV_PASSTHROUGH_ADDITIONS or to the deprecated
  BB_ENV_EXTRAWHITE
- bitbake: make sure that variables actually make it into bitbake

The BB_ENV_EXTRAWHITE variable is still present, so a test case has also
been added for this.

Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
[Jan: ensure python 3.5 compatibility, adjust copyright header]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-11-11 13:35:45 +01:00
Jan Kiszka
9afb1d8239 tests: Fix style warnings around assert usage
assert is not a function. Latest pycodestyle and flake8 warn about this.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-04 19:09:12 +02:00
Peter Hoyes
7647d7ad3e tests: Add failing test with repository includes
Add a new test to test_commands.py which:
 * Has an incomplete definition of externalrepo
 * Defines a local repository, subrepo
 * Includes a file from subrepo
 * The included file completes the definition of externalrepo
 * It also attempts to include a file from externalrepo

This test previously failed with the error message "No such file or
directory: '.../test_commands/tests/test_layers/test.yml'"

Signed-off-by: Peter Hoyes <Peter.Hoyes@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-07-12 09:21:07 +02:00
Felix Moessbauer
94e0e999f2 use relative layer dirs to make build relocatable
This patch replaces the absolute paths that are injected into BBLAYERS
by relative ones. These are relative to TOPDIR.
By that, the whole build directory becomes relocatable.

This is of value when using a shared sstate cache and build machines
with varying build locations (e.g. gitlab-ci runners).

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-04-27 15:57:08 +02:00
Tobias Schaffner
1c53b817b5 fix: Raise an error if no refspec is provided
Only local repositories may be configured without refspec.
Raise an error if a repository url is given but refspec is not.

Signed-off-by: Tobias Schaffner <tobias.schaffner@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-02-21 20:05:31 +01:00
Daniel Wagenknecht
358e5924c4 for_all_repos: adapt test
The `this` repo is skipped in the test for the `for-all-repos` command.
Adapt to checking for KAS_REPO_URL being empty instead of comparing the
KAS_REPO_NAME. This way the test will fail if the implementation
diverges from the documentation:
> KAS_REPO_URL: The URL from which this repository was cloned, or an
> empty string if no remote URL was given in the config file.

Signed-off-by: Daniel Wagenknecht <dwagenknecht@emlix.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-15 15:42:27 +02:00
Jan Kiszka
95df3720e1 tests: Add basic tests for menu plugin
This checks basic features of kas menu, some option selection, yaml
including, rebuilding and target selection.

The UI is operated by mocking smack's GridFormHelp.runOnce and some
result evaluation functions. Injecting keys from the keyboard was not
easily possibly (if at all), thus this approach. The menu rendering is
not shown due to pytest.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-09 15:02:22 +02:00
Jan Kiszka
ee2600a51b tests: test_patch: Restore current directory on exit
Avoids that succeeding tests may stumble over this. So far not an issue.

Drop a forgotten debug print at this chance.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-08 17:58:20 +02:00
Jasper Orschulko
fca78e5601 for_all_repos: Fix test case
An upcoming change will introduce returning non-zero status on the KAS
command if the for_all_repos command fails for one of the repos defined
in the kas configuration. This caused an issue with the non-git "this"
repo defined in the test.yml.

Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
[Jan: adjusted commit log to reflect different commit ordering]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-26 18:55:52 +02:00
Jan Kiszka
c7bd8393aa tests: Add include test case for multiple files in sub-dirs
This stresses the previously broken case of the incorrectly detected
repository root while including files.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-19 10:10:26 +02:00
Jan Kiszka
2077900b4e includehandler: Fix repo path detection for local includes
So far, repository paths for local includes were derived from the path
name of the config file containing the include, rather than using the
actual repository root as specified in the documentation. No one
complained so far, some layers simply adjusted their includes to this
inconsistency which was only discovered during refactorings.

Fix this issue by passing also the repository path along with the config
filename down the recursive _internal_include_handler calls. The
top-level repo path now needs to be retrieved before the creation of
IncludeHandler and passed to it then. This has the side effect of
enabling deviating top-level repo paths, a feature that will be used by
the upcoming changes for a .config.yaml file in KAS_WORK_DIR.

As there are existing users of the old behavior out there, fall back to
it if a local include cannot be found under the correct path and warn if
this succeeds. This allows smooth migration of layers to the right
pattern as they update their kas version.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-19 10:10:14 +02:00
Drew Reed
6b65de3e2f Enable gerrit/gitlab/github refspecs
By default git only fetches references under the refs/heads/ tree,
this patch adds support to kas to enable you to specify references
outside of the ref/heads tree.  This is useful as it allows you to
use uncommitted gerrit patchsets, Gitlab merge requests or github
pull requests that live under refs/changes/, refs/merge-requests
and refs/pull as the reference for a repo allowing the use of
in development changes.  When a refsepc is defined that starts
with refs/ an additional git fetch operation is preformed on the
repo to explicitly fetch the reference given so it can be checked
out for use.

Signed-off-by: Drew Reed <drew.reed@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-07-19 07:57:32 +02:00
Konrad Weihmann
b250c341fb repos: strip dot from layer name
when importing nested layer in combination with the top level layer like

meta-security
meta-security/meta-tpm

one can use

layers:
    .:
    meta-tpm:

in the spec file to import the parent layer.
This leads to the effect that

.
meta-tpm             = "HEAD:<hash>"

is shown in the build console and in /etc/build
when using image-buildinfo.bbclass.
To fix that just strip the trailing dot from each layer name.

Add test to avoid future regressions

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
[Jan: use only a single rstrip call, fix style]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-05-05 15:43:58 +02:00
Peter Hatina
f32290d425 Introduce KAS_BUILD_DIR environment variable
This variable can override default build path `${KAS_WORK_DIR}/build`.

Signed-off-by: Peter Hatina <peter@hatina.eu>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-03-10 07:53:09 +01:00
Paul Barker
9176d1f5b9 tests: Add tests for new for-all-repos & checkout commands
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-17 17:57:59 +01:00
Jan Kiszka
d5eb961a44 tests: Stress the build_system property
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-16 11:53:29 +01:00
Paul Barker
1253d22dee tests: Extract common changedir fixture
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-13 21:00:35 +01:00
Henning Schild
c3e1594b0e tests: small correction to git args
According to git-rev-parse(1) "-q" only does something in combination
with "--verify". Skip it, we do not need "--verify" because HEAD is
exactly one arg.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-09 08:57:04 +01:00
Adrian Stratulat
a5c8345ee6 repos: remove trailing slash from layer path
In order for oe-core to correctly display the layers and revisions
used in a build, the paths used in conf/bblayers.conf must have no
trailing path separator.

Signed-off-by: Adrian Stratulat <adrian.stratulat91@gmail.com>
[Jan: fix overlong line, update test case]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-09-23 09:34:28 +02:00
Henning Schild
4cfbf43f07 tests: add testcase for repo layers
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-07-07 11:48:57 +02:00
Henning Schild
0672592768 tests: introduce fixture to change back to right dir
Tests changing the working directory could leave us stranded somewhere
where all following tests could fail. Make sure to always go back on
failure.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-07-07 11:48:57 +02:00
Jan Kiszka
839c936abb Fix flake8 style reports
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-06-17 08:09:14 +02:00
Paul Barker
c2bbfee8f3 tests: Check refspec switch updates git branch/commit correctly
Mercurial has no concept of a local tracking branch so we only need to
test git repositories here.

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-06-17 07:58:44 +02:00
Paul Barker
23277836c4 tests: Check patches can still be applied after switching refspec
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-06-15 19:53:59 +02:00
Paul Barker
79bfe5a04d tests: Return to kas directory when test_patch finishes
This prevents subseqent tests from starting in the wrong directory.

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-06-15 19:53:54 +02:00
Henning Schild
e29035407f tests: test patching of symbolic refspecs i.e branches
That once caused issues when rolling back (checkout) since our local
branch we patched and commited on had the desired name.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-25 10:14:22 +02:00
Henning Schild
acc60f4f3a tests: make sure patches can create executables
Create a 755 file in a git and an hg repo, using a patch. Later make
sure the permissions are as expected.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-21 12:49:53 +02:00
Henning Schild
fafc017392 tests: add a testcase for applying patches
This adds a test where we clone a git and a hg repo and apply a single
patch and later a quilt series on each of them.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-20 17:54:15 +02:00
Daniel Wagner
25bc0c9c06 scripts: Remove pylint
pylint has been a great tool to get the initial code base into shape
but since then we mostely false postives. All those warnings needed to
be annonated.

This allong wouldn't be reason enough to remove it. The main problem
is that the pylint version on gitlab is updated in the back and
suddenly we get new warnings which are bogus.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-07 09:55:26 +01:00
Andreas Reichel
10cd13cf9c Correct grammar, spelling and copyright year in files
Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
2018-09-05 12:46:35 +02:00
Jan Kiszka
d4a615bb0b Command line based configuration file merge
This extends the command line syntax for specifying configuration files.
You can now combine files by concatenating them, separated by colons:

kas build base.yml:board.yml:feature.yml

The motivation for this feature is to avoid having to write tons of
configuration files that perform this combinations statically via
includes.

In order to avoid complications and prevent that users shoot themselves
too easily into their feet, we deny the case of distributing the
configuration files over multiple repositories. Either all files
specified on the command line come from the same repo, or they are all
local (without versioning control).

Based on idea by Claudius Heine.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:54:32 +02:00
Jan Kiszka
5c28272d73 includehandler: Fold GlobalIncludes into IncludeHandler class
No user for this abstraction in sight.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-08-27 07:48:08 +02:00
Daniel Wagner
9fc8c7735c test_includehandler: Fix indention
pylint reports

tests/test_includehandler.py:343:0: C0330: Wrong hanging indentation before block (add 4 spaces).
            'x.yml': header.format('''  includes: ["y.yml", "z.yml"]
            ^   | (bad-continuation)
tests/test_includehandler.py:345:0: C0330: Wrong hanging indentation before block (add 4 spaces).
            os.path.abspath('y.yml'): header.format('''  includes: ["z.yml"]
            ^   | (bad-continuation)
tests/test_includehandler.py:347:0: C0330: Wrong hanging indentation before block (add 4 spaces).
            os.path.abspath('z.yml'): header.format('''
            ^   | (bad-continuation)

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:26:47 +02:00
Daniel Wagner
f60a8b4e6d test_includehandler: Remove useless object inheritanc
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-07-18 16:26:47 +02:00