Commit Graph

415 Commits

Author SHA1 Message Date
Jan Kiszka
d02e87a0a7 Release 2.1.1 2020-05-19 08:03:26 +02:00
Jan Kiszka
d58a291513 repos: Silence pycodestyle error
New pycodestyle version, new warning: "l" might be confused with "1".

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-05-19 07:53:02 +02:00
Jan Kiszka
68809708e8 Release 2.1 2020-05-19 07:30:20 +02:00
Jan Kiszka
378457b2c8 kas-docker: Make loop device passing optional
This used to be required in older Isar versions (prio to commit
8b8a339cfaa4) for one configuration. It may still be required by
downstream layers, though it can be avoided. To overcome this
potential need to enter the host's root password without breaking those
cases, keep the feature but make it explicit (--with-loop-dev option).

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-05-15 18:37:18 +02:00
Jan Kiszka
385e42ff11 kas-docker: Warn when running as root
While this works from kas perspective, some build recipes will no like
to be run with ID 0 and may either fail loudly or subtly. Better warn
when a user applies sudo on kas-docker or otherwise runs it from a root
shell.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-05-15 18:37:13 +02:00
Claudius Heine
96df30a80f kas-docker: add aws-dir command parameter
The `--aws-dir` parameter for the kas-docker script allows to specify a
AWScli config directory which will then be mounted to `/etc/skel/.aws`
and copied by kas into the internal home directory using the `AWS_*`
environment variables.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-05-05 14:34:31 +02:00
Claudius Heine
267a53e78d kas/libcmds: add creation of $HOME/.aws
Bitbake provides a S3 fetcher (`lib/bb/fetch2/s3.py`), however the
`AWS_CONFIG_FILE` and `AWS_SHARED_CREDENTIALS_FILE` variables are not
transfered to the environment of the fetcher command (`aws` in this
case) in the `runfetchcmd` function (`lib/bb/fetch2/__init__.py`).

That means the location of these files need to be the default one, so
copying it in KAS to the new HOME directory is necessary.

This patch implements the copying of those files if the
`AWS_CONFIG_FILE` and `AWS_SHARED_CREDENTIALS_FILE` are set.

Per default there variables are not available in the bitbake
environment. To have them available there a `env` entry in the kas
configuration will be necessary.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-05-05 14:34:28 +02:00
Claudius Heine
e61dfb7650 Dockefile: add awscli for s3 fetcher
bitbake provides a fetcher for Amazon S3 storage using the awscli
command. This patch adds the awscli package to the container.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-04-29 14:54:25 +02:00
Jan Kiszka
2142fd855e Fix reporting of repo patch IDs
We were always printing the ID of the last patch of a repo because the
application loop only iterated over a set of patch paths, not the patch
entries. Fix that be assembling a tuple of path and ID in my_patches.

Reported-by: Mustafa Yuecel <mustafa.yuecel@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-04-10 18:10:17 +02:00
Jan Kiszka
1c1e3e592a kas-docker: Fix error message
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-18 17:02:46 +01:00
Jan Kiszka
789d3aa604 kas-docker: Fix a shellcheck warning
Resolves "SC2145: Argument mixes string and array". Harmless here, but
the result looks nicer anyway.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-18 17:02:46 +01:00
Jan Kiszka
a1d9ae15e0 kas-docker: Add support for spaces in paths
This requires consistent quoting of related variables. Furthermore, the
eval in trace() has to be removed, and along with it the additional
single-tick quotes of some parameters.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-18 17:01:44 +01:00
Jan Kiszka
5932dc77ff kas-docker: Eliminate intermediate DOCKER_ARGS variable
This will allow passing of all docker arguments in a properly quoted
way. More of them will need that in order to support paths with spaces.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-17 15:07:04 +01:00
Jan Kiszka
d96a26f1c8 kas-docker: Simplify SHELL variable passing
The SHELL variable we pass can neither contain spaces nor newlines - we
check for supported values. Simplify the related assignment.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-17 15:03:38 +01:00
Jan Kiszka
1ba260deeb kas-docker: Fix clean of non-isar builddirs
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-03-17 09:46:18 +01:00
Pierre-Jean Texier
71b9c43472 config: use 'qemux86-64' instead of 'qemu' as default for KAS_MACHINE
'qemu' is not a valid machine. So, let's use the default one defined in
poky -> 'qemux86-64' [1]

Fixes:

ERROR:  OE-core's config sanity checker detected a potential misconfiguration.
    Either fix the cause of this error or at your own risk disable the checker (see sanity.conf).
    Following is the list of potential problems / advisories:

    MACHINE=qemu is invalid. Please set a valid MACHINE in your local.conf, environment or other configuration file.

[1] - https://git.yoctoproject.org/cgit.cgi/poky/tree/meta-poky/conf/local.conf.sample#n37

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-02-18 13:05:13 +01:00
Jan Kiszka
6a67d69411 Address deprecated warnings with Python 3.8
All these chances work fine with 3.5, our minimally required version.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-01-09 11:53:29 +01:00
Jan Kiszka
e383e16570 Drop Python 3.4 compatibility, include 3.8
PyYAML started to require at least 3.5. There is possibly a way to pick
an older version of it to fulfill its needs, but it's also not worth to
invest effort in this old version anymore.

Use that chance to include 3.8 in the support scope.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-01-09 11:35:40 +01:00
Sam Lewis
b5ba370049 Ensure SSH key ends with newline
In at least some versions of the 'ssh-add' command, adding a SSH key
requires that SSH key is newline terminated. If it is not, ssh-add
doesn't add the key and, instead, unhelpfully prompts for a
passphrase for the key ('Enter passphrase for (stdin):').

This change makes sure the key is terminated with a newline character to
hopefully avoid this issue occurring as often.

Signed-off-by: Sam Lewis <sam.vr.lewis@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-01-09 11:18:54 +01:00
Jan Kiszka
a5d0153e84 Dockerfile: Use Debian's gosu
The binary download from upstream dates back to jessie times.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-12-05 20:02:06 +01:00
Jan Kiszka
141476203d Release 2.0 2019-11-20 06:53:15 +01:00
Tim Orling
f0e0fa5da4 Dockerfile: add ps for testimage
HOSTTOOLS requires ps when running testimage.

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-17 13:04:38 +01:00
Jan Kiszka
52c6ff0a0f Dockerfile: Update package list according to Yocto 3.0 essential needs
Move those needs to top at this chance and refactor the rest. The net
additions are:

 - python3-pexpect
 - debianutils
 - iputils-ping
 - python3-git
 - python3-jinja2
 - libegl1-mesa
 - libsdl1.2-dev
 - pylint3
 - xterm

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-17 12:54:59 +01:00
Amol Dalavi
f39d3e48d1 Dockerfile: add package git-lfs
Kas can clone git repositories with LFS objects

Signed-off-by: Amol Dalavi <dalavi.amol@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-08 10:36:41 +01:00
Jan Kiszka
230c5a9572 Add support for new multiconfig syntax
Recent bitbake requires to declare multiconfig targets via "mc:".

Use that chance to refactor the function and match on the colon as well.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-08 10:36:08 +01:00
Jan Kiszka
102eac9d9b Dockerfile: Add python3-wheel dependency
Silences error messages during kas installation step.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-09 16:51:08 +02:00
Jan Kiszka
8b91666943 travis: Add python 3.7 test
We just need to lift the dist for that target to xenial - but not to
bionic because 3.4 is then no longer supported.

In xenial, mercurial is part of travis image. Additionally installing it
is not needed and actually breaks things. But things are more
complicated because hg is under /usr/local/bin, and kas used a built-in
PATH for tools that does not include this.

At this chance, remove "sudo: required" - no longer needed.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-04 21:10:49 +02:00
Jan Kiszka
a67476a9ac travis: Enforce latest pytest
This fixes breakages due to outdated/inconsistent version in travis.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-04 20:20:51 +02:00
Henning Schild
63f8a28321 Docker: move on to buster
buster is the new stable, use it

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-01 16:57:20 +02:00
Jan Kiszka
e9ca55a239 Release 1.1 2019-10-01 16:21:40 +02:00
Jan Kiszka
a8dfbda062 scripts/release: Remove pypitest
There is no value in pushing to that registry.

Also drop the broken link at this chance.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-01 16:07:23 +02:00
Jan Kiszka
3ef3ac855e Avoid "RuntimeError: Event loop is closed"
Triggered by pytest now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-25 11:00:45 +02:00
Henning Schild
e29035407f tests: test patching of symbolic refspecs i.e branches
That once caused issues when rolling back (checkout) since our local
branch we patched and commited on had the desired name.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-25 10:14:22 +02:00
Jan Kiszka
26a1a0b03b Create from local branch before applying patches with git
If refspec is pointing to an upstream branch name, the checkout for a
second run will not reset a repo back to the upstream commit. That is
because we applied the repo patches to a local branch that carries the
name of the upstream one.

Fix that by switching to local branch prior to applying any patches.

Reported-by: Belisko Marek <marek.belisko@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-23 13:08:51 +02:00
Henning Schild
acc60f4f3a tests: make sure patches can create executables
Create a 755 file in a git and an hg repo, using a patch. Later make
sure the permissions are as expected.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-21 12:49:53 +02:00
Henning Schild
81c5beedaa repos: apply patches with version control system and not patch
patch can not handle file mode headers and can therefore not be used to
create executable files. fixes 8a2a21d799

In addition commit every single patch and treat git patches as
potentially just diffs and not mails.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-21 12:49:47 +02:00
Henning Schild
363eca077c Revert "Dockerfile: Patch oe-git-proxy to truly disable proxying by default"
This reverts commit 7bd738a43c.

The bug has been fixed upstream, ne need to patch the script anymore.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-18 22:13:03 +02:00
Jan Kiszka
02fcc52007 Dockerfile: Add tmux and libncurses-dev
Needed for devshell and menuconfig targets.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-12 15:12:17 +02:00
Jan Kiszka
61f31c75fd Update maintainer name
I took over the maintainership from Daniel.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 13:43:15 +02:00
Jan Kiszka
8e36b4f72e release: Adjust to new maintainer's preferences
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:52:01 +02:00
Jan Kiszka
0137a92f33 release: Pick up old version from git
The reasonable expectation is that the last tag is also the last
release.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:52:01 +02:00
Jan Kiszka
0a1072ec79 Encode image version into kas-docker on release
This interlocks kas-docker with the corresponding kas images. The goal
is to avoid surprises because of API changes between kas-docker and the
images as well as to allow users to freeze the builder image by using a
specific kas-docker version.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:52:01 +02:00
Jan Kiszka
dbde2a75fc Revert "Docker: move on to buster"
This reverts commit 97942197bc.

This is only temporarily in order to release another 1.x version that is
based on stretch. Older Yocto versions, maybe up to 2.4 or 2.5, require
stretch due to incompatibilities with host tools such as git.

After that release, this commit will be restored, and we will roll out a
2.x series that shall use buster in its images.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:51:39 +02:00
Jan Kiszka
419e196cbb build: Fix name of argument variable
Adding --cmd to the task argument changed the name of the destination
variable, and that broke the whole subcommand because 'cmd' is already
used by the top-level parser. Fix this by setting dest explicitly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-03 10:42:19 +02:00
Jan Kiszka
8558b566d9 build: Add -c and --cmd as aliases for --task
This aligns us with bitbake. Moreover, -c/--cmd is easier to tell apart
from --target. Keep --task for backward compatibility.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-31 12:30:53 +02:00
Jan Kiszka
08a1326825 kas-docker: Reformat help output
Make sure to fit into 80 columns.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-30 13:41:16 +02:00
Jan Kiszka
b2af1bcfe7 config: Remove dead code
Not used anywhere, dates back to dynamic configs.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-30 09:41:46 +02:00
Matteo Facchinetti
1dfb4ebb06 Dockerfile.isar: preserve /var/tmp directory from an unwanted removal
Fix a bug in the Docker image build process
that delete /var/tmp directory.
Desired is only delete its content.

Signed-off-by: Matteo Facchinetti <matteo.facchinetti@sirius-es.it>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-29 18:55:37 +02:00
Jan Kiszka
f3f004f4ea kas-docker: Allow to define docker image revision
By default, we pull latest when running kas-docker. That may not be
desired in all cases, e.g. testing different image revisions or nailing
down those used in a build. Allow to override the default via the
environment variable KAS_IMAGE_VERSION.

For now, we stick with the default "latest", but we may also hard-code
the last release here at some later point.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-23 15:24:36 +02:00
Henning Schild
fafc017392 tests: add a testcase for applying patches
This adds a test where we clone a git and a hg repo and apply a single
patch and later a quilt series on each of them.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-20 17:54:15 +02:00