Commit Graph

275 Commits

Author SHA1 Message Date
Jan Kiszka
0137a92f33 release: Pick up old version from git
The reasonable expectation is that the last tag is also the last
release.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:52:01 +02:00
Jan Kiszka
0a1072ec79 Encode image version into kas-docker on release
This interlocks kas-docker with the corresponding kas images. The goal
is to avoid surprises because of API changes between kas-docker and the
images as well as to allow users to freeze the builder image by using a
specific kas-docker version.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:52:01 +02:00
Jan Kiszka
dbde2a75fc Revert "Docker: move on to buster"
This reverts commit 97942197bc.

This is only temporarily in order to release another 1.x version that is
based on stretch. Older Yocto versions, maybe up to 2.4 or 2.5, require
stretch due to incompatibilities with host tools such as git.

After that release, this commit will be restored, and we will roll out a
2.x series that shall use buster in its images.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:51:39 +02:00
Jan Kiszka
419e196cbb build: Fix name of argument variable
Adding --cmd to the task argument changed the name of the destination
variable, and that broke the whole subcommand because 'cmd' is already
used by the top-level parser. Fix this by setting dest explicitly.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-03 10:42:19 +02:00
Jan Kiszka
8558b566d9 build: Add -c and --cmd as aliases for --task
This aligns us with bitbake. Moreover, -c/--cmd is easier to tell apart
from --target. Keep --task for backward compatibility.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-31 12:30:53 +02:00
Jan Kiszka
08a1326825 kas-docker: Reformat help output
Make sure to fit into 80 columns.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-30 13:41:16 +02:00
Jan Kiszka
b2af1bcfe7 config: Remove dead code
Not used anywhere, dates back to dynamic configs.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-30 09:41:46 +02:00
Matteo Facchinetti
1dfb4ebb06 Dockerfile.isar: preserve /var/tmp directory from an unwanted removal
Fix a bug in the Docker image build process
that delete /var/tmp directory.
Desired is only delete its content.

Signed-off-by: Matteo Facchinetti <matteo.facchinetti@sirius-es.it>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-29 18:55:37 +02:00
Jan Kiszka
f3f004f4ea kas-docker: Allow to define docker image revision
By default, we pull latest when running kas-docker. That may not be
desired in all cases, e.g. testing different image revisions or nailing
down those used in a build. Allow to override the default via the
environment variable KAS_IMAGE_VERSION.

For now, we stick with the default "latest", but we may also hard-code
the last release here at some later point.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-23 15:24:36 +02:00
Henning Schild
fafc017392 tests: add a testcase for applying patches
This adds a test where we clone a git and a hg repo and apply a single
patch and later a quilt series on each of them.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-20 17:54:15 +02:00
Henning Schild
8a2a21d799 repos: change the way we apply patches
Switch to applying all patches with "patch" and later do a "<vcs> add;
<vcs> commit". Also iterate over quilt series files ourselfs and apply
the patches one by one.

This means we can apply patches on mercurial as well as on git.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-20 12:53:54 +02:00
Henning Schild
df8d7b929d repos: Fix up the basic mercurial support
The interface cmd prototypes got out of sync so mercurial stopped
working. Align with git again to fix that.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-14 12:10:08 +02:00
Pascal Bach
66bd078e7f repos: make sure the remote url is matching the kas file
Currently the the remote URL is only set during the initial clone.
This might lead to confusion with the user as changing the URL in the kas.yml
does not get reflected and thus errors about references not found show up.

This change makes sure the URL is always set to the value specified in the
kas.yml file.

For mercurial this currently prints a warning as there is no command to change
the default path.

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
[Jan: fix up style, adjust comment on hg's set_remote_url_cmd]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-13 20:38:03 +02:00
Jan Kiszka
e9659cd19b Docker: Add quilt to kas-isar
Will become a hard dependency when switching the (currently broken)
patch class of Isar to one derived from OE. Prepare for that.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-13 18:41:08 +02:00
Jan Kiszka
1988cdd26b kas-docker: Propagate only supported SHELL settings
Other values will cause the container to fail because the shell binary
is missing. This avoids non-obvious failures when the host SHELL is
configured differently.

Reported-by: Phillipp von Rotenhan <phillipp.von_rotenhan.ext@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-08 14:48:52 +02:00
Henning Schild
97942197bc Docker: move on to buster
buster is the new stable, use it

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-08 14:39:40 +02:00
Vijai Kumar K
6ed21ad62c Dockerfile: Add gnupg package
- gnupg is currently installed into kas-docker due to reprepro's
requirement. Hence, available only in isar version of kas-docker.
- Add gnupg explicitly in Dockerfile. It is needed for both isar and
yocto builds.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-08 14:39:21 +02:00
Daniel Wagner
907816a5c4 Release 1.0 2019-03-21 13:07:32 +01:00
Jan Kiszka
2fc711a459 Dockerfile.isar: Adjust binfmt setup for qemu-user-static
It turned out [1] that our current way of setting up binfmt_misc during
container start does not reliably work, namely does not overwrite
incompatible host settings. Running dpkg-reconfigure solves this.

To avoid confusing warnings of the kind

update-binfmts: warning: qemu-aarch64 already enabled in kernel.

we filter them out. That will continue to flood the console with
warnings in case the container is started without privileges.

[1] https://groups.google.com/forum/#!msg/isar-users/2G-Diz0BR1Y/Y0X6eHggCgAJ

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-03-20 09:49:34 +01:00
Henning Schild
6fdbe299a4 docker: Take qemu-user-static from buster for Isar image
Doing that will allow building arm64 buster images with Isar. The new
qemu works just fine for all older suites like stretch.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-26 10:24:38 +01:00
Daniel Wagner
5b5694daf7 Release 0.20.1 2019-02-18 15:03:10 +01:00
Jan Kiszka
3eab427cc0 kas-docker: Restore KAS_PREMIRRORS support
Got broken by the merge in 0cb5d17e33.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-18 14:54:46 +01:00
Daniel Wagner
3ce0a1f941 Release 0.20.0 2019-02-18 11:15:00 +01:00
Hosgor, Tolga (CT RDA DS EU TR MTS)
0cb5d17e33 kas-docker: add --no-proxy-from-env option
Allow kas-docker to use proxy configuration of docker client instead of
inheriting proxy configuration from the current environment.

Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) <tolga.hosgor@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:31:44 +01:00
Jan Kiszka
2f24bdeac2 repos: Fail on unsuccessful patching
There is no good reason to only warn about an incorrect patch entry or
a missing patch file. This just causes CI systems to succeed with a
build - or fail at unexpected places.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:22:55 +01:00
Jan Kiszka
2f7650bb05 Add KAS_PREMIRRORS support
Analogously to bitbake's PREMIRRORS, this allows to define alternative
sources for repo URLs specified in kas files.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:20:35 +01:00
Jan Kiszka
dce5c0029a kas-docker: Enable multi-line variables
We need to double-quote the content of the variables in order to
preserve newlines.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:20:34 +01:00
Jan Kiszka
7bd738a43c Dockerfile: Patch oe-git-proxy to truly disable proxying by default
Upstream oe-git-proxy has a bug [1] and currently does proxying
unconditionally, just being stopped in the absence of other proxy
settings. Patch in the fix until upstream picks it up.

[1] http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278496.html

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:20:32 +01:00
Jan Kiszka
1b6e3f2fb9 kas-docker: Pass in NO_PROXY
Required to control oe-git-proxy exceptions.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:20:29 +01:00
Maciej Pijanowski
4d09e411ef docs/userguide.rst: fix typos, improve grammar
Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com>
2019-01-17 15:17:14 +01:00
Jan Kiszka
dabda7617f kas-docker: Translate all kas files into containter paths
Ensure that all file elements of the kas file parameter are properly
canonicalized and mapped onto the container volume. This is important
when kas-docker is called using relative paths while PWD is not the
repository root.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-01-02 16:56:04 +01:00
Henning Schild
70c568f05f CONTRIBUTING: add subscription links for list to document
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-12-17 10:58:33 +01:00
Henning Schild
ffbfedaa2a kas-docker: fix hg argument for working directory
Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-12-17 10:53:20 +01:00
Jan Kiszka
280a695ab3 kas-docker: Pass build and shell options into container
This allows invocations like "kas-docker build --task clean kas.yml".

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-12-17 10:53:20 +01:00
Jan Kiszka
5a68998ba7 kas-docker: Allow passing multi-word variables into the container
This is complex, not just because we are passing the arguments to the
trace function, but already because they need to be protected in their
structure when unrolling them at the top level. The trick seems to be
using the argument list via set and "$@" to preserve the individual
words. Due to the trace indirection, we additionally need the single
quotes around the variable values.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-12-17 10:53:20 +01:00
Jan Kiszka
ccc58c7010 kas-docker: Fix SSH_DIR check
We were missing a closing brace, and the line became rather long as well.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-12-17 10:53:20 +01:00
Daniel Wagner
fd58905753 kas-docker: Add missing } to variable derefencing
Reported-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-17 10:50:55 +01:00
Cirujano Cuesta, Silvano
49a6bbfc82 kas-docker: enable passing SSH configs
This commit enables passing the Docker container a directory containing
SSH configuration and optionally SSH identities and list of known SSH
hosts. Basically what you might expect to find in the ~/.ssh directory
of the container.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Suggested-by: Henning Schild <henning.schild@siemens.com>
2018-12-14 14:33:45 +01:00
Daniel Wagner
46c7d6e38c kas: Better support introspection
PEP8:
"""
To better support introspection, modules should explicitly declare the
    names in their public API using the __all__ attribute.
"""

flake8 reports

./kas/kas.py:51:1: F401 '.build' imported but unused
./kas/kas.py:52:1: F401 '.shell' imported but unused

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-07 09:56:09 +01:00
Daniel Wagner
294d4df286 __init__: Better support introspection
PEP8:
"""
To better support introspection, modules should explicitly declare the
names in their public API using the __all__ attribute.
"""

reported by flake8:

./kas/__init__.py:26:1: F401 '.__version__.__version__' imported but unused
./kas/__init__.py:27:1: F401 '.__version__.__file_version__' imported but unused
./kas/__init__.py:27:1: F401 '.__version__.__compatible_file_version__' imported but unused
./kas/__init__.py:28:1: F401 '.configschema.CONFIGSCHEMA' imported but unused

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-07 09:56:09 +01:00
Daniel Wagner
e24f8cff7c scripts: Add flake8 checker
Add code and style checker flake8.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-07 09:56:08 +01:00
Daniel Wagner
25bc0c9c06 scripts: Remove pylint
pylint has been a great tool to get the initial code base into shape
but since then we mostely false postives. All those warnings needed to
be annonated.

This allong wouldn't be reason enough to remove it. The main problem
is that the pylint version on gitlab is updated in the back and
suddenly we get new warnings which are bogus.

Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2018-12-07 09:55:26 +01:00
Claudius Heine
98b4d888bc kas-docker: made loop back device setup more idiot proof
I understood the message:

    Setting up loop device requires root privileges

as suggestion that I should start this script as root and hit
^C before `sudo` could prompt for my password.

This patch removes that line and adds a sudo prompt to elaborate on why
root permissions are needed and what is done with them. It also adds a
check if this was unsuccessful and suggests to the user to do it
themselves. This will also be shown if sudo is not installed.

Signed-off-by: Claudius Heine <ch@denx.de>
2018-11-30 11:17:02 +01:00
Jan Kiszka
90ae592ff1 docker: Align also caller's group ID with container
This ensures that both UID and GID of the builder user inside the
container is aligned with the caller of kas-docker - or that of "docker
run" when "-e GROUP_ID=..." is specified.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-11-30 11:15:47 +01:00
Jan Kiszka
fe344f93f1 Fix --target and --task options
They were ignored so far is the config file was specifying a target or a
task, respectively. But the proper priority is (in descending order):

1. command line options
2. environment (KAS_TARGET, KAS_TASK)
3. kas config

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-11-07 17:24:36 +01:00
Cirujano Cuesta, Silvano
0eb2f5edeb kas-docker: make usage printing POSIX compatible
The usage of 'echo -e' for printing the usage message is not POSIX
conform and is not being supported in the default shell of Debian
(dash).

As the 'Rationale' of the POSIX 'printf' documentation states, it was
created "due to irreconcilable differences in the various versions of
echo extant" and it should provide better compatibility throughout
different shells.

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
2018-11-07 14:54:45 +01:00
Jan Kiszka
7155769f06 libkas: Ensure Python 3.7 compatibility
In 3.7, async became a reserved word. So we can no longer use it
directly for pre-3.5 compatibility.

At this chance, extract the common compat part into a helper function.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-11-07 14:51:45 +01:00
Jan Kiszka
5025f89c01 shell: Drop pointless --target parameter
The target plays no role when invoking the shell because bitbake is not
called.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-11-07 14:17:14 +01:00
Jan Kiszka
a84e2c2222 kas-docker: Pass ftp_proxy variable
Forgotten because it's rarely used by now.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-10-02 10:38:58 +02:00
Jan Kiszka
ecbda0494b Remove SSH_AGENT_PID forwarding
This variable plays no role in using the agent for authentication, only
in managing its life-cycle.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-09-17 16:23:46 +02:00