Commit Graph

61 Commits

Author SHA1 Message Date
Jan Kiszka
a39d5a00c5 container: Add copyright headers and improve comments
Dockerfile and container-entrypoint were missing that header.

Furthermore, the leading comment in Dockerfile got out-of-date, and we
should rather add section marker for the kas and kas-isar targets.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-02-07 10:38:39 +01:00
Jan Kiszka
492b2c56ab container: Rework uid/gid alignment with caller
Already create the builder user/group during container image build and
only align the IDs in the entrypoint if started with a non-zero USER_ID.
The primary gain is code simplification because this removes some
dynamics from the entrypoint.

As this refactoring avoids that gitlab-ci runners start the container as
root, it was also supposed to resolve the mismatch between the owner of
the checked-out repo and builder user. Unfortunately, this does not work
yet, and the reason is still unclear.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-02-07 10:38:27 +01:00
Jan Kiszka
a596e2ac33 kas-isar: Rework isar-specific entrypoint setup
Leave the isar-only commands commented-out in the container-entrypoint
and simply remove the comment when building kas-isar. This is simpler,
more readable and also more robust against changes of the entrypoint
file.

While at it, avoid a separate layer for modifying container-entrypoint.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-02-06 10:30:04 +01:00
Hannah Kiekens
b27ab659f3 Dockerfile: Add unzip package to kas-base
Was lost in the refactoring of a6b18abc8a.

Signed-off-by: Hannah Kiekens <hannah.kiekens@mind.be>
[Jan: refer to the causing commit]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2023-01-24 07:11:12 +01:00
Jose Quaresma
c0d00aa637 Dockerfile: avoid deploy the python pip cache
This patch prevents deployment of the pip cache folder that is only
needed during installation. Which is also done with the apt,
where at the end of the install operation we delete the
cache and all temporary folders.

The reducion of the image size is insignificant:
| builder@1373d0b95f44:/build$ sudo du -sh /root/.cache/pip
| 1.6M    /root/.cache/pip

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-10-14 11:51:48 +02:00
Jose Quaresma
044de41a9a Dockerfile: adds a comment about the installation on the downstream layers
The install package list are actually taking 1:1 from their documentation,
so there some packages that can already installed by other downstream layers.
This will not change any image sizes on all the layers in use.

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-10-14 11:51:22 +02:00
Jan Kiszka
10aa308334 Dockerfile: Test kas installation by printing its version
Same effect but less verbose and potentially confusing output.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-26 15:16:50 +02:00
Jan Kiszka
a6b18abc8a Dockerfile: Rework generation of kas images
While kas and kas-isar surely have a common ground like the kas tool
itself, kas-isar does not need all the toolchain packages that are
essential Yocto dependencies. Splitting up the images earlier allows to
shrink kas-isar by almost 400 MB.

Use this chance to model both images as different build stages of the
same Dockerfile. That is simpler than creating a temporary "kas-base"
image via a separate Dockerfile.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-26 14:34:37 +02:00
Jan Kiszka
46b1c199a3 Dockerfile: Drop bc as Yocto dependency
Prior to the official first release of kas, this was once introduced to
fulfill kernel build dependencies. That is not longer be needed with
modern Yocto versions (if it ever actually was).

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-26 14:32:40 +02:00
Jan Kiszka
3863d46f17 Dockerfile: Refresh Yocto build dependency list
Use the list from Kirkstone. This should cover the needs back to Dunfell
at least as well.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-08-26 13:47:14 +02:00
Diego Sueiro
720aebc79b Dockerfile: Add telnet
Some emulators use telnet to provide console access to the system.

Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-06-21 12:57:24 +02:00
Zhibin Dong
be4501b872 Dockerfile: reduce the image size
At my debian11/amd64 machine, the size of newer image is 889MB, while the
older one is 908MB.

Refs: #81
Signed-off-by: Zhibin Dong <zhibin.dong@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-05-16 09:12:06 +02:00
Felix Moessbauer
94641a7b72 add pigz package to enable parallel compression of tar.gz archives
This patch adds the pigz package to enable parallel compression
and decompression support for the sstate artifacts.
This significantly speeds up compression of large artifacts on
build systems with many cores.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2022-02-01 19:32:35 +01:00
Adriaan Schmidt
41dbef918a Dockerfile: base containers on bullseye
Currently the containers are based on buster, but use buster-backports,
and (in case if the Isar container) even plain bullseye with pinning as
additional deb soure.
This changes that to use bullseye only.

Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-11-29 18:32:02 +01:00
Jan Kiszka
a753a14316 Dockerfile: Drop world-write permission from /kas folder
The github CI environment causes the copied /kas folder to become
world-writable. This is at least undesired, so fix up the permissions.
This is done the classic way, i.e. via a separate chmod, rather than
using the new and not yet widely available "COPY --chmod".

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-29 12:51:10 +02:00
Jan Kiszka
f38f287178 Dockerfile: Relocate and rework oe-git-proxy installation
Moving this file to /usr/local/bin broke user configurations that had
/usr/bin/ hard-coded, and it also broke inside the kas sandbox which
does not have /usr/local/bin in its PATH. So move things back to the
original location.

Do that by setting a link to the /kas/contrib folder, rather than
copying the file once more.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-29 12:25:25 +02:00
Jan Kiszka
da62e0bfbd kas-container: Add support for menu plugin
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.

Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.

Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-09 15:00:05 +02:00
Vivien Didelot
3484f1c208 Dockerfile: install g++-multilib
Some recipes, especially meta-qt5 packages, requires g++-multilib in
the host as stated in a few projects issues [1] [2].

[1] https://github.com/siemens/meta-iot2000/issues/18#issuecomment-293417336
[2] https://github.com/meta-qt5/meta-qt5/issues/348#issuecomment-677154661

Install the package in the kas container.

Signed-off-by: Vivien Didelot <vdidelot@pbsc.com>
[Jan: fix arm64 build]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-10-06 15:03:51 +02:00
Jan Kiszka
6a97879ad0 Dockerfile: Carry oe-git-proxy locally
This imports revision aa9b9dc9a9 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.

Reported-by: Raphael Lisicki <raphael.lisicki@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-30 21:56:04 +02:00
Ross Burton
ff99bf2226 Dockerfile: install lz4
OpenEmbedded Core is about to depend on lz4c, pzstd, and zstd being on
the host.  The Dockerfile was already installing zstd which provides
zstd/pzstd, so also install lz4 for lz4c.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-18 12:41:21 +02:00
Steffen Hieber
41d9f68ec3 Dockerfile: install git-lfs from buster-backports.
Debian Buster comes with version 2.7.1 of git-lfs which does not support
git+ssh yet, therefore we need a newer version of git-lfs for cloning a
repository which uses LFS.

Signed-off-by: Steffen Hieber <steffen.hieber@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-08-06 12:56:10 +02:00
Jon Mason
eb63b1873b Dockerfile: move sudo to standard Dockerfile
To expand the usability of the standard Docker image, move sudo
enablement from Docker.isar.

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-04-27 07:25:59 +02:00
Henning Schild
8234936902 Dockerfile: do not allow pip to fetch any deps anymore
We want all our deps from debian to make sure we do not get random
versions from pip. So we call pip install in such a way, and test that
kas can include all its deps later.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-24 20:54:06 +01:00
Jan Kiszka
5253d3c1d2 Dockerfile: Pull all Python dependencies from Debian
This provides a clearer and more stable environment.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-02-24 13:14:29 +01:00
Jan Kiszka
e8b6a4d89b Add support for multi-arch containers
Create the kas and kas-isar containers also for arm64. This allows using
them for arm-native builds, e.g. in AWS instances.

We need to drop a couple of packages from arm64 images which do not
exist on that arch. If the kas image is usable for Yocto builds is
untested, though. The Isar image works.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-03 18:39:35 +01:00
Jan Kiszka
2f4b89c77f Dockerfile: Fold two related ENV settings together
One layer less.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-02 15:53:06 +01:00
Jan Kiszka
cc159c471f Dockerfile: Make DEBIAN_FRONTEND non-persistent
Only use this setting during non-interactive build, not in the possibly
interactive image.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2021-01-02 15:50:17 +01:00
Mert Kirpici
348894158d Dockerfile: add zstd
It would be nice to have zstd frontend installed in the docker
environment. Since this is a relatively new and trending compression
algorithm, some packages might use it as part of their SRC's.

Also debian buster's tar has incorporated zstd support from upstream
GNU Tar version 1.31, starting from 1.30+dfsg-3. Therefore '.tar.zst'
extension files would be recognized and treated correctly by the tar
utility during unpack, etc.

Signed-off-by: Mert Kirpici <mert.kirpici@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-11-02 09:10:37 +01:00
Claudius Heine
bb303ced8c rename docker-entrypoint to container-entrypoint
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-09-29 08:32:57 +02:00
Claudius Heine
e61dfb7650 Dockefile: add awscli for s3 fetcher
bitbake provides a fetcher for Amazon S3 storage using the awscli
command. This patch adds the awscli package to the container.

Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2020-04-29 14:54:25 +02:00
Jan Kiszka
a5d0153e84 Dockerfile: Use Debian's gosu
The binary download from upstream dates back to jessie times.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-12-05 20:02:06 +01:00
Tim Orling
f0e0fa5da4 Dockerfile: add ps for testimage
HOSTTOOLS requires ps when running testimage.

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-17 13:04:38 +01:00
Jan Kiszka
52c6ff0a0f Dockerfile: Update package list according to Yocto 3.0 essential needs
Move those needs to top at this chance and refactor the rest. The net
additions are:

 - python3-pexpect
 - debianutils
 - iputils-ping
 - python3-git
 - python3-jinja2
 - libegl1-mesa
 - libsdl1.2-dev
 - pylint3
 - xterm

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-17 12:54:59 +01:00
Amol Dalavi
f39d3e48d1 Dockerfile: add package git-lfs
Kas can clone git repositories with LFS objects

Signed-off-by: Amol Dalavi <dalavi.amol@gmail.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-11-08 10:36:41 +01:00
Jan Kiszka
102eac9d9b Dockerfile: Add python3-wheel dependency
Silences error messages during kas installation step.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-09 16:51:08 +02:00
Henning Schild
63f8a28321 Docker: move on to buster
buster is the new stable, use it

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-10-01 16:57:20 +02:00
Henning Schild
363eca077c Revert "Dockerfile: Patch oe-git-proxy to truly disable proxying by default"
This reverts commit 7bd738a43c.

The bug has been fixed upstream, ne need to patch the script anymore.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-18 22:13:03 +02:00
Jan Kiszka
02fcc52007 Dockerfile: Add tmux and libncurses-dev
Needed for devshell and menuconfig targets.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-12 15:12:17 +02:00
Jan Kiszka
dbde2a75fc Revert "Docker: move on to buster"
This reverts commit 97942197bc.

This is only temporarily in order to release another 1.x version that is
based on stretch. Older Yocto versions, maybe up to 2.4 or 2.5, require
stretch due to incompatibilities with host tools such as git.

After that release, this commit will be restored, and we will roll out a
2.x series that shall use buster in its images.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-09-11 12:51:39 +02:00
Henning Schild
97942197bc Docker: move on to buster
buster is the new stable, use it

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-08 14:39:40 +02:00
Vijai Kumar K
6ed21ad62c Dockerfile: Add gnupg package
- gnupg is currently installed into kas-docker due to reprepro's
requirement. Hence, available only in isar version of kas-docker.
- Add gnupg explicitly in Dockerfile. It is needed for both isar and
yocto builds.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2019-08-08 14:39:21 +02:00
Jan Kiszka
7bd738a43c Dockerfile: Patch oe-git-proxy to truly disable proxying by default
Upstream oe-git-proxy has a bug [1] and currently does proxying
unconditionally, just being stopped in the absence of other proxy
settings. Patch in the fix until upstream picks it up.

[1] http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278496.html

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
2019-02-14 09:20:32 +01:00
Jan Kiszka
3bada55d30 Docker: Update to Debian stretch
Time to move forward, buster is almost in sight. We may break some older
Yocto versions this way, but they can always use older images.

Along this update, this migrates the locale setup towards the
recommendation on https://docs.docker.com/samples/library/debian/#locales.
Furthermore, DEBIAN_FRONTEND=noninteractive is moved to the top-level
Dockerfile - it's generic.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-07-19 09:25:34 +02:00
Daniel Sangorrin
9bd33ac221 Dockerfile: add iproute2
runqemu requires /sbin/ip

Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
2018-01-29 10:07:24 +01:00
Jan Kiszka
37cd2f42d5 Docker: Install xz-utils from stretch
The stretch version supports multi-thread compression, allowing to
accelerate e.g. artifact compression tasks. The newer version of xz also
seem to resolve mysterious compression failures that created too short
output images.

As we install the stretch source in the base image now, we can remove
that port from the isar image and align both regarding the setup.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2018-01-24 10:44:50 +01:00
Henning Schild
ec69bcb0c2 docker: add mercurial to our toolbox
With kas now supporting mercuial as well, add that tool to the docker
containers.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
2018-01-08 17:34:59 +01:00
Claudius Heine
bb36b907f0 Dockerfile: added less and vim
If you are using kas interactivly vim and less are very convenient
tools to have.

Signed-off-by: Claudius Heine <ch@denx.de>
2017-08-01 09:35:17 +02:00
Jan Kiszka
bd61564b58 docker: Set proxy for pip explicitly
Some versions/configurations of pip do not seem to pick up the proxy
settings from the environment. Help out via the command line. It's a nop
when https_proxy is unset.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 16:08:34 +02:00
Jan Kiszka
047a8b6878 docker: Add oe-git-proxy to base image
Useful to bypass nasty firewalls via http proxies while pulling from
git repos using the native protocol. In order to use oe-git-proxy,
define or clear NO_PROXY according to local needs in the CI script or
via --env on docker run.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
2017-07-20 16:08:34 +02:00
Mustafa Yücel
b1f343a3fd add yocto 2.1 in comment
Signed-off-by: Mustafa Yücel <mustafa.yuecel@siemens.com>
2017-06-29 09:05:02 +02:00