Dockerfile and container-entrypoint were missing that header.
Furthermore, the leading comment in Dockerfile got out-of-date, and we
should rather add section marker for the kas and kas-isar targets.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Already create the builder user/group during container image build and
only align the IDs in the entrypoint if started with a non-zero USER_ID.
The primary gain is code simplification because this removes some
dynamics from the entrypoint.
As this refactoring avoids that gitlab-ci runners start the container as
root, it was also supposed to resolve the mismatch between the owner of
the checked-out repo and builder user. Unfortunately, this does not work
yet, and the reason is still unclear.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Leave the isar-only commands commented-out in the container-entrypoint
and simply remove the comment when building kas-isar. This is simpler,
more readable and also more robust against changes of the entrypoint
file.
While at it, avoid a separate layer for modifying container-entrypoint.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Was lost in the refactoring of a6b18abc8a.
Signed-off-by: Hannah Kiekens <hannah.kiekens@mind.be>
[Jan: refer to the causing commit]
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch prevents deployment of the pip cache folder that is only
needed during installation. Which is also done with the apt,
where at the end of the install operation we delete the
cache and all temporary folders.
The reducion of the image size is insignificant:
| builder@1373d0b95f44:/build$ sudo du -sh /root/.cache/pip
| 1.6M /root/.cache/pip
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The install package list are actually taking 1:1 from their documentation,
so there some packages that can already installed by other downstream layers.
This will not change any image sizes on all the layers in use.
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
While kas and kas-isar surely have a common ground like the kas tool
itself, kas-isar does not need all the toolchain packages that are
essential Yocto dependencies. Splitting up the images earlier allows to
shrink kas-isar by almost 400 MB.
Use this chance to model both images as different build stages of the
same Dockerfile. That is simpler than creating a temporary "kas-base"
image via a separate Dockerfile.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Prior to the official first release of kas, this was once introduced to
fulfill kernel build dependencies. That is not longer be needed with
modern Yocto versions (if it ever actually was).
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Some emulators use telnet to provide console access to the system.
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
At my debian11/amd64 machine, the size of newer image is 889MB, while the
older one is 908MB.
Refs: #81
Signed-off-by: Zhibin Dong <zhibin.dong@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This patch adds the pigz package to enable parallel compression
and decompression support for the sstate artifacts.
This significantly speeds up compression of large artifacts on
build systems with many cores.
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Currently the containers are based on buster, but use buster-backports,
and (in case if the Isar container) even plain bullseye with pinning as
additional deb soure.
This changes that to use bullseye only.
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The github CI environment causes the copied /kas folder to become
world-writable. This is at least undesired, so fix up the permissions.
This is done the classic way, i.e. via a separate chmod, rather than
using the new and not yet widely available "COPY --chmod".
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Moving this file to /usr/local/bin broke user configurations that had
/usr/bin/ hard-coded, and it also broke inside the kas sandbox which
does not have /usr/local/bin in its PATH. So move things back to the
original location.
Do that by setting a link to the /kas/contrib folder, rather than
copying the file once more.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
For this script, this is almost straightforward - except that we need to
extract the static KAS_BUILD_SYSTEM config setting from the selected
Kconfig file so that the correct container image and mode is chosen.
Two new dependencies need to be added to the container image. While
python3-newt can come from Debian, kconfiglib only exists as Python
package. To make sure we are not pulling any other packages via pip,
install kconfiglib upfront. It has no own dependencies, thus can use
--no-deps as well.
Finally, the container-entrypoint needs to be updated to make it aware
of the new plugin.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This imports revision aa9b9dc9a9 from
https://git.yoctoproject.org/git/poky to avoid fetching it - and having
to add the missing content validation to prevent supply-chain attacks.
Reported-by: Raphael Lisicki <raphael.lisicki@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
OpenEmbedded Core is about to depend on lz4c, pzstd, and zstd being on
the host. The Dockerfile was already installing zstd which provides
zstd/pzstd, so also install lz4 for lz4c.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Debian Buster comes with version 2.7.1 of git-lfs which does not support
git+ssh yet, therefore we need a newer version of git-lfs for cloning a
repository which uses LFS.
Signed-off-by: Steffen Hieber <steffen.hieber@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
To expand the usability of the standard Docker image, move sudo
enablement from Docker.isar.
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
We want all our deps from debian to make sure we do not get random
versions from pip. So we call pip install in such a way, and test that
kas can include all its deps later.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Create the kas and kas-isar containers also for arm64. This allows using
them for arm-native builds, e.g. in AWS instances.
We need to drop a couple of packages from arm64 images which do not
exist on that arch. If the kas image is usable for Yocto builds is
untested, though. The Isar image works.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
It would be nice to have zstd frontend installed in the docker
environment. Since this is a relatively new and trending compression
algorithm, some packages might use it as part of their SRC's.
Also debian buster's tar has incorporated zstd support from upstream
GNU Tar version 1.31, starting from 1.30+dfsg-3. Therefore '.tar.zst'
extension files would be recognized and treated correctly by the tar
utility during unpack, etc.
Signed-off-by: Mert Kirpici <mert.kirpici@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
bitbake provides a fetcher for Amazon S3 storage using the awscli
command. This patch adds the awscli package to the container.
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
HOSTTOOLS requires ps when running testimage.
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Move those needs to top at this chance and refactor the rest. The net
additions are:
- python3-pexpect
- debianutils
- iputils-ping
- python3-git
- python3-jinja2
- libegl1-mesa
- libsdl1.2-dev
- pylint3
- xterm
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This reverts commit 7bd738a43c.
The bug has been fixed upstream, ne need to patch the script anymore.
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
This reverts commit 97942197bc.
This is only temporarily in order to release another 1.x version that is
based on stretch. Older Yocto versions, maybe up to 2.4 or 2.5, require
stretch due to incompatibilities with host tools such as git.
After that release, this commit will be restored, and we will roll out a
2.x series that shall use buster in its images.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
- gnupg is currently installed into kas-docker due to reprepro's
requirement. Hence, available only in isar version of kas-docker.
- Add gnupg explicitly in Dockerfile. It is needed for both isar and
yocto builds.
Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Upstream oe-git-proxy has a bug [1] and currently does proxying
unconditionally, just being stopped in the absence of other proxy
settings. Patch in the fix until upstream picks it up.
[1] http://lists.openembedded.org/pipermail/openembedded-core/2019-January/278496.html
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Time to move forward, buster is almost in sight. We may break some older
Yocto versions this way, but they can always use older images.
Along this update, this migrates the locale setup towards the
recommendation on https://docs.docker.com/samples/library/debian/#locales.
Furthermore, DEBIAN_FRONTEND=noninteractive is moved to the top-level
Dockerfile - it's generic.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
The stretch version supports multi-thread compression, allowing to
accelerate e.g. artifact compression tasks. The newer version of xz also
seem to resolve mysterious compression failures that created too short
output images.
As we install the stretch source in the base image now, we can remove
that port from the isar image and align both regarding the setup.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Some versions/configurations of pip do not seem to pick up the proxy
settings from the environment. Help out via the command line. It's a nop
when https_proxy is unset.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Useful to bypass nasty firewalls via http proxies while pulling from
git repos using the native protocol. In order to use oe-git-proxy,
define or clear NO_PROXY according to local needs in the CI script or
via --env on docker run.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
