Creating a Raspbian repository

By Maximilian Batz | 2019-04-03

If you want to host your own Raspbian repository, this article is for you.

A Raspbian repository consists of a special directory structure on a webserver. The files, including the packages, are all static – therefore this repository could also simply be hosted on an Amazon S3 instance, for example.

The sources.list entry in Raspbian

/etc/apt/sources.list

deb http://raspbian.raspberrypi.org/raspbian/ stretch main contrib non-free rpi

/etc/apt/sources.list.d/raspi.list

deb http://archive.raspberrypi.org/debian/ stretch main ui

These are the default entries for the package repositories on Raspbian.

The structure is as follows:

deb : source for binary packages
http://raspbian.raspberrypi.org/raspbian/ : URL of the repository
stretch : the “distribution” / release / codename (in some cases also possible to specify subdirectories nested deeper, e.g. stretch/updates)
main contrib non-free rpi : the components you are interested in from this repository, separated by a space

NB: apt starting from Debian / Raspbian Stretch includes support for https by default. Otherwise you would have to install the apt-transport-https package;

Here’s a webpage explaining why https is not necessary required for package installation.

A repository directory structure

Have a look at the http://raspbian.raspberrypi.org/ repository directory structure:

http://raspbian.raspberrypi.org/

This main directory contains several subdirectories, and notably the raspbian.public.key

Some of the subdirectories are for miscellaneous purposes, e.g. images – not necessary for a normal Debian / Raspbian Raspberry Pi repository.

Of interest here are the

mate
raspbian

subdirectories, and the already mentioned raspbian.public.key.

The key is a text file, generated by GnuPG. Here’s how it looks like:

etc.

This key is required to verify that you are indeed communicating with the right repository, which it claims to be. If you trust the repository, you add the key to your apt keyring (see for instructions how to add your own key and install packages from your own new repository below).

Then it can be used by apt to verify the signatures of the APT repository metadata. The packages themselves are usually not GPG signed / not being verified. Refer to the packagecloud article for details.

Repositories as subfolders

mate and raspbian are two different repositories (for the Raspberry Pi in this case).

Please note: aptly – the tool to publish your own repository explained below – will default to the webroot, and will show the dists folder in the webroot, but you can also specify a path to publish under.

We’ll look at the raspbian folder, which for our purposes is one such repository:

The dists folder contains the releases / distributions of Raspbian – codenamed in sync with the Debian releases. The current stable – as of 2.4.2019 – being Stretch.

They are named after Toy Story characters. The upcoming release is Buster.

“stable” is identical to “stretch” (currently). It will point to the most recent stable release.

This directory contains several files required for the functioning of apt tools. Notably the InRelease, Release and Release.gpg files.

InRelease: signed in-line
Release – goes along with Release.gpg (signature)

these files list the index files for the distribution and their hashes.

…

an example InRelease file, with the inline signature at the bottom.

It is like a kind of map for apt – where the Package listing files can be found, and in which formats ( .gz and .xz being compressed formats) they can be downloaded.

Here lives the metadata for the packages. The file Packages is human-readable (but very long).

notably this file contains a link to the “pool”, for the actual package.

Filename: pool/rpi/g/gst-omx1.0/gstreamer1.0-omx_1.0.0.1-0+rpi12+jessiepmg_armhf.deb

rpi is the name of the component, and the packages are further split up by first letter.

the pool directory is used to avoid duplication of packages.

Look at this informative site for more information about the Debian Repository Format:

https://wiki.debian.org/DebianRepository/Format

Using aptly – publishing your own repository, with your own packages

Aptly is a tool for easily setting up and publishing your own .deb Repository – for Debian and it’s siblings and cousins.

This includes Ubuntu, but of course also Raspbian.

So, if you want to publish a Raspbian repository, this is how you do it.

Set up a Docker container

I have created a Docker container, based on this git repository:

https://github.com/urpylka/docker-aptly

changes I have introduced:

FROM ubuntu:bionic
gnupg1
gpgv1
gpg command run explicitly as gpg1 (otherwise the public and private keys will be saved differently, and everything will not work as expected)

Therefore there will be some additional explaining below. On the whole urpylka’s script is a very convenient starting point, if you want to run on Docker.

Furthermore, I strongly recommend you to use docker-compose, instead of running those long docker run commands manually.

Here’s my dockerfile for your reference

# Copyright 2019 Maximilian Batz
# Copyright 2018 Artem B. Smirnov
# Copyright 2018 Jon Azpiazu
# Copyright 2016 Bryan J. Hong
#
# Licensed under the Apache License, Version 2.0 (the “License”);
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an “AS IS” BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

FROM ubuntu:bionic

LABEL maintainer=”<deleted to protect from spam>”

ENV DEBIAN_FRONTEND noninteractive

RUN apt-get -q update \
   && apt-get -y install \
   gnupg1 && apt-key adv –keyserver pool.sks-keyservers.net –recv-keys ED75B5A4483DA07C \
   && echo “deb http://repo.aptly.info/ squeeze main” >> /etc/apt/sources.list

# Update APT repository & install packages
RUN apt-get -q update \
   && apt-get -y install \
     aptly=1.3.0 \
     bzip2 \
     gpgv1 \
     graphviz \
     supervisor \
     nginx \
     wget \
     xz-utils \
     apt-utils \
   && apt-get clean \
   && rm -rf /var/lib/apt/lists/*

# Install Aptly Configuration
COPY assets/aptly.conf /etc/aptly.conf

# Install scripts
COPY assets/*.sh /opt/

# Install Nginx Config
RUN rm /etc/nginx/sites-enabled/*
COPY assets/supervisord.nginx.conf /etc/supervisor/conf.d/nginx.conf
RUN echo “daemon off;” >> /etc/nginx/nginx.conf

# Bind mount location
VOLUME [ “/opt/aptly” ]

# Execute Startup script when container starts
ENTRYPOINT [ “/opt/startup.sh” ]

Note, as of 2.4.2019, aptly 1.3.0 is still the most recent version.

Connecting to the aptly container

docker exec –it aptly /bin/bash

Your container being named aptly in this case, opens an interactive terminal with /bin/bash. All the other commands are executed inside the container.

Generating the GPG keys

The docker script mentioned above automatically creates GPG keys for you, by passing in your passphrase into a file in the container. The relevant script snippet is this:

#! /usr/bin/env bash

# Copyright 2016 Bryan J. Hong
# Licensed under the Apache License, Version 2.0

cat << EOF > /opt/gpg_batch
%echo Generating a GPG key, might take a while
Key-Type: RSA
Key-Length: 4096
Subkey-Type: ELG-E
Subkey-Length: 1024
Name-Real: ${FULL_NAME}
Name-Comment: Aptly Repo Signing
Name-Email: ${EMAIL_ADDRESS}
Expire-Date: 0
Passphrase: ${GPG_PASSWORD}
%pubring /opt/aptly/aptly.pub
%secring /opt/aptly/aptly.sec
%commit
%echo done
EOF

This is run if the keys do not exist, from the entrypoint of the container:

# If the repository GPG keypair doesn’t exist, create it.
if [[ ! -f /opt/aptly/aptly.sec ]] || [[ ! -f /opt/aptly/aptly.pub ]]; then
   echo “Generating new gpg keys”
   cp -a /dev/urandom /dev/random
   /opt/gpg_batch.sh
   # If your system doesn’t have a lot of entropy this may, take a long time
   # Google how-to create “artificial” entropy if this gets stuck
   gpg1 –batch –gen-key /opt/gpg_batch
else
   echo “No need to generate new gpg keys”
fi

Note my “improved” version (gpg1 instead of just gpg being specified) – this seems to be necessary for Ubuntu bionic beaver, as otherwise you will get gpg 2 as default.

Note: this file will contain your passphrase! Therefore I recommend to remove it after the initial setup has been completed.

Unfortunately, gpg1 does NOT seem to support the %ask-passphrase control. Therefore, if you want to do batch generation of the keys, the key password needs to be in the batch file.

Publish / export the GPG public key

mkdir -p /opt/aptly/public

gpg1 –export –armor > /opt/aptly/public/repository.picockpit.key

name the key appropriately. –armor is used to export the key in an ASCII format (as opposed to binary).

note these are two dashes, I hope WordPress doesn’t destroy them. Also, here gpg1 is used instead of just gpg

Create a new repository

aptly repo create -comment=”picockpit.com Repository” -component=”main” -distribution=”stretch” pcp-repository

Creates a new local repository, into which you can add packages. This repository is not published yet.

-comment : text to describe the local repository

-component: default component when publishing

-distribution: default distribution (release) when publishing

pcp-repository: this is a custom name I have given to this local repository, you can use your own here. Be sure to modify the other commands accordingly and use the name you choose here.

Add .deb packages to the repository

aptly repo add pcp-repository /opt/aptly/testpackage/

Here I am adding a folder of packages to the local repository I created in the step before. aptly will acknowledge the added packages, like this:

Create snapshot

While it is possible to publish a repo directly, it is strongly recommended to create a snapshot, and publish the snapshot.

aptly snapshot create pcp-snapshot from repo pcp-repository

pcp-snapshot being a custom name, and pcp-repository the name you gave to the repository further above.

Publish snapshot

aptly publish snapshot pcp-snapshot

During the publishing of the snapshot, you will be prompted for your gpg passphrase two times:

aptly will publish to the directory /opt/aptly/public, which Nginx or another webserver of your choice will serve from. No need for

Also refer to this documentation:

https://www.aptly.info/doc/aptly/publish/snapshot/

Switch published repository

If you have made a mistake, and would like to switch out the published repository to a different snapshot, you can use aptly publish switch:

aptly publish switch stretch raspbian pcp-snapshot

This will switch the distribution stretch, under the path raspbian, with the new snapshot pcp-snapshot.

https://www.aptly.info/doc/aptly/publish/switch/

Adding your repository to a system

Import the key

curl -L http://192.168.1.2:3200/repository.picockpit.key | sudo apt-key add –

The command fetches the key using curl, and then pipes it into apt-key. There is also an apt-key key fetching functionality, but it depends on a package which is not installed on Raspbian by default.

If you do not do this, apt will not be able to fetch packages from the new repository, and will throw errors!

Add the repository

echo “deb http://192.168.1.2:3200/raspbian stretch main” | sudo tee –append /etc/apt/sources.list.d/picockpit.list

The subdirectory under /etc/apt/sources.list.d is only writable by root. the –append ensures that the file is not overwritten, if it should exist already.

It would also be possible to create the file manually, and e.g. using nano, add the line

deb http://192.168.1.2:3200/raspbian stretch main

Install packages

sudo apt-get update

sudo apt-get install packagename

This screenshot shows that apt is reading the new repository as well.

installing a sample package.

Creating a package

I refer you to this excellent article how to create a Debian package.

Only be sure to change Architecture to armhf for the Raspberry Pi platform.

Need professional support?

We do all things Raspberry Pi for a living. Get in touch, if you need professional support with your setup.

We currently charge a flat 1500 € daily rate (+taxes). We’ll be glad to help you 🙂

Further readling

About packages:

https://www.debian.org/doc/debian-policy/ch-controlfields.html

Posted in Raspberry Pi Blog, Tips & tricks and tagged aptly, bionic, buster, deb, Debian, jessie, key, package repository, package sign, packages, Raspbian, repo, repository, signing, snapper, stretch