Setting up envoy as a front proxy on Docker with communication to other Docker containers

By Maximilian Batz | 2019-01-16

I have already existing containers which I want to use envoy as a proxy & https manager in front of. I am learning to use envoy, and am sharing some of my learnings here, as documentation is a bit confusing to start with.

My already existing container is called “taxgod” – it runs a Crystal application on port 3000 (http). I want to proxy it to port 80 on the host for http, and port 443 for https.

I will use a fake certificate for demonstration purposes. Certificate creation is outside the scope of this article.

The envoy container is going to be called penvoyage (picockpit envoy )

Building penvoyage

The penvoyage folder used for building contains

Dockerfile
envoy.yaml
example.crt (certificate)
example.key (key for the certificate)

Dockerfile

FROM envoyproxy/envoy:latest
COPY envoy.yaml /etc/envoy/envoy.yaml
COPY example.crt /etc/example-com.crt
COPY example.key /etc/example-com.key
#VOLUME /etc/envoy/envoy.yaml

EXPOSE 80/tcp
EXPOSE 443/tcp
#Educated guess that this is TCP
EXPOSE 9901/tcp

LABEL version=”0.2.0″ \
description=”penvoyage is envoy proxy for PiCockpit” \
maintainer=”<deleted>”

Take care with the ” characters, WordPress might mess them up!

I will go into the envoy.yaml in a bit more detail soon. Suffice it to say, that it is in this folder and will be copied into the newly created container image.

To build the container run (from the directory containing the penvoyage directory):

docker build -t jasmine/penvoyage:0.2.0 -t jasmine/penvoyage:latest penvoyage

Your output should terminate with the lines

Successfully tagged jasmine/penvoyage:0.2.0
Successfully tagged jasmine/penvoyage:latest

Create the docker network

docker network create -d bridge my-bridge-network

We will use this network to attach both containers to each other, so they see each other.

Containers can talk to each other using their container names, when they are on the same container network.

E.g. you can do

ping taxgod

where taxgod is the container name.

To add a container to a network after it has been run, do the following:

docker network connect my-bridge-network taxgod

In this case my-bridge-network is the network, and taxgod is the container to be connected to it.

To see how a container which is run is connected to a particular network, see below.

To see which containers are on a particular Docker network, run the following command:

docker network inspect my-bridge-network

Run the penvoyage container image

docker run –rm -it -p 80:80 -p 443:443 -p 9901:9901 –network=my-bridge-network jasmine/penvoyage

This will:

–rm remove the container after it’s shutdown
-it attach to the container (so you can see envoy’s output)
-p 80:80 map port 80 to port 80 inside the container (in the Docker file I have defined this to be the tcp port)
repeat for port 443 and port 9901
attach the container to my-bridge-network
build the container from the jasmine/penvoyage:latest image (:latest is implicit here)

What will this container do?

It will proxy connections to the host (in my case on IP 192.168.1.2) to the taxgod container, on port 3000.

It will act as a https proxy with the sample certificates, and proxy the connections to the same taxgod container, on port 3000.

envoy.yaml

Please note: yaml uses whitespace for structure, most likely WordPress will MESS this up. So use this as a reference, rather than a copy & paste thing!

static_resources:
   listeners:
   – address:
       socket_address:
         address: 0.0.0.0
         port_value: 80
     filter_chains:
     – filters:
       – name: envoy.http_connection_manager
         config:
           codec_type: auto
           stat_prefix: ingress_http
           route_config:
             name: local_route
             virtual_hosts:
             – name: local_service
               domains: [“*”]
               routes:
               – match: { prefix: “/” }
                 route: { cluster: target_taxgod }
           http_filters:
           – name: envoy.router
   – address:
       socket_address:
         address: 0.0.0.0
         port_value: 443
     filter_chains:
     – tls_context:
         common_tls_context:
           tls_certificates:
           – certificate_chain: { filename: “/etc/example-com.crt” }
             private_key: { filename: “/etc/example-com.key” }
       filters:
       – name: envoy.http_connection_manager
         config:
           stat_prefix: ingress_https
           route_config:
             virtual_hosts:
             – name: default
               domains: [“*”]
               routes:
               – match: { prefix: “/” }
                 route: { cluster: target_taxgod }
           http_filters:
           – name: envoy.router
   clusters:
   – name: target_taxgod
     connect_timeout: 0.25s
     type: strict_dns
     lb_policy: round_robin
     hosts:
     – socket_address:
         address: taxgod
         port_value: 3000
admin:
   access_log_path: “/tmp/envoy.log”
   address:
     socket_address:
       address: 0.0.0.0
       port_value: 9901

Short explanation of the envoy yaml file:

There are two main sections, static_resources and admin

admin is for the administration of envoy and outside the scope of this article. In a production environment you will probably need to secure it appropriately. DO NOT COPY & PASTE THAT PART!!!

static_resources defines a static configuration for envoy (envoy can also be set up using it’s API)

in listeners, we define which ports we want to listen on, and what should happen to the connections.

address – defines address and port to listen on

envoy.http_connection_manager – manages http connections, reads headers, and so on. it is a predefined filter
stat_prefix: defines a human readable name for logs / statistics
domains: [“*”] – matches all domains
route: { cluster: target_taxgod } – will route traffic to our cluster target_taxgod (will match for /, that is everything on the site)

the address and it’s sub configuration is repeated two times, one for port 80, and one for port 443. For port 443 we also add the tls_context and common_tls_context in the filter_chains to inject the certificates.

Note that it still is envoy.http_connection_manager, NOT https_connection_manager for port 443.

in clusters we define our endpoints we want to be able to connect to / proxy traffic to.

the cluster has a name
a connection timeout
lb_policy – the load balancing policy. There is only one host, so it will get all of the traffic (round_robin means that each host will get traffic in turn)
in the hosts, the socket_address is set as address taxgod – this is the Docker name of the container, and the port as port_value 3000

The port 3000 is the internal port of the container taxgod, it is not the port which you bind it to on the external host!

I hope this blog post helps people trying to set up a https:// setup in envoy.