Did you ever work with a docker-compose.yml that defined multiple services based on the same docker image and with only decent differences (like a different command per service)? I did. And I learned that there is no need to duplicate the code for each service thanks to extension fields. Let us have a look at how that works by example.

In my current search engine project I am working with a docker-compose.yml (used for a docker stack working with docker swarm mode) that defines some crawler services (A crawler is a tool that scans and saves the content of one or multiple web pages to make them searchable later). Now I wanted to create two crawler services. They should behave exactly the same but with two exceptions: Each crawler should crawl a different web page and each crawler should run on a specific host. Therefore I defined two crawler services like the following (I left out secrets/volumes to have a focus on the services):

version: "3.6"
services:
  crawler-one:
    image: docker.gotcha-app.de/gotcha/crawler
    command: -n CrawlerOne - u http://www.google.com
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == worker
          - node.hostname == crawler-two
    volumes:
     - "crawler-volume:/root/nutch/volume"
    secrets:
     - index-username
     - index-password
     - crawler-api-username
     - crawler-api-password
    networks:
     - gotcha-net
  crawler-two:
    image: docker.gotcha-app.de/gotcha/crawler
    command: -n CrawlerTwo - u http://www.bing.com
    deploy:
      mode: replicated
      replicas: 1
      placement:
        constraints:
          - node.role == worker
          - node.hostname == crawler-two
    volumes:
     - "crawler-volume:/root/nutch/volume"
    secrets:
     - index-username
     - index-password
     - crawler-api-username
     - crawler-api-password
    networks:
     - gotcha-net

Pay attention to the fact that both service definitions only differ in terms of the used command and the constraints (marked bold). The command differs because each crawler should crawl a different web page and the placement constraint says that crawler-one should run on the swarm worker host (node) called crawler-one, and crawler-two should run on node crawler-two.

This is a lot of duplicated code and only few differences.. what is the trick to reuse the code that is equal in both definitions?

The trick is called extension fields and it is described in the docker docs.

Let us have a look straight at how the docker-compose.yml file seems when we use extension fields here to remove the duplicated code:

version: "3.6"
x-defaultcrawler:
  &default-crawler
  image: docker.gotcha-app.de/gotcha/crawler
  deploy:
    mode: replicated
    replicas: 1
    placement:
      constraints:
        - node.role == worker
  volumes:
   - "crawler-volume:/root/nutch/volume"
  secrets:
   - index-username
   - index-password
   - crawler-api-username
   - crawler-api-password
  networks:
   - gotcha-net

services:
  crawler-one:
    <<: *default-crawler
    command: -n CrawlerOne -u http://www.google.com/
    deploy:
      placement:
        constraints:
          - node.hostname == crawler-one
  crawler-two:
    <<: *default-crawler
    command: -n CrawlerTwo -u http://www.bing.com/
    deploy:
      placement:
        constraints:
          - node.hostname == crawler-two

In the first part we are defining a default-Crawler fragment. The first line with the prepended x- says that we are defining a reusable fragment here. The second line with the prepended & defines the name we will use to import that fragment where ever we want. And then follows all the code, that was exactly the same in the crawler-one and the crawler-two service.

After that fragment we define our services crawler-one and crawler-two. The first line after the service name (<<: *default-crawler) says „For that service take the fragment with the name default-crawler, and after that add everything that follows and merge it into the fragment“.

So using the docker-compose.yml with the extension fields will behave exactly like the one above. But now we no longer have that ugly code duplication containing elements that are completely equal in both services.