Managing Data in Docker

Data persistence and sharing in Docker are primarily handled through Data Volumes and Data Volume Containers.

Data Volumes

A Data Volume is a special directory within a container that bypasses the Union File System. It allows direct mapping of a host directory into the container, enabling real-time synchronization. Modifications made in the volume do not affect the base image.

Mount the host directory /srv/webapp to the container path /app_data. Host paths must be absolute; Docker creates them if missing.

mkdir -p /srv/webapp
docker run -itd --name web_primary -v /srv/webapp:/app_data centos:7 bash
docker exec -it web_primary bash
echo 'initial data' > /app_data/startup.log

Verify on the host:

cat /srv/webapp/startup.log

Append from host and verify in container:

echo 'host update' >> /srv/webapp/startup.log
docker exec -it web_primary cat /app_data/startup.log

Sharing between containers via host mount:

docker run -itd --name web_secondary -v /srv/webapp:/shared_data centos:7 bash

Mounting a single file:

docker run -itd --name file_container -v /srv/webapp/config.yaml:/app/config.yaml centos:7 bash

Read-only mounts restrict container write access:

docker run -itd --name read_only_container -v /srv/webapp/config.yaml:/app/config.yaml:ro centos:7 bash

Data Volume Containers

Data Volume Containers facilitate data sharing directly between containers without directly involving the host path in subsequent container creations. A dedicated container serves as the volume mount point.

docker run -itd --name data_hub -v /storage_one -v /storage_two centos:7 bash
docker run -itd --name app_worker_1 --volumes-from data_hub centos:7 bash
docker run -itd --name app_worker_2 --volumes-from data_hub centos:7 bash

docker exec -it data_hub bash
echo 'shared file 1' > /storage_one/doc.txt
echo 'shared file 2' > /storage_two/note.txt

Verify from worker containers:

docker exec -it app_worker_1 cat /storage_one/doc.txt
docker exec -it app_worker_2 cat /storage_two/note.txt

Container Interconnection

Container IP addresses change upon recreation. Container linking establishes a network tunnel using container names, allowing the target container to reach the source container by name or alias.

docker run -itd --name db_service centos:7 bash
docker run -itd --name web_client --link db_service:database_alias centos:7 bash
docker exec -it web_client ping database_alias

Constructing Docker Images

Images can be created from existing containers, imported from OS templates, or built using Dockerfiles.

Creating from Existing Containers

Modify a running container and commit its state as a new image.

docker run -itd --name base_node centos:7 bash
docker cp custom.repo base_node:/etc/yum.repos.d/
docker exec -it base_node bash -c "yum install -y httpd"

docker commit -m "Added httpd" -a "Admin" base_node custom_httpd:v1
docker run -itd --name running_httpd -p 8080:80 custom_httpd:v1 bash -c "httpd -D FOREGROUND"

Importing from Local Templates

Use docker import to load an OS template file (e.g., from OpenVZ) into an image.

wget http://openvz.org/Download/template/precreated/ubuntu-20.04-x86_64.tar.gz
docker import ubuntu-20.04-x86_64.tar.gz -- custom_ubuntu:20.04
docker run -itd custom_ubuntu:20.04 bash

Creating via Dockerfile

Image Loading and UnionFS

Docker images utilize a Union File System (UnionFS), a layered, lightweight file system. Changes are applied as successive layers. Images consist of bootfs (bootloader and kernel) and rootfs (standard OS directories like /dev, /proc).

When the kernel loads into memory, bootfs is unmounted. rootfs represents the OS distribution. Minimal OS images (e.g., a 200MB CentOS image) are small because they share the host's kernel (bootfs) and only ship the necessary userland tools (rootfs).

Dockerfile Syntax and Layer Caching

Dockerfile is a script containing instructions to automate image building. Each instruction creates a new layer. Layers are cached; if an instruction changes, its cache and all subsequent caches are invalidated. Layers are immutable; deleting a file in a later layer merely hides it in the final container.

Core Instructions

• FROM : Sets the base image. Must be the first instrcution.
• LABEL =: Adds metadata (replaces deprecated MAINTAINER).
• RUN : Executes commands during build.
• ENTRYPOINT ["exec", "param"]: Configures the container's main executable. Overrides are possible via docker run --entrypoint.
• CMD ["exec", "param"]: Provides default execution parameters. Overridden by docker run arguments or ENTRYPOINT. Only the last CMD takes effect.
• EXPOSE : Documents intended network ports.
• ENV =: Sets environment variables.
• ADD : Copies files from the host or URLs. Automatically extracts local tar archives. URL download and auto-extraction are mutually exclusive.
• COPY : Strictly copies local files to the destination without extraction or URL support.
• VOLUME ["/path"]: Creates a mount point for anonymous volumes, preventing data loss if the container is removed without explicit host mounts.
• USER : Sets the runtime user.
• WORKDIR : Sets the working directory for subsequent instructions.
• ONBUILD : Triggers execution when the image is used as a base image for another build.
• ARG =: Defines build-time variables. ENV overrides ARG with the same name.

Case 1: Package Manager Installation (Apache on Ubuntu)

mkdir -p /build/apache
cd /build/apache
vim Dockerfile

FROM ubuntu:22.04

LABEL maintainer="devops"

COPY sources.list /etc/apt/sources.list

RUN apt-get update && apt-get install -y apache2

EXPOSE 80

CMD ["apachectl", "-D", "FOREGROUND"]

Build and run:

docker build -t apache_server:v1 .
docker run -itd -p 9090:80 apache_server:v1

Customizing the web root:

mkdir -p /build/apache/html
docker cp <container_id>:/var/www/html/index.html /build/apache/html/
docker rm -f <container_id>
docker run -itd -p 9091:80 -v /build/apache/html:/var/www/html apache_server:v1
echo 'Custom Apache Page' > /build/apache/html/index.html

Case 2: Source Compilation (Nginx)

mkdir -p /build/nginx_src
cd /build/nginx_src
# Download nginx-1.25.0.tar.gz here

FROM centos:7

LABEL maintainer="devops"

ADD nginx-1.25.0.tar.gz /usr/local/src/
COPY CentOS-Base.repo /etc/yum.repos.d/

RUN yum -y install pcre-devel zlib-devel openssl-devel gcc gcc-c++ make && \
    useradd -M -s /sbin/nologin nginx && \
    cd /usr/local/src/nginx-1.25.0 && \
    ./configure --prefix=/usr/local/nginx --user=nginx --group=nginx --with-http_stub_status_module && \
    make && \
    make install

EXPOSE 80

CMD ["/usr/local/nginx/sbin/nginx", "-g", "daemon off;"]

Build and run with a mounted content directory:

docker build -t nginx_compiled:v1 .
mkdir -p /build/nginx_src/site
cp -r /default/nginx/html /build/nginx_src/site/
docker run -itd --name web_gateway -p 8888:80 -v /build/nginx_src/site:/usr/local/nginx/html nginx_compiled:v1
echo 'Source Compiled Nginx' > /build/nginx_src/site/index.html