Clustering docker: A guide, with examples....
Basic Manual Cluster
Bash Code For Basic Manual Cluster
Here’s what you’ll need to run docker containers in production in a manual fashion over a cluster of hosts:
Docker Daemon (get.docker.com | bash)
Downloads Docker images
Builds docker images
Runs docker containers
Weave Networking
Gives each container its own internal ip address ex: 10...*
If needed, provides each host with an internatl IP address as well to make connecting system services to those containers easy
Caddy Web server
Automated HTTPS certificates
The "Caddyfile" can be configured to forward a container address like 10.32.0.1 to a web address with a public IP address like www.jimmysfriedchicken.com.
Now, the above platform is manual. It doesn’t automatically assign addresses based on container labels or other factors, and it won’t change the host that containers run on based on system resource availablity. If you want that, you need to know that there are a number of incomplete attempts at achieving such containerization nirvana, such as:
Docker Clustering Systems
Images from: http://www.nextplatform.com/2016/03/29/rancher-rides-herd-containers-borg-style/ (Thanks!)
Docker Swarm (not built into the docker binary)
Docker Swarm Mode 1.12 (built into the docker binary post 1.12)
Kubernetes
Nomad + Consul + Registrator
Rancher
Mesos (to some degree: Mesos is a cluster manager first and foremost, and it is able to use Docker)
Mesosphere (again, to some degree: Mesosphere is Mesos’s Open Source/Commercially Supported big brother that integrates more PaaS like features.)
Kontina
Flynn
Fastest (To get running) Automated Cluster
Docker Swarm Mode
Docker Daemon (get.docker.com | bash)
Downloads Docker images
Builds docker images
Runs docker containers
Swarm Mode
Container networking (won’t need weave)
Host Networking
Scheduling
Orchestration (managing computational load)
But that leaves a significant gap, or depending on your perspective as to what constitutes a complete platform, several signficiant gaps:
Virtual Hosting / Reverse Proxying (though technically the RP/LB is covered, then again, it’s not covered….)
Storage management
Monitoring Service Metrics
Logging Container Logs
But suppose that you could forgeo these items, with the exception of Virtual Hosting. Then, your best bet by far would be to set up Docker-flow-proxy, which mercifully has full support ofr swarm mode since its latest release. So basically docker-flow-proxy is an automated HAPROXY setup that will provide an API to you to call when you launch a container, like this:
From https://github.com/vfarcic/docker-flow-proxy:
Reconfigure
Reconfigures the proxy using information stored in Consul
The following query arguments can be used to send as a reconfigure request to Docker Flow: Proxy. They should be added to the base address :/v1/docker-flow-proxy/reconfigure.
| Query | Description | Required | Default | Example |
| consulTemplateBePath | The path to the Consul Template representing a snippet of the backend configuration. If specified, the proxy template will be loaded from the specified file. | /consul_templates/tmpl/go-demo-be.tmpl | ||
| consulTemplateFePath | The path to the Consul Template representing a snippet of the frontend configuration. If specified, the proxy template will be loaded from the specified file. | /consul_templates/tmpl/go-demo-fe.tmpl | ||
| distribute | Whether to distribute a request to all the instances of the proxy. Used only in the swarm mode. | No | false | true |
| pathType | The ACL derivative. Defaults to path_beg. SeeHAProxy path for more info. | No | path_beg | |
| port | The internal port of a service that should be reconfigured. The port is used only in the swarmmode | Only in swarmmode | ||
| serviceDomain | The domain of the service. If specified, the proxy will allow access only to requests coming to that domain. | No | ecme.com | |
| serviceName | The name of the service. It must match the name stored in Consul. | Yes | books-ms | |
| servicePath | The URL path of the service. Multiple values should be separated by a comma (,). | Yes (unless consulTemplatePath is present) | /api/v1/books | |
| skipCheck | Whether to skip adding proxy checks. This option is used only in the defaultmode. | No | false | true |
Remove
Removes a service from the proxy
The following query arguments can be used to send a remove request to Docker Flow: Proxy. They should be added to the base address [PROXY_IP]:[PROXY_PORT]/v1/docker-flow-proxy/remove.
| Query | Description | Required | Default | Example |
| serviceName | The name of the service. It must match the name stored in Consul | Yes | go-demo | |
| distribute | Whether to distribute a request to all the instances of the proxy. Used only in the swarm mode. | No | false | true |
So This is probably the fastest path available today to a functional docker platform. Just use docker swarm mode and then add in docker-flow-proxy. There’s a A LOT that this doesn’t do but that’s kind of the point. If you want to know more about more advanced implementations, just drop me a line at faddat@gmail.com preferably using Google Hangouts.