Download as PDF, PPTX
Uploaded byBret Fisher
Building Your Docker Tech Stack
Bret Fisher discusses the importance of building a cohesive tech stack for container clusters using Docker tools and avoiding cloud vendor lock-in. He outlines key strategies for small to medium-sized DevOps teams, emphasizing the use of Docker Swarm and maintaining a focus on quality in deployments. The document includes practical examples and options for app services, orchestration, monitoring, and next steps for scalable infrastructure.
Building Your Docker Tech Stack
- 1. BRET FISHER Docker Captain,DevOps Dude, Author of Docker Mastery Building Your Docker Tech Stack bretfisher.com/dockercon18 @bretfisher
- 2. People ask "Where'smy LAMP for container clusters?" Your solutions will be a "stack" of infrastructure tools Cloud-native container tools are new We need patterns and examples of full cluster stacks Problem: No Server Tool Lives In Isolation
- 3. Build examples offull-er/ish stacks on different tools Options for solo to medium-sized DevOps/Ops teams Use Docker Swarm latest stable as orchestrator Avoid cloud vendor lock-in* 4 Goals for Today * Lock-in: A service I can't swap out in my server stack
- 4. Limit "going productionDocker" project scope. Go Lean! Focus first on quality Dockerfiles Stay on your familiar host OS with 4.x Kernel Use base images of familiar OS (keep same pkg mgr) Swarm CE can be 1 or more nodes, use it everywhere Swarm EE is your "easy button" for security and ops Last Time On Bret's DockerCon Talk
- 5. Two Stacks, SameCore Docker CE Swarm Used for Dev/Test Heavy use of OSS/free Gluttony of choice for 3rd party Newest engine features Docker EE Swarm Used for Staging/Prod Heavy use of paid support Use Docker Solution Briefs Mature engine with hotfixes ++
- 6. Clusters are Complex Startsmall and simple, grow them as you grow
- 7. EE Platform Architecture PhysicalVirtualizationPublicCloud Platform Security Developer Services Registry Services Access Policies App Lifecycle Management Automation & Extensibility Networking Orchestration Storage Container Engine ENTERPRISE EDITION PLATFORM
- 8. CE Platform Architecture PublicCloud Overlay Swarm Container Engine COMMUNITY EDITION PLATFORM PhysicalVirtualization Platform Security
- 9. v Sci-Fi, Am IRight!
- 10. v Epic Battle Royale,on Swarm dogvs.cat VS
- 11. v Epic Battle Royale,on Swarm dogvs.cat Sci-Fi Sounds Edition
- 12. dogvs.cat App Services www.dogvs.catvote.dogvs.cat blog.dogvs.catresult.dogvs.cat (Stack Files)
- 13. dogvs.cat Swarm CE, (Han) SoloSysadmin or
- 14. Cloud agnostic, minimalinfra Apps auto-recover on node fail Incoming TLS Centralized logging Centralized monitoring Healthcheck all containers Infra Requirements of dogvs.cat Performance auto-scaling Self-healing nodes Support serverless functions Services highly available Han Solo Requirements Optional Requirements for Later
- 15. Simple Infrastructure, EasyDeployment "How can I deploy a multi-tier app on a few servers, with all the bells and whistles of orchestration with load balancing and auto recovery?" Han Solo, The Sysadmin
- 16. 3+ Droplets (Ubuntu16.04) Block Storage (Volumes) Load Balancer (incoming HTTP) Digital Ocean dogvs.cat Services Needed for High Availability
- 17. App Services www.dogvs.cat vote.dogvs.catblog.dogvs.catresult.dogvs.cat
- 18. App Services +L7 Proxy www.dogvs.cat vote.dogvs.cat blog.dogvs.catresult.dogvs.cat
- 19. App Services +L7 Proxy + Overlay www.dogvs.cat vote.dogvs.cat blog.dogvs.catresult.dogvs.cat
- 20. App Services +L7 Proxy + Ops www.dogvs.cat vote.dogvs.cat blog.dogvs.catresult.dogvs.cat
- 21. Cluster + ExternalLoad Balancer www.dogvs.cat vote.dogvs.cat blog.dogvs.catresult.dogvs.cat
- 22. Open Source Stack SwarmGUI Portainer Central Monitoring Prometheus + Grafana Central Logging Elastic ELK Layer 7 Proxy Traefik + Let's Encrypt Storage REX-Ray + Digital Ocean Volumes Networking Docker Swarm Overlay Orchestration Docker Swarm Runtime Docker CE HW / OS Docker Machine + Digital Ocean
- 23. Deploy Nodes: DockerMachine ./create-servers.sh ./enable-monitoring.sh ./create-swarm.sh
- 24. Deploy Storage: RexRayPlug-in docker stack deploy -c stack-rexray.yml rexray
- 25. Deploy Proxy Stack:Traefik with Let's Encrypt docker network create --driver overlay proxy docker stack deploy -c stack-proxy.yml proxy http://www.dogvs.cat:8080/dashboard/
- 26. Deploy Ops Stacks: Prometheus+ ELK + Portainer docker stack deploy -c stack-swarmprom.yml prom docker stack deploy -c stack-elk.yml elk docker stack deploy -c stack-portainer.yml portainer
- 27. Our Apps: Voting+ Ghost + Static Site docker stack deploy -c stack-ghost.yml ghost docker stack deploy -c stack-voting.yml vote docker stack deploy -c stack-menu.yml menu
- 28. Deploy Stacks: MaintenanceTasks docker stack deploy -c stack-prune.yml prune backups (in stack file with app)
- 29. Day Two Operations:Updates stack deploy ∞ micromanage update_config and healthcheck tune your limits and reservations
- 30. Security? host setup scanning:Docker Bench image scanning: Aqua Microscanner behavior monitoring: Sysdig Falco user namespaces
- 31. Next Steps more nodes?make 'em workers CI/CD stacks: gitlab, jenkins make redis, mysql, psql HA add app metrics to Prometheus swap Overlay for Weave Net swap ELK for Papertrail, etc. swap Prometheus for Sysdig, Datadog, Librato, etc. add socat proxy to Traefik
- 32.
- 33. Complex Infrastructure, HarderDeployment "How can I deploy many multi-tier app on a many servers, with all the b e l l s a n d w h i s t l e s o f H A orchestration, have load balancing at all levels, with failover and auto recovery?" Amazonian Team
- 34. CE Platform Architecture PhysicalVirtualizationPublicCloud Platform Security Networking Orchestration Container Engine COMMUNITY EDITION PLATFORM
- 35. EE Platform Architecture PhysicalVirtualizationPublicCloud Platform Security Developer Services Registry Services Access Policies App Lifecycle Management Automation & Extensibility Networking Orchestration Storage Container Engine ENTERPRISE EDITION PLATFORM
- 36. No More OneSize Fits All Docker for AWS Docker for Azure
- 37.
- 38.
- 39.
- 40. Docker EE onAWS Stack Swarm GUI Docker EE UCP Central Monitoring AWS Cloudwatch + Telegraph Central Logging AWS Cloudwatch Logs Registry Docker EE DTR Layer 7 Proxy HTTP Routing Mesh (Interlock+Nginx) Storage Docker Cloudstor EBS/EFS Networking Docker Swarm Overlay Orchestration Docker Swarm Runtime Docker EE HW / OS Terraform + Ansible + AWS
- 41. Deploy Nodes: Terraform+ Ansible terraform apply ansible-playbook -i inventory install.yml
- 42. Advantages more flexible deploymenttools more SecOps tools ops tools are fully HA ops tools are team-ready deploy to K8s just as easy
- 43. Deploy Apps docker stackdeploy -c stack-ghost.yml ghost docker stack deploy -c stack-voting.yml vote docker stack deploy -c stack-menu.yml menu
- 44. Next Steps All thethings in Swarm CE monitoring via CloudWatch and Telegraph logging via CloudWatch Logs
- 45. Summary Infrastructure as code,make everything repeatable No "special" nodes, use remote management Grow as you go, assume you'll resize Look for compose files of popular tools to make stacks Don't throw out the good in search of the perfect
- 46. I'd like to thankthe internet
- 47.
- 48. PLEASE VOTE SOI WIN ALL THE DOCKERS FRIDAY! Thanks! 🤗 bretfisher.com/dockercon18 "Building Your Docker Tech Stack"