Managing 40+ Servers in a Data Center at a Medical University
Running in Production
English - July 12, 2021 04:00 - 53 minutes - 48.9 MB - ★★★★★ - 8 ratingsTechnology News Tech News deployment elixir flask hosting node rubyonrails phoenix production python techstacks Homepage Download Apple Podcasts Google Podcasts Overcast Castro Pocket Casts RSS feed
Maciej Delmanowski goes over managing 40+ servers and 200+ VMs with Ansible. It's been an ongoing project he's worked on since 2007.
In this episode of Running in Production, Maciej Delmanowski talks about building
out a 40+ server / 200+ VM data center with Ansible. It’s hosted on premises,
he’s been working on it all since 2007 and started using Ansible in 2013.
Maciej talks about automating everything with Ansible, being a sysadmin for
over a decade, how he picked Debian, splitting up a project into 120+ git repos
and then back to 1, writing 60k+ lines of YAML, using Linux Containers,
maintaining an open source project and more.
Topics Include
2:35 – There’s about 40 physical servers and 200+ virtual machines / containers
4:50 – Transitioning to using Ansible over time
7:40 – A 10+ year friendship stemming from Debian and open source
8:23 – How Ansible’s role and inventory abstractions help manage a lot of VMs
10:35 – How DebOps as a name came into existence and its philosophy on being stable
14:24 – Motivation for choosing and staying with Debian
15:31 – Figuring out what new Ansible roles and playbooks to work on
19:14 – Going from a mono repo to 120+ repos and then back to a mono repo
24:48 – 67,000+ lines of YAML and 40,000+ lines of documentation
26:28 – Setting up a brand new server with Ansible and DebOps hands free in 15 minutes
28:29 – Automatically generating random passwords for services
31:22 – Not having to deal with HIPAA compliance laws and handling student emails
34:01 – Let’s Encrypt is being used on specific publicly accessible servers
34:26 – Breaking down the process of creating a new role from scratch
36:39 – Using Linux Containers in development and rolling things out to production
41:41 – Using dnsmasq in development for fully qualified domain names
43:07 – Dealing with backing everything up
46:02 – Being a maintainer of an open source project that’s extracted from work
49:19 – Best tips? Have 1 role for 1 service and find ways to connect them
51:26 – Check out DebOps, it’s on GitHub, #debops on Libera and Maciej is on Twitter
Links
📄 References
https://en.wikipedia.org/wiki/Orson_Scott_Card_bibliography
https://github.com/nickjj/ansigenome
https://github.com/nickjj/rolespec
https://linuxcontainers.org/
https://en.wikipedia.org/wiki/Dnsmasq
⚙️ Tech Stack
ansible →
python →
bash →
debian →
lets-encrypt →
open-source →
🛠 Libraries Used
https://www.veeam.com/virtual-machine-backup-solution-free.html
https://rsnapshot.org/
https://libera.chat/ (we’re in #debops)
Support the Show
This episode does not have a sponsor and this podcast is a labor of love. If
you want to support the show, the best way to do it is to purchase one of my
courses or suggest one to a friend.
Dive into Docker is a video course that takes you from not knowing what Docker is
to being able to confidently use Docker and Docker Compose for your own apps.
Long gone are the days of "but it works on my machine!". A bunch of follow
along labs are included.
Build a SAAS App with Flask is a video course where we build a real
world SAAS app that accepts payments, has a custom admin, includes high test
coverage and goes over how to implement and apply 50+ common web app features.
There's over 20+ hours of video.