Introduction I started reading the Miniflux project’s code on September 27th. Reading a mature open-source project is like reading a book that suits you – the pace is slow but rewarding. As I’m nearing the end of reading this project’s code, I’m putting together this blog post as a record and to share the experience. Before we begin, a brief introduction to Miniflux. Miniflux is an open-source RSS reader service, similar to Tiny Tiny RSS. If you’ve used RSS reader services before, this should feel familiar – deploying Miniflux is essentially having your own Feedly service. The project is a SaaS application built with Go and Vanilla JS (native JavaScript APIs without any frameworks), advocating for minimal external dependencies, simplicity, maintainability, and extensibility. As a result, the project contains many foundational implementations (details you’d typically only encounter when developing a web framework itself – when building projects based on frameworks like Django/Rails, these features come built-in and ready to use). ...

Introduction Why study the implementation of Python’s enum module? Python is an object-oriented programming (OOP) language, and the enum module in the standard library involves many OOP concepts, such as mixins, duck typing, metaclasses, and magic methods. Reading the enum module’s source code is an effective way to understand object-oriented programming. This article is based on the CPython 3.11 branch, corresponding to Python version 3.11.4. Topics covered: Enum classes Duck typing Magic methods Protocols Decorators Descriptor classes Metaclasses Concepts Enum Classes The members of an enum class are special instances of the enum class itself. In other words, the attribute names remain unchanged, but their values become special instances of the enum class. In an enum class, the originally defined attribute names and values are stored in the name and value attributes of these special instances. ...

Preface This article is a set of notes from reading the code used by the Arch Linux team to deploy, manage, and integrate their Identity and Access Management (IAM) service. It covers: How to deploy and manage a Keycloak service: Ansible How to manage users and permissions in Keycloak: Terraform How to integrate a web application with Keycloak: OAuth2.0, OIDC, Python, Authlib How to implement permission management based on information from Keycloak: decorators and enum classes IAM Service What Is It For Let’s first consider what problems arise without an IAM service. ...

Introduction I recently deployed a backup system for my local infrastructure, drawing inspiration from the Ansible code in the Arch Linux infrastructure repository. This post documents the process and my thoughts along the way, covering the considerations before implementation, how it was implemented, and how the backup system itself is monitored. Backup System Solution I chose borg as the backup solution. borg is a backup program that supports deduplication, compression, and encryption. ...

Introduction Over the past year at my previous company (a telecom provider), I spent a great deal of time automating infrastructure. The primary approach was codifying middleware deployments with Ansible – replacing what had been manual processes – to enable efficient deployment and ongoing operations. However, cloud resources such as virtual machines and domain names were still provisioned manually through web consoles. When setting up new data centers, nearly ten thousand servers were all created by hand through click-ops. ...

Introduction Both Arch Linux and Rust have open-sourced their infrastructure code. So how do they keep sensitive information secure while making their infrastructure code public? Their implementations can be found here: Arch Linux infrastructure rust simpleinfra Arch Linux uses ansible-vault and GnuPG to manage sensitive information, including Terraform access keys, secret keys, and tokens. The Rust Team uses AWS STS for permission management and AWS SSM to manage sensitive information in Ansible and Terraform. Terraform state is stored on S3. ...

My infrastructure environment is primarily supported by a workstation running 24/7 on my home LAN. The infrastructure I use daily – such as GitLab, GitLab Runner, Jira, and even the blog you are reading right now – all runs on this workstation. The workstation runs Arch Linux with libvirt virtualization, where different services run in isolated virtual machines. Below is a brief introduction to how I use this virtualization setup. ...

Introduction In the past, manually deploying a single highly available Apollo cluster and verifying it typically took 1 to 2 business days. Now, 24 clusters can be deployed in a single business day (7 hours) – each taking only 18 minutes – and every cluster is production-ready. How was this achieved? TL;DR Unified development environment based on VSCode Matching test environment based on libvirtd Encrypted communication virtual LAN between private GitLab and production server clusters via WireGuard Multi-branch development workflow based on GitLab Automated operations code synchronization based on GitLab CI Unified Development Environment & Reproducible Test Environment In our office area data center, there is a Linux physical server that I use as the unified development environment. All Ansible code is written by connecting to this physical server via VSCode Remote Development. Below are the two common access methods: ...

This is an expansion of the previous post. Problems that sustainable ops solves: 1. Reusability of operations code. Once operations are codified, the same code can be reused across different servers. Simply copy the operations code, run it, and you get two identical service environments. 2. Composability of operations code. Codified operations can be stacked and composed. For example, deploying Kafka requires installing ZooKeeper first, and deploying a Codis cluster also requires ZooKeeper. Once ZooKeeper operations are codified, we can directly invoke the ZooKeeper operations code from both the Kafka and Codis operations code – like building blocks – without reimplementing ZooKeeper operations. ...

I have recently been reading the code in the archlinux infrastructure repository. This is the operations repository of the Arch Linux operations team, where they have achieved sustainable operations through Infrastructure as Code. What sustainable operations means: As time passes and operations personnel change, operational procedures are not lost and servers do not become black boxes; Whether managing 1 server or 10,000, long-term operability, maintainability, and strong consistency are guaranteed. Examples include: Continuously upgrading k8s clusters to secure versions – whether one cluster or many, the middleware upgrade process remains essentially the same (continuous service updates); New team members can understand existing services and architecture by reading the operations code. Once they have the foundational skills in Infrastructure as Code (after passing the interview), they are capable of sustained operations (continuous human resource renewal); Here, “as code” primarily means using Ansible to save server state (what services are running, what configurations are needed, etc.) in YAML format within a Git version control system, and collaborating and continuously iterating through GitLab. ...