Never Test in Production 🚫🔧

Definition of Production 📦

• Production Environment: The live environment where services are actively delivered to users. For example, a website's live server or an authentication service handling user logins.

Importance of Testing Changes 🧪

Before implementing any changes, it’s crucial to test them in a non-production environment to avoid disrupting live services.

Test Environment 🖥️

• Purpose: Mimics the production environment but does not serve actual users.

• Benefit: Allows you to identify and fix issues without impacting users.

Secondary or Stand-By Machine 💻

• Purpose: A duplicate of the production machine that remains inactive until needed.

• Process: Apply changes to the stand-by machine first. Once verified, switch it with the primary machine and update the original primary machine.

Canary Releases 🦠

• Concept: Similar to coal miners’ canaries used to detect gas, use a small subset of servers to test changes.

• Purpose: Minimize risk by exposing only a small user base to new changes before full deployment.

• Process: After successful testing on canaries, deploy changes to the entire fleet.

Testing Minor Changes 🧩

• Principle: Even minor changes should be tested in a test environment before applying to production.

• Reason: Prevent unforeseen issues and ensure stability.

Summary 📋

1. Always Test First: Use a test environment to verify changes before applying them to production.

2. Utilize Secondary Machines: Apply changes to a stand-by machine first to ensure stability.

3. Employ Canary Releases: Test changes on a subset of servers to mitigate risk.

4. Even Minor Changes: Test all changes, regardless of size, to maintain service reliability.