Course Overview
Introduction
Introduces distributed systems and explains why patterns and reusable components can make such a difference in the rapid development of reliable distributed systems
Part I, Single-Node Patterns
Discusses reusable patterns and components that occur on individual nodes within a distributed system. It covers the side-car, adapter, and ambassador single-node patterns.
Part II, Serving Patterns
Covers multi-node distributed patterns for long-running serving systems like web applications. Patterns for replicating, scaling, and master election are discussed.
Part III, Batch Computational Patterns
Covers distributed system patterns for large-scale batch data processing covering work queues, event-based processing, and coordinated workflows.
Who Should Attend
A system engineer and developer would find the entire course very useful if he / she is interested in building distributed systems on Azure. If you are an experienced distributed system engineer, first couple of chapters may seem easy, though you may want to skim them to understand how we expect these patterns to be applied and why we think the general notion of distributed system patterns is so important. Depending on your goals and the systems you are interested in developing, you can choose to focus on either large-scale big data patterns, or patterns for long-running servers (or both). The two parts are largely independent from each other and can be read in any order.
Likewise, if you have extensive distributed system experience, you may find that some of the early patterns chapters (e.g. on naming, discovery, and load balancing) are redundant with what you already know, so feel free to skim through to gain the high-level insight.
Course Duration
5 days with Hands on Labs
Course Outline
Introduction
A Brief History of Systems Development
A Brief History of Patterns in Software Development
Formalization of Algorithmic Programming
Patterns for Object-Oriented Programming
The Rise of Open Source Software
The Value of Patterns, Practices, and Components
Standing on the Shoulders of Giants
A Shared Language for Discussing Our Practice
Shared Components for Easy Reuse
Part I, Single-Node Patterns
The Sidecar Pattern
An Example Sidecar: Adding HTTPS to a Legacy Service
Dynamic Configuration with Sidecars
Modular Application Containers
Hands On: Deploying the topz Container
Building a Simple PaaS with Sidecars
Designing Sidecars for Modularity and Reusability
Parameterized Containers
Define Each Containers API
Documenting Your Containers
Ambassadors
Using an Ambassador to Shard a Service
Hands On: Implementing a Sharded Redis
Using an Ambassador for Service Brokering
Using an Ambassador to Do Experimentation or Request Splitting
Hands-On: Implementing 10% Experiments
Adapters
Monitoring
Hands On: Using Prometheus for Monitoring
Logging
Hands On: Normalizing Different Logging Formats with Fluentd
Adding a Health Monitor
Hands-On: Adding Rich Health Monitoring for MySQL
Part II, Serving Patterns
Introduction to Microservices
Replicated Load-Balanced Services
Stateless Services
Readiness Probes for Load Balancing
Hands-On: Creating a Replicated Service in Kubernetes
Session Tracked Services
Application-Layer Replicated Services
Introducing a Caching Layer
Deploying Your Cache
Hands-On: Deploying the Caching Layer
Expanding the Caching Layer
Rate Limiting and Denial-of-Service Defense
SSL Termination
Hands On: Deploying nginx and SSL Termination
Sharded Services.
Sharded Caching
Why You Might Need a Sharded Cache
The Role of the Cache in System Performance
Replicated, Sharded Caches
Hands On: Deploying an Ambassador and Memcache for a Shared Cache
An Examination of Sharding Functions
Selecting a Key
Consistent Hashing Functions
Hands On: Building a Consistent HTTP Sharding Proxy
Sharded, Replicated Serving
Hot Sharding Systems
Scatter/Gather
Scatter/Gather with Root Distribution
Hands On: Distributed Document Search
Scatter/Gather with Leaf Sharding
Hands On: Sharded Document Search
Choosing the Right Number of Leaves
Scaling Scatter/Gather for Reliability and Scale
Functions and Event-Driven Processing
Determining When FaaS Makes Sense
The Benefits of FaaS
The Challenges of FaaS
The Need for Background Processing
The Need to Hold Data in Memory
The Costs of Sustained Request-Based Processing
Patterns for FaaS
The Decorator Pattern: Request or Response Transformation
Hands On: Adding Request Defaulting Prior to Request Processing
Handling Events
Hands On: Implementing Two-Factor Authentication
Event-Based Pipelines
Hands On: Implementing a Pipeline for New-User Signup
Ownership Election
Determining If You Even Need Master Election
The Basics of Master Election
Hands On: Deploying etcd
Implementing Locks
Hands On: Implementing Locks in etcd
Implementing Ownership
Hands On: Implementing Leases in etcd
Handling Concurrent Data Manipulation
Part III, Batch Computational Patterns
Work
A Generic Work Queue System Queue Systems
The Source Container Interface
The Worker Container Interface
The Shared Work Queue Infrastructure
Hands On: Implementing a Video Thumbnailer
Dynamic Scaling of the Workers
The Multi-Worker Pattern
Event-Driven Batch Processing
Patterns of Event-Driven Processing
Copier
Filter
Splitter
Sharder
Merger
Hands On: Building an Event-Driven Flow for New User Sign-Up
Publisher/Subscriber Infrastructure
Hands On: Deploying Kafka
Coordinated Batch Processing
Join (or Barrier Synchronization)
Reduce
Hands On: Count
Sum
Histogram
Hands On: An Image Tagging and Processing Pipeline
Hands On Labs
LAB 1: Single Node Pattern: Ambassador
LAB 2: Single Node Pattern: Circuit Breaker Pattern
LAB 3: Serving Pattern: Load Balancing Server
LAB 4: Serving Pattern: Decorator Function
LAB 5: Batch Computation Pattern
Categories
More Information
- (Local Institution) NTUC LearningHub
Add a review