Resilience Pattern: Circuit Breaker

Protects systems from failures, improves reliability, and reduces latency in distributed systems. Fails fast, provides fallbacks, and speeds up recovery.

In this article, we will explore one of the most common and useful resilience patterns in distributed systems: the circuit breaker. The circuit breaker is a design pattern that prevents cascading failures and improves the overall availability and performance of a system.

What Is a Circuit Breaker?

A circuit breaker is a component that monitors the health of a dependency, such as a remote service, an external API, or a database. A dependency can become unhealthy or unavailable for various reasons, such as network failures, high latency, timeouts, errors, or overload. When a dependency is unhealthy, it can cause failures in the components that depend on it, leading to a domino effect that can bring down the whole system.

A circuit breaker acts as a proxy between a component and its dependency. It intercepts the requests and responses and maintains a state that reflects the health of the dependency. The state can be one of the following:

• Closed: The circuit breaker allows all requests to pass through to the dependency. This is the normal state when the dependency is healthy and responsive.
• Open: The circuit breaker blocks all requests and fails them immediately without contacting the dependency. This state is triggered when the dependency becomes unhealthy and exceeds a predefined threshold of failures or timeouts. The circuit breaker enters this state to prevent further failures and give the dependency time to recover.
• Half-open: The circuit breaker allows some requests to pass through to the dependency while blocking the rest. This state is used to test the health of the dependency after some time in the open state. If the requests succeed, the circuit breaker transitions back to the closed state. If the requests fail, the circuit breaker returns to the open state.

How Does a Circuit Breaker Work?

A circuit breaker works by keeping track of some metrics related to the requests and responses between a component and its dependency. These metrics can include:

• The number of requests
• The number of successful responses
• The number of failed responses
• The number of timeouts
• The response time
• The error rate

Based on these metrics, the circuit breaker applies some rules to determine when to change its state. These rules can be configured according to the needs and characteristics of each system. Some common parameters are:

• Failure threshold: The percentage or number of failed requests or timeouts that trigger the transition from closed to open state.
• Reset timeout: The duration that the circuit breaker stays in the open state before transitioning to a half-open state.
• Success threshold: The percentage or number of successful requests that trigger the transition from half-open to closed state.

An example of how a circuit breaker works is shown in the following diagram:

Circuit Breaker State Diagram

In this example, we have a component A that depends on a remote service B. A circuit breaker C is placed between them to monitor their communication.

1.     Initially, both A and B are healthy, and C is in a closed state. All requests from A are allowed to pass through C and reach B, and all responses from B are returned to A.

2.     At some point, B becomes slow or unavailable due to some reason. This causes some requests from A to fail or time out. C detects these failures and increments its failure count.

3.     When the failure count reaches a predefined threshold (e.g., 50% of requests fail), C changes its state from closed to open. This means that C will block all subsequent requests from A and return an error or fallback response immediately without contacting B.

4.     After a predefined period (e.g., 10 seconds), C changes its state from open to half-open. This means that C will allow some requests from A (e.g., one request per second) to pass through and reach B while blocking the rest.

5.     If B has recovered and responds successfully to these requests, C changes its state from half-open to closed. This means that C will resume allowing all requests from A to pass through and reach B.

6.     If B is still unhealthy and responds with failures or timeouts to these requests, C changes its state from half-open back to open. This means that C will continue blocking all requests from A until another reset timeout elapses.

Why Use a Circuit Breaker?

A circuit breaker provides several benefits for distributed systems, such as:

• Improving availability: By blocking unhealthy dependencies, a circuit breaker prevents cascading failures that can affect other components or services in the system. This way, it reduces the impact of failures and improves the overall availability of the system.
• Reducing latency: By failing fast, a circuit breaker avoids wasting time and resources on waiting for unhealthy dependencies. This way, it reduces the latency and improves the performance of the system.
• Providing fallbacks: By returning an error or a fallback response, a circuit breaker can provide alternative or degraded functionality when a dependency is unavailable. This way, it can improve the user experience and maintain some level of service quality.
• Facilitating recovery: By isolating unhealthy dependencies, a circuit breaker can reduce the load and stress on them, allowing them to recover faster. This way, it can improve the resilience and stability of the system.

How To Implement a Circuit Breaker

There are different ways to implement a circuit breaker, depending on the programming language, framework, or platform used. Some examples of libraries or tools that provide circuit breaker functionality are:

• Hystrix: A Java library that implements the circuit breaker pattern and provides other features for building resilient distributed systems. It is part of the Netflix OSS suite of tools.
• Polly: A .NET library that provides various resilience and transient-fault-handling policies, including circuit breaker, retry, timeout, bulkhead isolation, and more.
• Resilience4j: A Java library that provides lightweight fault tolerance modules for Java 8 and functional programming. It includes a circuit breaker, rate limiter, retry, bulkhead, and more.
• Istio: A service mesh that provides a uniform way to connect, secure, control, and observe microservices. It includes circuit breaker, retry, timeout, load balancing, and more.

Conclusion

The circuit breaker is a resilience pattern that improves the availability and performance of distributed systems by preventing cascading failures and providing fallbacks. It works by monitoring the health of a dependency and changing its state according to some rules. It can be implemented using various libraries or tools, depending on the technology stack used.

I hope you enjoyed this blog post and learned something new. If you have any questions or feedback, please leave a comment below. Thank you for reading!

We Provide consulting, implementation, and management services on DevOps, DevSecOps, DataOps, Cloud, Automated Ops, Microservices, Infrastructure, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

 If this seems interesting, please email us at [email protected] for a call.