Say you’re running a web service that requires input. In turn, the web service delivers it to another backend service. If the backend service isn’t available for an extended time, do you have a fail-proof system in place? This is where the circuit breaker design pattern comes in.
Consider a situation when a request goes to a middleware application, and you need to call another remote backend service. If everything goes smoothly, the application can forward the request to the backend service and send the response back to the client. But, if the backend service is down, the request will not be completed. Even if the client makes subsequent requests to call the backend service, all of them will fail. Here, we can use the circuit breaker model to manage backend errors. As shown in Figure 1, according to the circuit breaker pattern, the middleware may be in the following states:
Figure 1: Circuit Breaker Pattern
You may have a backend application that executes a database query and sends the outcome back to the client. The number of individuals sending transactions to your application can be as high as 10,000 TPS (transactions per second). However, the backend server may crash and slowly recover. However, the transactions will still keep coming. To manage this situation, the system should be designed in a way that waits for a while until the service is available again. Once it is, requests can be sent.
In the circuit breaker pattern, if there is a failure, it waits for a timeout and attempts to send a request to the backend service after the timeout. The idea behind this pattern is to encapsulate the logic of managing unforeseen mistakes. This pattern is particularly helpful in circumstances such as database migration and software updates.
Circuit Breakers in Microservices
Containers/Virtual machines (VM) are dynamic in a microservices architecture. One container could disappear, and in the meantime, another container could be launched. The concept of microservices highlights mentions there could be a component failure, and the microservices architecture handles these failures. Expecting errors will make your software architecture more resilient.
Here are some practical complex scenarios where load balancing circuit breakers are used. To explain and provide examples, we will use NGINX as the circuit breaker and WSO2 Micro Integrator Circuit Breaker Pattern. NGINX microservices reference architecture has a resizer service that can resize, rotate, and shrink an image when uploaded to the system. This resizer operation is a CPU and memory-intensive job that can cause memory errors. Here, we can place the circuit breaker between the resizer and the image uploader, as shown by Figure 2.
Figure 2: Circuit Breaker in NGINX
Here, the uploader service examines the health of the resizer service. It checks if the resizer has sufficient memory to perform operations. If NGINX observes the service to be defective, the uploader will distribute loads among other instances. Once the resizer becomes ‘healthy’, the NGINX gradually ramps back up to the restarted resizer.
Here’s the practical application of the WSO2 Micro Integrator Circuit Breaker Pattern. This is an open-source middleware platform used to interconnect distinct web services. The integrator is between two endpoints. One endpoint (the client) sends a request to the other endpoint (the backend). The integrator performs the required conversion to the request through a sequence of mediators, before sending it to the backend endpoint. This scenario can be implemented with the following configurations on WSO2 Micro Integrator.
<api name="ServiceAPI" context="/cal">
<header name="To" action="remove"></header>
<property name="RESPONSE" value="true"></property>
<property name="NO_ENTITY_BODY" scope="axis2" action="remove"></property>
<ns:Error>We can't response you at this time, we will reponse through E-mail soon</ns:Error>
Here, we identify the Service API that forwards the incoming application to the backend endpoint. The mediator, <inSequence> processes the incoming request. Inside, the mediator redirects the application to the endpoint, TimeoutEP. The response to this endpoint will be directed to the <outSequence> mediator. Here, the backend reply is sent back to the client. Next, the <faultSequence> section replies to information if failures exist.
The endpoint definition would be as follows.
In the endpoint section, we can define timeout parameters for the circuit breaker pattern. The Address URL is the backend URL to which the request is redirected to. Here’s a list of configurable parameters:
The circuit breaker design pattern is a basic pattern used in monolithic and microservice-based deployments. It helps systems prevent themselves from sending unnecessary loads to a failed backend service, by providing a delay to recover from errors. It plays a key role in the microservices architecture as its containers are dynamic and can stop working at any time. The middleware platform used to manage traffic should be ready to manage the application when the backend server goes down. Circuit breaker patterns provide an elegant way to handle errors.
To learn more about improving your product resilience in WSO2 API Manager, click here.