In this post we saw how to do client side load balancing in Spring Boot Microservices.
One disadvantage of client side load balancing is every microservice client need to implement this load balancing. This might not be a big deal given how simple the changes required are. Still there is some coupling on the client side.
To avoid this you can go for server side load balancing.
Spring provides its own load balancer Spring Cloud Gateway. This can be used both as a load balancer and a API gateway. One popular option for Spring developers for server side load balancing has been Netflix Zuul. But Spring has stopped support for it and so it is better to go for Spring Cloud Gateway.
Here is a simplified example of how to configure a server side load balancer along with Netflix Eureka service registry.
Let’s use the same set of microservices used in this post.
The Eureka Server runs on port 8761.
Eureka client (eureka-client) runs on port 8080.
I have spun three instances of Eureka Client 2 (eureka-client-2) at ports 8082.8083 and 8089 respectively to test if the load is getting distributed among the instances.
In addition to these projects we need to create a new gateway project.
To do that follow the below steps:
STEP1: Create a spring boot project with spring cloud gateway and spring cloud eureka client dependencies
STEP2: Configure application.yml to handle routing.
STEP1: Create a spring boot project
Create a simple spring boot project and add the below dependencies:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
You need eureka-client dependency since the gateway project needs to register itself to the Eureka Server and then only it can communicate with the other microservices in the service registry.
The dependency spring cloud starter gateway takes care of converting your microservice into an API gateway.
Here is the full pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.6.6</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>springcloudgatewaydemo</artifactId>
<version>1.0.0</version>
<name>springcloudgatewaydemo</name>
<description>Demo project for Spring Cloud Gateway</description>
<properties>
<java.version>11</java.version>
<spring-cloud.version>2021.0.1</spring-cloud.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
STEP2: Configure application.yml to handle routing
Once you add the dependencies add configuration in application.yml to handle routes.
We had already deployed a REST API in “eureka-client-2” microservice with the path “/client2”.
Here is the REST API from the client project:
package com.example.eureka.client;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.core.env.Environment;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class TestController {
@Autowired
private Environment environment;
@GetMapping("client2")
public String client2() {
String serverPort = environment.getProperty("local.server.port");
return "I am a REST API in client 2 running on port "+serverPort;
}
}
I have spun up three instances of the above project at port numbers 8082,8083,8089.
Let’s try calling the above API from “eureka-client” microservice through the gateway project we just created.
Before that let’s configure application.yml file of the gateway project:
server:
port: 8088
spring:
application:
name: gateway-app
cloud:
gateway:
routes:
- id: client1
uri: lb://eureka-client
predicates:
- Path=/test
- id: client2
uri: lb://eureka-client-2
predicates:
- Path=/client2
I have configured the gateway app to run on port 8088.
I have defined two routes:
One with id client1:
This routes all requests with the path “/test” as defined in predicates attribute to the load balancer (denoted as “lb” in short) and forwards to the service “eureka-client” (this is the name with which the client has registered itself in the service registry)
Another with id client2:
This routes all requests with path “/client2” to the service “eureka-client-2” on the load balancer.
That’s it!
Notice that I have not configured the eureka server details here. The gateway project automatically detects the eureka server.
How?
It looks for the server at port 8761 and on the same machine by default.
If you have deployed your eureka server anywhere else you need to configure that in application.yml so that the gateway project knows where to look for.
Now let’s look at all the microservice instances running on the service registry:
As expected there is one instance of eureka-client microservice,
three instances of eureka-client-2 microservice
and one instance of gateway-app
Now let’s try to call the API /client2 defined on microservice “eureka-client-2” from the microservice “eureka-client”.
Let’s create a REST API (this is modified from this post) :
package com.example.eureka.client;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
@RestController
public class TestController {
@GetMapping("/test")
public String test() {
String url = "http://localhost:8088" + "/client2";
System.out.println(url);
String response = new RestTemplate().getForObject(url, String.class);
return response;
}
}
As you see we are making a call to the load balancer deployed on port 8088 with the path “/client2”.
As this path has been configured to be routed to the service “eureka-client-2” on the gateway project routing happens.
And since there are three instances the hits are evenly distributed among the three instances.
Let’s hit the above API:
First hit:
Second hit:
Third hit:
The hits got distributed among the three instances automatically!
The gateway project took care of that.
Now if you add more instances , hits will be distributed to the new instances after a default interval of 30 seconds (this is the rate at which the microservices send a heart beat to the load balancer to say they are alive).
If you want the instances to be immediately reflected , you can add spring boot actuator dependency and then call refresh API of the actuator .
Configuring Routes:
In this example we configured routes based on the path (request with “/test” path were routed to first microservice client etc)
You can also configure the routes based on time :
- Requests after a specific time should be routed to this microservice
- Requests before a specific time should be routed to this microservice
- Requests between specific times should be routed to this microservice
And based on query parameters, cookies etc.
Refer the documentation for more details:
https://cloud.spring.io/spring-cloud-gateway/reference/html/#gateway-request-predicates-factories
That’s it!
Here is the code for the gateway project:
Leave a Reply