Microservices: Introduction to Spring Cloud Config Server and Config Client (Part 1)

I am a big fan of Spring family. In this article, i am going to explain how to use Spring Cloud Config Server for externalizing and versioning the configuration properties of your microservice.

 

The Problem!

The one of the most challenge in the distributed application environment (or rather microservices environment) is to maintain and manage the configuration related properties in the microservices.

In microservice environment, there may be hundred of isolated services and each service may have different configuration properties depending on the environment (eg:- qa, dev, prod, uat etc…).

Some of the properties are shared among services and some of the properties are specific/private to that service. Those properties may be changed or updated in later and the update need to be reflected. If the configuration maintaining and management is not properly planned, you have to face for a difficult time when those update happens.

Lets dig into deeper and look at what are the real difficulties associated with maintaining and managing those configuration properties in a microservice environment.

  • Assume that the configuration properties are maintained in each application service. If any property is changed, the service need to be restarted. Otherwise it will not be reflected in the service. It is not possible to update those properties in runtime. we have to make the required property change, rebuild the application and deploy it again. This is not practical when it comes to handle large number of services (as the configurations may change rapidly). Wouldn’t be nicer if the change is reflected without deploying or restarting the service?

 

  • Some of the configurations are common for all or few services.  e.g:- database and security related configurations. If we maintain the properties in the service, then we have to  duplicate those configuration properties in each service. But that is not a good way. If the configuration property is changed, all the related services need to be modified to reflect that change. Wouldn’t be nicer if there is any centralized way to maintain the properties that allows to share them across multiple services?

 

The Solution!

As  described above the ideal solution is to have centralized mechanism (service) to maintain and manage application related properties for all services. If there is any property change, only the centralized server (or service) will update the change and all the related service will received the updated property without restarting themselves.

Spring family has introduced nice solution called “Spring Cloud Config” that addresses all the above problems and provides the exact solution as described above.

 

What is Spring Cloud Config?

Spring Cloud Config (http://projects.spring.io/spring-cloud/) has the client server communication architecture. It has both server and client modules. server will be responsible for maintaining the properties and client will retrieve them.

Spring Cloud Config Server can be used to centralize all the applications configuration. Then the Spring Cloud Config Client module can be used by service applications to consume configuration properties from Config Server. It is possible to update the configuration properties at runtime without restarting or redeploying the application.

The Spring Cloud Config Server retrieves the properties from the configured property source. The property source can be a Git repository, svn or Consul service.  In this article, we will use Git repository as a property source.

 

15359236.png

 

 

What we are going to build….

The below diagram demonstrates the architecture of the application that we are going to build for this article.

Slice 1.png

  1. department-service requests the property from the spring-cloud-config-server 
  2. spring-cloud-config-server access the configured property source (here it is the the Git repository)
  3. spring-cloud-config-server receives the latest configurations from Git repository (property source).
  4. department-service receives the latest properties from config-server.
  5. user has made a changed to property files and updated the Git repository.
  6. invoke the /actuator/refresh endpoint on the relevant service to get the change reflected in the targeted service.

 

Lets build each component as follows.

Adding all the property files

Here we will add all the property files for the Git repository and set up the property source for the Spring Cloud Config Server. All the application related property files should be stored here.

The property files for all services related to this example can be found at GitHub. Click here to visit. 

 

The property file naming rule

Since all the property files related to different services are placed in a single location, there should be a way to distinguish them among services. Otherwise the Spring Cloud Config Server will have to face for a problem when picking up the correct configuration file for the service. This can be achieved with name of the property file. The property file should be named based on following rules.

  • The name should start with the application name as declared in the relevant service.

e.g:-  spring.application.name = department-service

  • If there different profiles, the profile name should comes after the application name.

e.g:- department-service-uat.properties , department-service-qa.properties  (or .yml)

The spring cloud config server will pick the correct property file based on the application name and the activated profile name. If no profile is activated/mentioned explicitly, it will pick the property file with no profile suffix.

e.g:- department-service.properties (or .yml)

 

What is the purpose of application.properties?

It is used to declare the properties that should be shared among all services. (will be discussed later)

 

Building Spring Cloud Config Server

This is a just spring boot application with spring-cloud-config-server dependency. This can be create as follows.

 

Screen Shot 2018-07-13 at 8.26.59 PM.png

The following configurations should be made to build this project as the spring-cloud-config-server.

  • Annotate the main spring boot application class with @EnableConfigServer.
  • Add the following configurations in the application.properties.
server.port = 8080  # only to change the server port. 
spring.cloud.config.server.git.uri = YOUR_GIT_REPOSITORY_URL
spring.cloud.config.server.git.searchPaths = configurations

 

server.port :- This is just to change the server port of the application. (not a spring cloud config related property)

spring.cloud.config.server.git.uri :- This specifies the Git repository location where the property files are stored.

spring.cloud.config.server.git.searchPaths :- If the property files are stored under the sub directory of the repository, the directory name should be specified here. if there are multiple sub directories, those can be declared with spaces. If the property files are stored in the root of the repository, then you can neglect this configuration. In this example, i have stored the property files under the “configurations” directory of the repository. Therefore i have used the “searchPaths” configuration here.

 

Screen Shot 2018-07-15 at 1.41.17 AM.png

Now the development of the Spring Cloud Config Server is completed and you can run the server.  Spring Cloud Config Server exposes the following REST endpoints to get the application specific configuration properties.

GET /{application}/{profile}[/{label}]
GET /{application}-{profile}.yml
GET /{label}/{application}-{profile}.yml
GET /{application}-{profile}.properties
GET /{label}/{application}-{profile}.properties
Here {application} refers to value of spring.config.name property, {profile} is an active
profile and {label} is an optional git label (defaults to “master”).

Now if you access the URL http://localhost:8080/department-service/default then you will get the following response with department-service default configuration details:

 Screen Shot 2018-07-15 at 2.09.49 AM.png
Similarly, if you access the URL http://localhost:8080/employee-service/default then you will get the following response with employee-service default configuration details:

 

Screen Shot 2018-07-15 at 2.10.37 AM.png

 

 

Let’s develop the department-service

department-service is an another spring boot web application with few dependencies as follows.

  • Config Client :- for retrieving the properties from the Config Server.
  • Actuator :-  for invoking the /actuator/refresh endpoint for retrieving the latest property changes from Config Server. (this will be discussed later)

You can visit the Spring Initializer and initiate the department-service project as follows.

 

 

Create a bootstrap.properties file and add the following configuration properties 

These properties will make  a connection with spring cloud config server for retrieving the remote properties.

spring.application.name=department-service
spring.cloud.config.uri=http://localhost:8080

spring.application.name is the identifier for the application and there will be some property file with the same name.

(e.g:- department-service.properties ,  department-service-dev.properties)

spring.cloud.config.uri  tells where the spring clcoud config server is located.

 

Add the following configuration for the application.properties

server.port = 8081
management.security.enabled=false
management.endpoints.web.exposure.include=*

 

management.security.enabled=false :- this will disable the spring security check on actuator endpoints such as /actuator/refresh etc…

management.endpoints.web.exposure.include=*  :- this will enable the exposing of all Spring Boot Actuator endpoints (such as /actuator/refresh etc…)

 

 

WelcomeController

This will contains the relevant endpoints which will retrieve the properties in the department-service.properties and application.properties.  You can check them as follows.

 

 

@RefreshScope  and   “/actuator/refresh”  event

By default, the configuration values are read on the client’s startup, and not again. You can force a bean to refresh its configuration – to pull updated values from the Config Server – by annotating the WelcomeController with the Spring Cloud Config @RefreshScope and then by triggering a /actuator/refresh event.

Once the /actuator/refresh event is triggered for the service, all the beans that are annotated with @RefreshScope will be refreshed (that means the values will be retrieved from the Config Server and updates the bean).

 

Now your project structure should looks as follows. 

Screen Shot 2018-07-15 at 5.49.47 PM.png

 

 

Invoke the department-service endpoints

Checking whether the correct property file is picked for the department-service. since we haven’t activated any profile, it should pick the property file without profile suffix.    YES! it is correctly picking up the department-service.properties

Screen Shot 2018-07-15 at 9.15.24 PM.png

 

Checking whether the properties specified in the application.properties is accessible. 

Screen Shot 2018-07-15 at 9.16.44 PM.png

 

 

 

Trigger refresh event with “actuator/refresh”

Now we will change the value of the “app.service-name” property in the department-service.properties and commit the changes.

Then we will try to access the /service endpoint of the department service.  You can see that the new value is not reflected yet. This is because we haven’t trigger the refresh event for the application.  (it can be triggered with Spring Boot Actuator)

Screen Shot 2018-07-15 at 9.15.24 PM.png

 

 

Lets trigger the refresh event with Spring Boot Actuator as follows. This will result in refreshing all the beans annotated with @RefreshScope. Since the WelcomeController is annotated with @RefreshScope, its properties will also get refreshed.

Screen Shot 2018-07-15 at 9.15.40 PM.png

 

 

After triggering the refresh event, access the /service endpoint of the department-service. You can see that updated value is reflected now.

Screen Shot 2018-07-15 at 9.16.01 PM.png

 

 

Why spring-cloud-config-server related properties are placed in bootstrap.properties (bootstrap.yml) instead of application.properties (or application.yml) ?

In microservice environment, the properties related to the service may be stored in the Spring Cloud Config Server and it is identified as a external property source. Therefore those external properties should be fetched and made available to the application before the main application context is loaded.  If we put those spring-cloud-config-server (fetching related) configuration properties in the application.properties file, the remote properties will not be loaded before the main application context loads. This may lead for some runtime errors saying that the targeted properties does not exist in the main application context. The solution is to load those external properties before the main application context started.

As the Spring Cloud documentation says A Spring Cloud application operates by creating a “bootstrap” context, which is a parent context for the main application content. It is responsible for loading configuration properties from the external sources and for decrypting properties in the local external configuration files.

Yes. we have found the solution. we should place the spring-cloud-config related properties in the bootstrap.properties (bootstrap.yml). It will be loaded before the  application.properties (application.yml) is loaded. It will helps to create a “bootstrap” context which is a parent context of the main application context and it is responsible for fetching the related properties from the external configuration sources (such as Spring Cloud Config Server)

 

Let’s develop the employee-service

You can follow the same steps used for the department-service with following configuration changes.

server.port=8082   # application.properties
spring.application.name=employee-service   # bootstrap.properties

 

The complete source code related to the employee-service can be found at GitHub.

 

Please refer the below screenshot for more details.  Those endpoints are related to the employee-service running on port 8082.

Screen Shot 2018-07-15 at 10.34.04 PM.png

 

This is the result of the shared property.

Screen Shot 2018-07-15 at 10.36.06 PM.png

 

 

How to share properties (common properties) across all services?

As i have described above, there are some properties which are common to all or few services. e.g:- database and security related properties etc..

we will look at how those properties are managed with spring cloud config server.

Any property that is added in to the application.properties (or .yml)  will be shared among all services.  Therefore if you want some properties to be shared among all or few services, add them into a file called application.properties (or application.yml) and commit the file. Then the Spring Cloud Config Server will enable those properties available for all services (as shared properties).

In this example, you can see that i have added  the “app.shared.attribute” property in the application.properties in this example.

 

Source Code

The full source code of this article can be found at GitHub.

 

What is next?

Did you feel that it is difficult (rather less practical) to manually trigger the refresh event for all services whenever a property is changed? I also felt the same feeling. This is not practical approach when it comes to manage large number of microservices.  Isn’t it good if we trigger the refresh event for a single service and it broadcast the event for other related services?

Spring has a nicer solution to overcome this issue and it is called the Spring Cloud Bus.  In the next post, we will see how to use the Spring Cloud Bus to get rid of manually  triggering refresh events for all services.

Click here to go to Part 2 of this article. 

 

References:

https://cloud.spring.io/spring-cloud-config/single/spring-cloud-config.html

 

2 thoughts on “Microservices: Introduction to Spring Cloud Config Server and Config Client (Part 1)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s