All our web applications deal with data.
We store this data mostly in a database.
The data which is stored thus represents the current state of the data.
For example , if you have a Customer table the table will have entries which represent the current state of the customer (what their name is and address is, etc) . If the customer address is changed you will update the entry and the entry will now represent the current state of the customer.
In some scenarios though you would want more than the current state , you might need all the states which the customer entry went through.
For such cases , the design pattern “Event Sourcing” helps.
This post explains the basics of this design pattern with a use case.
Contents:
- Use Case
- Application without Event Sourcing
- What is Event Sourcing
- Application with Event Sourcing
- Conclusion
Use Case:
Gaurav is a shop keeper.
He sells electronic items like mobile phones , laptops etc.
He wants to keep track of the stock in his shop.
His requirement is simple:
He wants to know whether there is stock of a particular item in his shop without manually checking it
And he wants an app for it.
He approaches his software engineer friend and asks for one.
His friend agrees and develops an application in spring boot in the traditional way.
The app has three functionalities:
- User can add new stock
- User can remove stock after selling it
- User can find the current stock of a particular item
His friend confirms if these are what Gaurav wants.
Gaurav nods. He also mentions that his co-owner Karthik will also be using the app.
His friend got the requirements, delivered the app and Gaurav started using it.
Whenever he adds new stock to his inventory, he uses the app to add new stock.
Whenever he sells an item he removes the number of items from his stock using the app.
Whenever he wants to know the count of a particular stock in inventory he uses the app.
Application without Event Sourcing:
These are the three back end API s that his friend developed:
Adding a stock item:
@PostMapping("/stock")
public void addStock(@RequestBody Stock stock) {
List<Stock> existingStockList = repo.findByName(stock.getName());
if (existingStockList != null && existingStockList.size() > 0) {
Stock existingStock = existingStockList.get(0);
int newQuantity = existingStock.getQuantity() + stock.getQuantity();
existingStock.setQuantity(newQuantity);
existingStock.setAddedBy(stock.getAddedBy());
repo.save(existingStock);
} else {
repo.save(stock);
}
}
Removing a stock item:
@DeleteMapping("/stock")
public void removeStock(@RequestBody Stock stock) {
int newQuantity = 0;
List<Stock> existingStockList = repo.findByName(stock.getName());
if (existingStockList != null && existingStockList.size() > 0) {
Stock existingStock = existingStockList.get(0);
newQuantity = existingStock.getQuantity() - stock.getQuantity();
if (newQuantity <= 0) {
repo.delete(existingStock);
} else {
existingStock.setQuantity(newQuantity);
existingStock.setAddedBy(stock.getAddedBy());
repo.save(existingStock);
}
}
}
Getting current count of stock :
@GetMapping("/stock")
public List<Stock> getStock(@RequestParam("name") String name) {
return repo.findByName(name);
}
Here is the stock entity model:
package com.example.stock.management;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import lombok.Data;
@Entity
@Data
public class Stock {
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
private String name;
private int quantity;
private String addedBy;
}
Here is the repository:
package com.example.stock.management;
import java.util.List;
import org.springframework.data.repository.CrudRepository;
public interface StockRepo extends CrudRepository<Stock, Integer> {
List<Stock> findByName(String name);
}
That is the bare essential backend code.
Let’s ignore the front end code for simplicity of this post.
Let’s add some stock:
Gaurav added 16 iPhones, 25 Macbooks and Karthik added 15 iPhones to the stock.
Here is the entry in the database now:
31 iPhones and 25 Macbooks is the current stock and “added_by” column has the value of the user who last added stock.
Let’s remove some items from the stock as few items got sold:
Gaurav sold 6 iPhones and 6 Macbooks.
Now let’s find the current stock status using API:
There are 25 iPhones(31 – 6) right now as expected.
And 19 Macbooks(25 – 6) as expected.
As you noticed the above app just stores the current state of a stock item in a database.
Whenever the stock is added or removed , the current state is updated.
Gaurav was fine with this initially.
But one day he checked the stock for iPhone and felt a doubt that it was wrong.
He wanted to check what was the stock count the previous day.
He can’t do that as the previous states of entities are not stored in traditional programming.
He went to his friend and explained his predicament.
He also told his friend that he wants to check the history of what was added to the stock by whom and what was removed by whom and when these happened.
Luckily , his friend knew Event Sourcing design pattern.
He didn’t write separate code to include these additional functionalities.
He refactored the code in such a way that it was easy to achieve the above functionalities.
So how to do Event Sourcing and what is this?
What is Event Sourcing:
In Event Sourcing you just capture user events and add them in database.
You just keep adding new events for every user action.
No record is updated or deleted in the database , just events are added. Along with the events you also add event data specific to the event.
This way you maintain the history of user action. This is useful if your application has security requirements to audit all user actions. This is also useful in any application where you want the history of user actions (example github commits , analytics applications etc)
And to know the current state of an entity , you just rerun the events through your code and get that.
For example in the previous example , you could create two events :
StockAddedEvent
StockRemovedEvent
You can then store these events in database in a “event store” table.
Along with the event you can also store event data like what item was added ,the quantity and the user who added the item.
And if you want to know the current stock of an item , you fetch all the events corresponding to the item , rerun them in your code and calculate the current stock. This particular logic looks cumbersome particularly when the records are huge in number. But there are ways to minimize the complexity. Also Event Sourcing works well with other design patterns like CQRS and with Domain Driven Design (DDD) concepts.
Application with Event Sourcing:
Now let’s see how Gaurav’s friend refactored the code for Event Sourcing and how it helped Gaurav:
Here are the new REST APIs:
To add a new stock you just add a new event with the stock data (event data):
@PostMapping("/stock")
public void addStock(@RequestBody Stock stockRequest) throws JsonProcessingException {
StockAddedEvent event = StockAddedEvent.builder().stockDetails(stockRequest).build();
service.addEvent(event);
}
As you see , you create a Stock Added Event and store it in database through a service. I have used lombok builder to populate the StockAddedEvent model object.
Here is the stock event model:
package com.example.stock.management;
import lombok.Builder;
import lombok.Data;
@Builder
@Data
public class StockAddedEvent implements StockEvent {
private Stock stockDetails;
}
Stock model:
package com.example.stock.management;
import lombok.Data;
@Data
public class Stock {
private String name;
private int quantity;
private String user;
}
Stock Event interface:
package com.example.stock.management;
public interface StockEvent {
}
Here is the event service which adds the event to database:
public void addEvent(StockAddedEvent event) throws JsonProcessingException {
EventStore eventStore = new EventStore();
eventStore.setEventData(new ObjectMapper().writeValueAsString(event.getStockDetails()));
eventStore.setEventType("STOCK_ADDED");
eventStore.setEntityId(event.getStockDetails().getName());
eventStore.setEventTime(LocalDateTime.now());
repo.save(eventStore);
}
As you see EventStore represents the event store table and the event details , the time at which the event occured , the entity id corresponding to the event and the event type are stored in this table. Every time an event happens a new record is added to the event store table. It is never updated or deleted!
Here is the repository:
package com.example.stock.management;
import java.time.LocalDateTime;
import org.springframework.data.repository.CrudRepository;
import org.springframework.stereotype.Component;
@Component
public interface EventRepository extends CrudRepository<EventStore, Long>{
}
Similar code for removing stock items. You don’t delete any entry in Event Store table. You just add one more event to it (StockRemovedEvent):
@DeleteMapping("/stock")
public void removeStock(@RequestBody Stock stock) throws JsonProcessingException {
StockRemovedEvent event = StockRemovedEvent.builder().stockDetails(stock).build();
service.addEvent(event);
}
The logic gets complicated when you want to retrieve the current stock of an item. It was pretty straightforward in the previous traditional approach.
Here the code looks like this:
@GetMapping("/stock")
public Stock getStock(@RequestParam("name") String name) throws JsonProcessingException {
Iterable<EventStore> events = service.fetchAllEvents(name);
Stock currentStock = new Stock();
currentStock.setName(name);
currentStock.setUser("NA");
for (EventStore event : events) {
Stock stock = new Gson().fromJson(event.getEventData(), Stock.class);
if (event.getEventType().equals("STOCK_ADDED")) {
currentStock.setQuantity(currentStock.getQuantity() + stock.getQuantity());
} else if (event.getEventType().equals("STOCK_REMOVED")) {
currentStock.setQuantity(currentStock.getQuantity() - stock.getQuantity());
}
}
return currentStock;
}
As you see you initialize a Stock object , retrieve all the events corresponding to the item , iterate through the events and whenever you see Stock Added Event you increment the quantity of the stock and whenever you see Stock Removed Event you decrement the quantity of the stock . At the end of the iteration you get the current stock of the item.
Let’s test it out:
Adding stock:
I performed the same REST API calls as in the previous case:
Example for adding a new stock:
Gaurav added 16 iPhones and Karthik added 25 Macbooks.
Karthik then added 15 iPhones.
Gaurav then removed 6 iPhones and 6 Macbooks.
Let’s check the database now:
Oops , there are so many records , as many records as the events performed by Gaurav and Karthik together. As you see all user events are captured in the event store table.
Let’s see if we are able to get the current stock by firing the GET API:
Yes ! there are 25 iPhones currently in the stock . We got this current state by iterating through all the events.
Now if Gaurav wants to know what was the stock the day before , we could write an API which takes a date as request parameter and runs all the events until that date and return the state until that date:
@GetMapping("/stock/history")
public Stock getStockUntilDate(@RequestParam("date") String date,@RequestParam("name") String name) throws JsonProcessingException {
String[] dateArray = date.split("-");
LocalDateTime dateTill = LocalDate.of(Integer.parseInt(dateArray[0]), Integer.parseInt(dateArray[1]), Integer.parseInt(dateArray[2])).atTime(23, 59);
Iterable<EventStore> events = service.fetchAllEventsTillDate(name,dateTill);
Stock currentStock = new Stock();
currentStock.setName(name);
currentStock.setUser("NA");
for (EventStore event : events) {
Stock stock = new Gson().fromJson(event.getEventData(), Stock.class);
if (event.getEventType().equals("STOCK_ADDED")) {
currentStock.setQuantity(currentStock.getQuantity() + stock.getQuantity());
} else if (event.getEventType().equals("STOCK_REMOVED")) {
currentStock.setQuantity(currentStock.getQuantity() - stock.getQuantity());
}
}
return currentStock;
}
As you see , all the events for a particular item until a particular date are fetched. The events are then iterated and the stock quantity is calculated.
I changed the date of the first event in the backend to 3 days prior.
And then fired the above /stock/history API to get the status of the stock 3 days prior:
It shows 16 iPhones instead of 25 since the rest were added and removed after 14-June-2022!
We can find out the stock state of any particular day!
Also if Gaurav wants to know what stock was added when and by whom , his friend could write a new API with minimum effort which returns all the user events :
@GetMapping("/events")
public Iterable<EventStore> getEvents(@RequestParam("name") String name) throws JsonProcessingException {
Iterable<EventStore> events = service.fetchAllEvents(name);
return events;
}
Let’s fire this API:
Gaurav can now see what happened with this inventory across time!
That is the benefit of event sourcing.
Conclusion:
As already mentioned , retrieving the current state of an entity is not straightforward and not scalable in event sourcing. This can be mitigated by taking snapshots of events at a particular time , calculate the state of the entity for the snapshot at that time , store it somewhere and then rerun only those events which happened after that snapshot time.
Event Sourcing is more beneficial in CQRS pattern where you have a separate data store for storing “commands” (insert, update and delete) and a separate data store for “read” operations. The “command” data store can be populated as an event store and for each event in that table you can fire a asynchronous event which populates the “read” datastore as required by the business.
Also since you never modify or delete data in event sourcing , there is no issue of “database locking” which will improve the performance of your application.
That’s a very simplified example for event sourcing which helps to gain an understanding of what event sourcing is.
Here is the code for the traditional and event store stock inventory app:
Leave a Reply