Using Domain Events in Microservices

Using Domain Events in Microservices
If we think back to the working principle of computer hardware, we will find that the entire working process of a computer is actually a process of processing events. When you click the mouse, tap the keyboard, or plug in a USB flash drive, the computer processes various external events in the form of interrupts. In the field of software development, Event Driven Architecture (EDA) has long been used by developers in various practices. Typical application scenarios include browser processing of user input, message mechanisms, and SOA. Reactive Programming, which has re-entered the vision of developers in recent years, regards events as first-class citizens in this programming model. It can be seen that the concept of "event" has always played an important role in the field of computer science. [[196313]]

Understanding Domain Events

Domain Events is a concept in Domain Driven Design (DDD) that is used to capture what has happened in the domain we are modeling. Domain events themselves have also become part of the ubiquitous language and become the communication language of all project members, including domain experts. For example, in the user registration process, we may say "When the user successfully registers, send a welcome email to the customer." At this time, "the user has registered" is a domain event. Of course, not everything that has happened can become a domain event. A domain event must be valuable to the business and help form a complete business closed loop, that is, a domain event will lead to further business operations. Take the coffee shop modeling as an example. When a customer comes to the front desk, the "customer has arrived" event will be generated. If you are concerned about customer reception, such as reserving a seat for the customer, then "customer has arrived" is a typical domain event because it will be used to trigger the next step - "reserve a seat" operation; but if you are modeling a coffee checkout system, then "customer has arrived" is not necessary at this time - you can't ask the customer for money immediately when the user arrives, right? "Customer has placed an order" is an event that is useful to the checkout system. In the practice of microservices architecture, people have borrowed a lot of concepts and technologies from DDD, such as a microservice should correspond to a bounded context in DDD; in microservice design, the aggregate root in DDD should be identified first; and the anti-corruption layer (ACL) in DDD should be used when integrating between microservices; we can even say that DDD and microservices have a natural tacit understanding. For more information about DDD, please refer to my other article or "Domain Driven Design" and "Implementing Domain Driven Design". There is a principle in DDD: one business use case corresponds to one transaction, and one transaction corresponds to one aggregate root, that is, in one transaction, only one aggregate root can be operated. However, in actual applications, we often find that one use case needs to modify multiple aggregate roots, and different aggregate roots are in different bounded contexts. For example, when you buy something on an e-commerce website, your points will increase accordingly. The purchase behavior here may be modeled as an order object, and the points can be modeled as a property of the account object. Both orders and accounts are aggregate roots and belong to the order system and account system respectively. Obviously, we need to maintain data consistency between orders and points. However, updating both in the same transaction violates the DDD design principle, and heavyweight distributed transactions (also called XA transactions or global transactions) need to be used between two different systems. In addition, this approach also creates a strong coupling between the order system and the account system. By introducing domain events, we can solve the above problems well. In general, domain events bring us the following benefits:
  1. Decoupled Microservices (Bounded Contexts)
  1. Helps us to deeply understand the domain model
  1. Providing data sources for audits and reports
  1. Towards Event Sourcing , CQRS, and More
Taking the e-commerce website above as an example, when the user places an order, the order system will issue a domain event "user has placed an order" and publish it to the message system, and the order is now complete. The account system subscribes to the "user has placed an order" event in the message system, processes the event when it arrives, extracts the order information in the event, and then calls its own points engine (or another microservice) to calculate the points and finally update the user's points. It can be seen that after the order system sends the event, the entire use case operation is completed, and there is no need to care who received the event or what was done with the event. The consumer of the event can be the account system or any third party interested in the event, such as the logistics system. As a result, the coupling relationship between the microservices is untied. It is worth noting that at this time, the microservices are no longer strongly consistent, but based on the final consistency of the event.

Event Storming is a team activity that aims to identify aggregate roots through domain events and then divide the bounded context of microservices. In the activity, the team first lists all domain events in the domain through brainstorming, integrates them to form the final set of domain events, and then for each event, marks the command that caused the event, and then marks the role of the command initiator for each event. The command can be initiated by the user, a third-party system call, or a timer trigger. ***Classify the events to sort out the aggregate roots and bounded contexts. Another additional benefit of event storming is that it can deepen the participants' understanding of the domain. It should be noted that domain experts must be present in event storming activities.

[[196315]]

Creating Domain Events Domain events should answer questions like "who did what and when?" In actual coding, you can consider using layer supertypes to include some common attributes of events: [[196316]]
public abstract class Event {
    private final UUID id;
    private final DateTime createdTime;

    public Event() {
        this.id = UUID.randomUUID();
        this.createdTime = new DateTime();
    }
}
[[196316]] You can see that domain events also contain IDs, but this ID is not an ID concept at the entity level, but is mainly used for event tracing and logging. In addition, since domain events describe things that happened in the past, we should model domain events as immutable. From the DDD concept, domain events are more like a special value object. For the coffee shop example mentioned above, create the "Customer has arrived" event as follows: [[196316]]
public final class CustomerArrivedEvent extends Event {
    private final int customerNumber;

    public CustomerArrivedEvent(int customerNumber) {
        super();
        this.customerNumber = customerNumber;
    }
}
[[196316]] In this CustomerArrivedEvent event, in addition to the attributes inherited from Event, a business attribute closely related to the event is also customized - the number of customers (customerNumber) - so that subsequent operations can reserve the corresponding number of seats. In addition, we declare all attributes and CustomerArrivedEvent itself as final, and do not expose any methods that may modify these attributes to the outside, thus ensuring the invariance of the event. Publishing domain events When using domain events, we usually use the "publish-subscribe" approach to integrate different modules or systems. Within a single microservice, we can use domain events to integrate different functional components. For example, in the example of "sending a welcome email to the user after user registration" mentioned above, the registration component sends an event, and the email sending component sends an email to the user after receiving the event.

When using domain events within a microservice, we do not necessarily have to introduce message middleware (such as ActiveMQ, etc.). Taking the above "sending a welcome email after registration" as an example, although the registration behavior and the email sending behavior are integrated through domain events, they still occur in the same thread and are synchronous. It should also be noted that when using domain events within a bounded context, we still need to follow the principle of "one transaction only updates one aggregate root". Violating it often means that our splitting of aggregate roots is wrong. Even if such a situation does exist, different transactions should be used for different aggregate roots in an asynchronous way (message middleware needs to be introduced at this time). At this time, you can consider using background tasks. In addition to being used within microservices, domain events are more often used to integrate different microservices, such as the "e-commerce order" example above.

Usually, domain events are generated in domain objects, or more precisely, in aggregate roots. In specific coding implementation, there are multiple ways to publish domain events. A direct way is to directly call the Service object that publishes the event in the aggregate root. Taking the "e-commerce order" in the above text as an example, when an order is created, the "order created" domain event is published. At this time, you can consider publishing the event in the constructor of the order object: [[196316]]
public class Order {
    public Order(EventPublisher eventPublisher) {
        //create order        
        //…        
        eventPublisher.publish(new OrderPlacedEvent());    
        }
}
[[196316]] Note: In order to focus on event publishing, we have simplified the Order object. The Order object itself is not referenced in actual coding. As you can see, in order to publish the OrderPlacedEvent event, we need to pass in the Service object EventPublisher. This is obviously an API pollution, that is, Order as a domain object only needs to focus on business-related data, rather than infrastructure objects such as EventPublisher. Another method was proposed by Udi Dahan, the founder of NServiceBus, which is to publish domain events in the domain object by calling static methods on EventPublisher: [[196316]]
public class Order {
    public Order() {
        //create order
        //...
        EventPublisher.publish(new OrderPlacedEvent());
    }
}
[[196316]] Although this method avoids API pollution, the publish() static method here will have side effects, which makes it difficult to test the Order object. At this point, we can improve it by "temporarily saving domain events in the aggregate root": [[196316]]
public class Order {

    private List<Event> events;

    public Order() {
        //create order
        //...
        events.add(new OrderPlacedEvent());
    }

    public List<Event> getEvents() {
        return events;
    }

    public void clearEvents() {
        events.clear();

    }
}
[[196316]] When testing the Order object, you can verify the events collection to ensure that the Order object actually publishes the OrderPlacedEvent event when it is created: [[196316]]
@Test
public void shouldPublishEventWhenCreateOrder() {
    Order order = new Order();
    List<Event> events = order.getEvents();
    assertEquals(1, events.size());
    Event event = events.get(0);
    assertTrue(event instanceof OrderPlacedEvent);
}
[[196316]] In this way, the aggregate root can only save domain events temporarily. After the operation on the aggregate root is completed, we should publish the domain events and clear the events collection in time. You can consider doing this when persisting the aggregate root, which is the repository in DDD: [[196316]]
public class OrderRepository {
    private EventPublisher eventPublisher;

    public void save(Order order) {
        //save the order
        //...
        List<Event> events = order.getEvents();
        events.forEach(event -> eventPublisher.publish(event));
        order.clearEvents();
    }
}
[[196316]] In addition, there is another approach similar to "temporarily saving domain events", which is to "return domain events directly in the aggregate root method" and then publish them in the Repository. This approach still has good testability, and developers do not need to manually clear the previous event collection, but they still have to remember to publish the events in the Repository. In addition, this approach is not suitable for scenarios where aggregate roots are created, because the creation process at this time must return both the aggregate root itself and the domain event. This approach also has disadvantages, such as requiring developers to remember to clear the events collection every time they update the aggregate root. Forgetting to do so will cause serious bugs in the program. However, despite this, this is still the approach I recommend. Atomicity of business operations and event publishing Although we use eventual consistency based on domain events between different aggregate roots, we still need to use strong consistency between business operations and event publishing, that is, the occurrence of these two should be atomic, either all succeed or all fail, otherwise eventual consistency is out of the question. Taking the "order points" in the above text as an example, if the customer successfully places an order but the event fails to be sent, the downstream account system will not get the event, resulting in the customer's points not increasing. To ensure the atomicity between business operations and event publishing, the most direct way is to use XA transactions, such as JTA in Java. This method is not favored by people due to its heavyweight. However, for some systems that do not require so much performance, this method is not an option. Some development frameworks can already support XA transaction managers independent of application servers (such as Atomikos and Bitronix). For example, Spring Boot, as a microservice framework, provides support for Atomikos and Bitronix. If JTA is not your option, you can consider using the event table method. This method first saves the event to the database where the aggregate root is located. Since the event table and the aggregate root table belong to the same database, the whole process only requires one local transaction to complete. Then, read the unpublished events in the event table in a separate background task and publish the event to the message middleware.

There are two issues to note in this approach. The first is that after the event is published, the event in the table needs to be marked as "published", which still involves operations on the database. Therefore, atomicity is required between publishing the event and marking it "published". Of course, XA transactions can still be used at this time, but this goes against the original intention of using the event table. One solution is to create the consumer of the event as idempotent, that is, the consumer can consume the same event multiple times. This process is roughly as follows: the event sending and database update use their own transaction management throughout the process. At this time, it is possible that the event is successfully sent but the database update fails. In this way, in the next event publishing operation, since the previously published event is still in the "unpublished" state in the database, the event will be republished to the message system, resulting in event duplication, but since the event consumer is idempotent, there will be no problem with event duplication. Another issue that needs attention is the choice of persistence mechanism. In fact, for aggregate roots in DDD, NoSQL is a more suitable choice than relational databases. For example, using MongoDB's Document to save aggregate roots is a very natural way. However, most NoSQL do not support ACID, which means that the atomicity between aggregate updates and event publishing cannot be guaranteed. Fortunately, relational databases are also moving towards NoSQL. For example, the new versions of PostgreSQL (version 9.4) and MySQL (version 5.7) can already provide JSON storage and JSON-based queries with NoSQL characteristics. At this time, we can consider serializing the aggregate root into JSON format data for storage, thereby avoiding the use of heavyweight ORM tools and ensuring ACID between multiple data. Why not? In summary , domain events are mainly used to decouple microservices, at which time the microservices will form eventual consistency. Event storming activities help us split microservices and help us gain a deeper understanding of a certain field. Domain events, as historical data that has already occurred, should be created as immutable special value objects when modeling. There are many ways to publish domain events, among which the method of "temporarily saving domain events in aggregates" is worthy of praise. In addition, we need to consider the atomicity between aggregate updates and event publishing. We can consider using XA transactions or using separate event tables. In order to avoid the problems caused by event duplication, the best way is to create the consumer of the event as idempotent.

<<:  Technical Tips | How to build an efficient operation and maintenance management platform under the microservice architecture?

>>:  A better way to visualize microservice architectures

Recommend

Growth strategy for advertising monetization: splash screen ads

When I was searching for relevant information on ...

8 short video transaction scripts

You may have spent thousands of dollars listening...

Tooth-stuck warning! Be careful when buying these types of mangoes!

(Pictures are from the Internet) Mangoes are not ...

Can eating chili peppers speed up the recovery of oral ulcers?

Recently, some netizens claimed that if you want ...

Commercial Photo Editing All-round Course 2

Course Catalog: ├──Section 01-Image Preprocessing...

How much does it cost to make a gardening applet in Guyuan?

How much does it cost to develop the Guyuan Garde...

He has been gone for many years, but we still miss him...

If you were a drop of water Have you moistened an...