A series of tiny examples around Kafka and event driven microservices using Spring Framework written in Kotlin & Java.
The world of software development is currently experiencing a radical change. Technologies such as Docker, Kubernetes or Kafka, but also paradigms such as event driven microservices are being used more and more widely. Not least because companies like Google, Facebook, LinkedIn or Amazon have achieved a great competitive advantage in recent years.
The great advantage of these technologies and the new approaches lies in their scaling. This does not necessarily mean the runtime performance of an application, but rather the Scaling the business model. Above all, the scaling of the actual software development process. The point is that you no longer have to develop a platform with ONE LARGE TEAM. Instead, many smaller teams can work in parallel and without any major arrangements bring different parts of an application into production very quickly and effectively and operate them there.
Decentralization and loose coupling bring the advantage not only in the code or in the infrastructure operation, but above all in the organization.
One of the reasons why this project was started, because it is very difficult to find really simple Microservices + Kafka examples. Most of the code examples are very complex and try to explain the use of Kafka in depth. All the details are of course important to be able to master a subject well at the end of the day. However, this level of detail makes difficult to get started. This project does not claim to fully explain the world of Kafka in its epic depth of detail.
This project aims to be a smooth and easy entry into the wonderful world of "event driven microservices with Kafka". Namasté.🧘
The goal of this project is to gain technical understanding in development of event driven microservices with Kafka.
these are the specific goals:
- Basic understanding of the Kafka system
- understanding of Stream processing.
- Understanding of the event driven microservice approach using Kafka.
- implementation of architectural patterns(CQRS, ES, SAGA...) made using kafka
The following implementation examples are built iteratively so that you can reach the big goal with small steps.
This repo contains several projects.
setting up a local kafka playground. More about the Kafka ecosystem.
a simple publish/subscribe sample using Kafka + Spring
a simple Unit-Test using Kafka + Spring + JUnit5 + EmbeddedKafka/KafkaContainer
a simple Kafka data streaming example using Kafka + Spring
Kafka Streams Api is not an additional runtime component of the Kafka system itself, but rather a library + architectural pattern. There is a separate lib or several different implementations for each technology stack(Java, Node/TypeSCript, Python..)
a simple event driven microservices example using Kafka + Spring + H2
There are 2 essential points that make up event-driven microservices:
- Each service has its own database
- The services do not call each other via. REST or HTTP. Instead, they only communicate with each other via events.
This guarantees maximum decoupling. This decoupling/autonomy makes an event driven microservice to a more maintainable microservice. Kafka is ideally suited as an event bus to ensure communication between the microservices.
a minimal CQRS microservice implementation using Kafka + Spring + H2
The purpose of CQRS (Command-Query-Responsibility-Segregation) lies in the decoupling of read and write operations. The communication between the decoupled Command- and Query-Services takes place by means of events.
a minimal Event Sourcing implementation using Kafka + Spring
event sourcing makes microservices more scalable and more resilient. And all of that by design. Kafka can be used excellently as an event store.
another event sourcing example using Streams API. In this example, the state is not stored in a local DB, but in a Kafka State Store.
a minimal SAGA pattern implementation using Kafka + Spring
...coming soon
a minimal Kafka Connect implementation using Kafka + Spring
...coming soon
a minimal stream processing example using Kafka ksqlDB
...coming soon
a minimal kafka security examples
...coming soon
You are invited to contribute to the project.
write me on twitter.
i ❤️ MIT