Team ID - c_uparog
- Arshdeep Singh Syal (asyal) {SonarLint}
- Jubeen Shah (jnshah2) {JMeter Testing + Community Statistics}
- Rayan Dasoriya (rdasori) {JMeter Testing}
- Sujal (ssujal) {Gatling Testing}
- Introduction
- Brokers Compared
- Resources Used
- Setting up the Environment
- Implementation Details
- Results
- Challenges
- Conclusion
Message queueing brokers are the so-called translators in the world of computer networks that are responsible for the translation of messages from the messaging protocol being used by the sender to the messaging protocol being used by the receiver and are the building blocks of the message-oriented middleware (MOM).
There are several message brokers out there, however, we have chosen three out of the bunch.
- Performance Testing
- Bugs and Errors Inspection
- Community and other Metrics
In this section, we briefly outline the steps needed to carry out the installation for the respective message brokers and tools. Clicking on each link would take you to the series of steps (compiled in a .pdf in this repo) that we performed to install them. Please note that we used the following configuration on our end, while performing the tests.
MacBook Pro, running macOS Mojave Version 10.14.2 Beta with a 2.7 GHz Intel Core i7 processor with 16 GB 2133 MHz LPDDR4 memory
- Apache ActiveMQ
- RabbitMQ
- Apache Kafka
- Gatling Installation
- Gatling Plugins and Extensions
- Installing the Tools
For specific configuration of the message brokers with JMeter, you can click on the following links to get a detailed description of the steps we followed to setup the producers and consumers with JMeter.
-
Message Throughput Comparision
-
Latency
-
RabbitMQ Testing using johnlonganecker/rabbitmq-performance-app repository
-
Kafka testing using custom scripts
-
The below image is one of the plots for the above log files, the rest of the images can be found here
-
SonarLint
-
FindBugs
In this section we briefly outline a few results that we gathered from studying the community as stated in the Resources Used section, however, the full documentation can be found here.
- RabbitMQ is the most popular in the industry, despite Kafka having better performance.
- This can be because, Kafka was late to the market, and by then RabbitMQ had already taken over the market share from ActiveMQ
- This can be because a majority of the companies that were previously using ActiveMQ found it very complex.
- The switching costs associated with RabbitMQ are very low
- It is simple, flexible, and has several tool integrations available
- Based on commits per year for each of the message queuing services, it can be seen that the community for ActiveMQ is becoming less and less active over the years, being the most active in the year 2006, and the least active being this year (2018), which could be correlated with the decline in popularity of the broker amongst developers and organizations alike.
- RabbitMQ’s community seemed to be the most active during the years 2009 through 2016. After which it was swiftly taken over by Kafka.
- It is interesting to know that the number of commits per year for Kafka increased at an average rate of 61% from
2011 to 2017 - For the same period, RabbitMQ had a growth rate -12%, with ActiveMQ having a growth rate of -6%.
- These results further concretely help us understand the reason for growing popularity amongst developers and organizations and their tendency of moving towards Kafka, thereby resulting in a slightly higher growth rate in adoption.
- Looking at the Google internet search for the topics directly related to RabbitMQ, Kafka, and ActiveMQ it can clearly be seen that In the past year, the most popular message queueing service has been Kafka.
- This fact can be further supported by the fact that Kafka has the highest number of stars (amongst the three) on Github, translating to very high preference amongst developers.
- This statistic is important to know, as it can help us with the
Growth Trendtowards a particular message queuing broker, which should be higher for Kafka, given its increased popularity amongst developers. - For example, a comparison is made between the industrial usage of the message queuing broker, as it was at the beginning of the semester and as it can be seen in the image below.
- As hypothesized, the growth in the number of companies using Kafka has 1% more increase than in RabbitMQ. This has resulted in more tools being developed for the integration with Kafka.
- According to our understanding, Gatling was not able to find the
ClockSingletonclass in the current release of Kafka server. We have raised the same issue on GitHub repository of the plugin we were trying to implement but haven’t been able to resolve it yet. Moreover, one possible way of implementing both the servers could be hosting an API on the servers and then exposing them using Gatling to test, which we did not implement due to time constraints. - The extension was built on
gatling 2.2and the current version is3.0series. Moreover, the Apache Kafka server needed for the plugin to build is0.10. We were able to start the latest2.12release of Kafka. Same was the case with RabbitMQ - The version compatibility between SBT, Java, Scala and the producer API was extremely tedious and it was quite cumbersome.
RabbitMQ is currently the most favored amongst the industry, but there is a shift in affinity towards Kafka, both from the perspective of developers and the industry adoption. The rate at which Kafka is growing is much higher than RabbitMQ, which in contrast seems to be slowly declining its growth rate. ActiveMQ is the least favored from both the developers and industrial perspective given the low industry adoption and developers retention rate. So, if a new developer wishes to contribute to a community, we would recommend contributing to the Kafka community, because of its high rate of activity, retention, support and overall clarity in the documentation. If however, a developer wants to start learning about message brokers, Message Oriented Middleware, and its implementation we personally found ActiveMQ to be a good starting point and then transitioning towards Kafka. RabbitMQ would require a higher learning curve if the developer is unfamiliar with Erlang.
Talking about the performance, Apache Kafka gave the best performance with a very high throughput and a low latency rate. ActiveMQ is preferred over Kafka when traditional enterprise messaging is taken into consideration, however, RabbitMQ does a much better job at throughput, latency and overall community support than ActiveMQ. Kafka, because of its low latency, and very high throughput, fault-tolerance, and its highly distributed architecture is most useful in stream processing, event sourcing, commit log and log aggregation, and traditional messaging. RabbitMQ would be more useful in pub-sub messaging, request-response messaging, and also act as an underlaying layer for IoT applications. Hence, depending on the specific use-case you can choose either RabbitMQ or Kafka.
You can view the Detailed Report for more comprehensive information about the project. Also, you can view the Project Report for a concise summary of the comparative analysis of the message brokers. Finally, the presentation is linked for your reference.