Is the volume of data it is meant for, the only main difference between Kafka and Messaging Queues

[Monica Shiralkar]

Both Kafka and Messaging queues are on Producer Consumer model where consumer subscribes to Producer and then consumes data from it. Is it the only difference that Kafka is for ingesting huge volume at low latency or there is some other main difference too?

[Himai Minh]

Yes. Kafka supports producer and consumer model. I also supports joining streams or table or  GlobalKTable.
It also supports KSQL to query from the streams or joined streams/table.

[Henry Wong]

Monica Shiralkar wrote:Is it the only difference that Kafka is for ingesting huge volume at low latency or there is some other main difference too?

In my opinion, the environments have gotten to a point, where they pretty much have equivalents of each other's features. I believe Kafka supports Pub/Sub in addition to Producer/Consumer, just like with various MQ products. And there are messaging environments that support huge volumes at low latency, such as Ultra Messaging (INFA) or FTL (Tibco). Even traditional message queues have gotten better in this regard.

I am going to predict that it will be harder and harder to make distinctions on which environment to choose in the future.

[Monica Shiralkar]

Thanks.

Henry Wong wrote:I believe Kafka supports Pub/Sub in addition to Producer/Consumer.

Based on what exactly can we say that Kafka supports pub/sub. For producer-consumer I know, but for pub-sub?

[Henry Wong]

Monica Shiralkar wrote:
Based on what exactly can we say that Kafka supports pub/sub. For producer-consumer I know, but for pub-sub?

Pub/Sub is the classic topic based messaging -- meaning every subscriber on the topic gets every message from every publisher on the  same topic. Producer/Consumer is the classic queue based messaging -- meaning all the messages from all the producers are queued, and then any message will only be delivered to only one consumer.

Pub/Sub are generally used for stuff like market data, where everyone needs to see everything. And queuing is good for stuff like orders, where only one consumer should be processing the order. Using the stock market as an example, when you want to see if you would like to buy/sell a stock, you want to see all executed trades on all markets to decide at what price you want to buy/sell at. However, when you actually buy or sell the stock, you only want one of the markets to take the order -- since you only want to make one trade.

IIRC, I thought Kafka supported both.

Henry