• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Is the volume of data it is meant for, the only main difference between Kafka and Messaging Queues

 
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Both Kafka and Messaging queues are on Producer Consumer model where consumer subscribes to Producer and then consumes data from it. Is it the only difference that Kafka is for ingesting huge volume at low latency or there is some other main difference too?
 
Bartender
Posts: 2419
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes. Kafka supports producer and consumer model. I also supports joining streams or table or  GlobalKTable.
It also supports KSQL to query from the streams or joined streams/table.
 
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Monica Shiralkar wrote:Is it the only difference that Kafka is for ingesting huge volume at low latency or there is some other main difference too?



In my opinion, the environments have gotten to a point, where they pretty much have equivalents of each other's features. I believe Kafka supports Pub/Sub in addition to Producer/Consumer, just like with various MQ products. And there are messaging environments that support huge volumes at low latency, such as Ultra Messaging (INFA) or FTL (Tibco). Even traditional message queues have gotten better in this regard.

I am going to predict that it will be harder and harder to make distinctions on which environment to choose in the future.
 
Monica Shiralkar
Ranch Hand
Posts: 2925
13
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks.

Henry Wong wrote:I believe Kafka supports Pub/Sub in addition to Producer/Consumer.


Based on what exactly can we say that Kafka supports pub/sub. For producer-consumer I know, but for pub-sub?
 
Henry Wong
author
Posts: 23951
142
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Monica Shiralkar wrote:
Based on what exactly can we say that Kafka supports pub/sub. For producer-consumer I know, but for pub-sub?



Pub/Sub is the classic topic based messaging -- meaning every subscriber on the topic gets every message from every publisher on the  same topic. Producer/Consumer is the classic queue based messaging -- meaning all the messages from all the producers are queued, and then any message will only be delivered to only one consumer.

Pub/Sub are generally used for stuff like market data, where everyone needs to see everything. And queuing is good for stuff like orders, where only one consumer should be processing the order. Using the stock market as an example, when you want to see if you would like to buy/sell a stock, you want to see all executed trades on all markets to decide at what price you want to buy/sell at. However, when you actually buy or sell the stock, you only want one of the markets to take the order -- since you only want to make one trade.

IIRC, I thought Kafka supported both.

Henry
reply
    Bookmark Topic Watch Topic
  • New Topic