Win a copy of Node.js Design Patterns: Design and implement production-grade Node.js applications using proven patterns and techniques this week in the Server-Side JavaScript and NodeJS forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Ron McLeod
  • Rob Spoor
  • Tim Cooke
  • Junilu Lacar
Sheriffs:
  • Henry Wong
  • Liutauras Vilda
  • Jeanne Boyarsky
Saloon Keepers:
  • Jesse Silverman
  • Tim Holloway
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
Bartenders:
  • Al Hobbs
  • Mikalai Zaikin
  • Piet Souris

Which is the best format to transit messages among microservices?

 
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Brian!
First of all, wellcome!
I'd some interactions with microservices and kafka and a thing that I left for another opportunit was: Which is the best format to transit messages among microservices? String Json or Object?

p.s: fascinating book, I want!

Thank you in advance!
 
Author
Posts: 7
6
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the question! Like many things in programming, there are multiple ways to solve the same problem. The "right" way really depends on what outcomes you are looking to achieve. When you are dealing with messaging and events, I would recommend using the simplest approach possible that meets your needs. Start as simple and possible, then evolve when required to. YAGNI (https://martinfowler.com/bliki/Yagni.html) is a good idea to live by.  

Using a structured textual representation, like JSON, is a good way to convey information in a clear and concise way. However, it can be fairly verbose if you let it be. XML is another textual representation that can be used in these scenarios, but it can be even more verbose than JSON. With more verbosity comes larger payload sizes and less efficient throughput.

If you need to support a high volume of messages, then you want your events to be as small as possible. One common data format used with Kafka is Avro. It is a binary format for your data that can be mapped to the language of your choice. If you pick the Avro path, then you also need to develop schemas to help describe the data since the raw data will be in a binary format.

I would recommend the following to read more about Avro, and its benefits:
* https://avro.apache.org/docs/current/
* https://www.confluent.io/blog/avro-kafka-data/ (While this was published back in 2015, it does a great job of describing the benefits of using a binary data format like Avro)
* https://docs.confluent.io/platform/current/schema-registry/serdes-develop/serdes-avro.html
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic