• Post Reply Bookmark Topic Watch Topic
  • New Topic

Unit Test a Sample Kafka Consumer and returned messages for a topic  RSS feed

 
Geane Norm
Ranch Hand
Posts: 35
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi, I am extremely new to Kafka and I have just written a Kafka Consumer following the example here: https://www.packtpub.com/books/content/writing-consumers
Kafka is running on a EC2 instance and I would like to test that my consumer is actually returning the messages from a topic as created by the Producer. I have been trying to understand how to Unit Test the Consumer app and came across this: https://ransilberman.wordpress.com/2013/07/19/how-to-unit-test-kafka/ that says that the Kafka API comes with an embedded Kafka server that can easily be used for unit tests. However, I am not sure I need to go down that road in order to test the outputted messages from a particular topic. Can you help? Is there an easy way to test the returned topic messages?

Thank you so much,

I.

[EDIT]

I am expecting my message in a JSON format, do I need a serializer to read them?
 
Junilu Lacar
Sheriff
Posts: 11494
180
Android Debian Eclipse IDE IntelliJ IDE Java Linux Mac Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would try to stop thinking about the Kafka infrastructure altogether. The point of unit testing is to see behavior in isolation. If you're going to unit test, then isolate your class under test (CUT). Use the test to say something like this: "Ok, I have this CUT, and if it were actually hooked up to Kafka, this is how it would be called..."

In code, that would look like this:

Whatever code would go where line 6 is, that should not include actually setting up Kafka, etc. That code would just do things to make your testSubject "think" that it's getting real input from the real Kafka.

It's kind of the same thing I taught my son. If you have a program that needs the user to enter something, don't test by asking the user to enter something. Instead, write your test so that it pretends that it's using something that the user entered. That way, you take the User out of the test equation.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!