• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Help with MapReduce input

 
Michael Hagar
Greenhorn
Posts: 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello, I am new Hadoop mapreduce, as I've only edited a basic WordCount program. I need my current MapReduce program to first read an input file(in some kind of initialization phase?) to produce multiple <key, value> pairs which are sent to the mapper, have the mapper output <key, value> pairs, and then do some aggregation on them in the reducer. After that, I want to feed those results back into the mapper and repeat for a set amount of iterations. I've read some stuff on Input Splits, but I'm not really sure to go about doing this. Any help is appreciated.
 
amit punekar
Ranch Hand
Posts: 544
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,
You could certainly do this using plain Map-Reduce tasks and chaining them together. However, it may be easier to do this using Cascading or Pig.

Regards,
amit
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic