Register / Login
this forum made possible by our volunteer staff, including ...
Stephan van Hulst
Why in hadoop source code, he used function JobSplitWriter.createSplitFiles(...)?
posted 2 years ago
while I am studying the part of splitting the input file into splits in hadoop code , I found that after he splits the input file he used that function :
950 JobSplitWriter.createSplitFiles(jobSubmitDir, conf, 951 jobSubmitDir.getFileSystem(conf), array);
and that function created 4 files in the harddisk : job.split , job.splitmetainfo , job.split.crc and job.splitmetainfo.crc
and when I open the class JobSplitWriter code, there is a comment : The class that is used by the Job clients to write splits (both the meta and the raw bytes parts)
I don't understand what is the use of this function ?
Boost this thread!
Something I can't understand in hadoop source code?
How to Write a MapReduce Program Using the Hadoop to find out positive and negative comments
how to use SequenceFileAsTextInputFormat in hadoop map?
Passed CCD-410 Hadoop Developer with 88%
Hadoop(Beginner Level Question)