Win a copy of The Java Performance Companion this week in the Performance forum!
    Bookmark Topic Watch Topic
  • New Topic

Reading a large JSON file in Java

 
Tanishka chandrasekaran
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Report post to moderator
Hi ,
Iam new to JSON and i need help in reading a large json file which is really huge and it contains pair of id and URL in a array as shown below.. i need to get this all id and URL from this file in my java code and need to perform operations . Can someone give me suggestions on how to read such a big file and store the contents in java code to process it further.
 
Ulf Dittmer
Rancher
Posts: 42968
73
  • Mark post as helpful
  • send pies
  • Report post to moderator
Is the question about how to consume JSON in general, or about a situation where the general approach is not fast enough due to data volume? If the latter, is it an actual problem, or do you fear it could become one?
 
Tanishka chandrasekaran
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Report post to moderator
Right now iam facing a memory issue due to reading a large json file from the url. Basically the file size is close to 90 mb .
I need to read this json file which just has one array WITH many ID,URL pairs in it .. I need to read each id and based on each id ,there is a file operation.
So i wanted to begin my code from scratch as something went wrong there. So i wanted in general the efficient way of parsing such a huge file and looping through each line for this ID and URL.
 
Frank Pavageau
Greenhorn
Posts: 11
  • Mark post as helpful
  • send pies
  • Report post to moderator
Use a JSON library providing a streaming mode, such as Jackson.
 
fred rosenberger
lowercase baba
Bartender
Posts: 12146
30
Chrome Java Linux
  • Mark post as helpful
  • send pies
  • Report post to moderator
is this question really different from this one? If not, please don't make duplicates, as it wastes everyone's time.
 
Tanishka chandrasekaran
Greenhorn
Posts: 14
  • Mark post as helpful
  • send pies
  • Report post to moderator
i have changed my code something like this:

My json file have to be read from a url through http . once i read the url ,open the url stream and i have to read line by line . its a json format file . Please help.
i have tried to read from the url as below:

InputStream is = new URL(url).openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(
is, Charset.forName("UTF-8")));
String line, results = "";
while ((line = reader.readLine()) != null) {
results += line;
}
reader.close();
is.close();
JSONObject json = new JSONObject(results);
JSONArray fileArray = json.getJSONArray("Documents");

then iam looping the Array again for each line to perform some actions.Any suggestions to improve the code here.
 
fred rosenberger
lowercase baba
Bartender
Posts: 12146
30
Chrome Java Linux
  • Mark post as helpful
  • send pies
  • Report post to moderator
You haven't said how this thread differs from the other, so I am closing this as a duplicate. Any follow-up posts should be in the other thread.
 
    Bookmark Topic Watch Topic
  • New Topic