Code generates invalid Json

[Yohan Weerasinghe]

I am trying to read a Json file (which contains a `JsonArray`) in `Amazon S3` using `Gson`, add extra 2 fields to every `JsonObject`, add the `JsonObject`s into a `JsonArray` and write the `JsonArray` back to a file.

Below is how the .json file I am reading looks like.

[ { "hash_index": "00102x05h06l0aj0dw", "body": "Who's signing up for Obamacare?", "_type": "ArticleItem", "title": "Who's signing up for Obamacare? - Jan. 13, 2014", "source": "money.cnn.com", "primary_key": 0, "last_crawl_date": "2014-01-14", "url": "http://money.cnn.com/2014/01/13/news/economy/obamacare-enrollment/index.html" }, { "hash_index": "00102x05h06l0aj0dw0iz0kn0l@0t#0", "body": "Who's signing up for Obamacare?", "_type": "ArticleItem", "title": "Who's signing up for Obamacare? - Jan. 13, 2014", "source": "money.cnn.com", "primary_key": 1, "last_crawl_date": "2014-01-14", "url": "http://money.cnn.com/2014/01/13/news/economy/obamacare-enrollment/index.html" } ]

Below is my code, trying to do everything explained above

    //Create the Hashed JSON
    public void createHash() throws IOException
    {
    	System.out.println("Hash Creation Started");
    	
    	strBuffer = new StringBuffer("");
    	
    	
    	try
    	{
    		//List all the Buckets
    		List<Bucket>buckets = s3.listBuckets();
    		
    		for(int i=0;i<buckets.size();i++)
    		{
    			System.out.println("- "+(buckets.get(i)).getName());
    		}
    		
    		
    		//Downloading the Object
    		System.out.println("Downloading Object");
    		S3Object s3Object = s3.getObject(new GetObjectRequest(inputBucket, inputFile));
    		System.out.println("Content-Type: "  + s3Object.getObjectMetadata().getContentType());
    		
    		
    		
            
            JsonReader reader = new JsonReader( new BufferedReader(new InputStreamReader(s3Object.getObjectContent())) );
            reader.beginArray();
            int gsonVal = 0;
            
            List><JsonObject>jsonObjectHolder = new ArrayList();
            List<String> jsonHashHolder = new ArrayList();
            
            while (reader.hasNext()) {
                JsonParser  _parser = new JsonParser();
                JsonElement jsonElement =  _parser.parse(reader);
                JsonObject jsonObject1 = jsonElement.getAsJsonObject();
                //Do something
                
                
            
            	StringBuffer hashIndex = new StringBuffer("");
            	
            	//Add Title and Body Together to the list
            	String titleAndBodyContainer = jsonObject1.get("title")+" "+jsonObject1.get("body");
            	
            	
            	//Remove full stops and commas
            	titleAndBodyContainer = titleAndBodyContainer.replaceAll("\\.(?=\\s|$)", " ");
            	titleAndBodyContainer = titleAndBodyContainer.replaceAll(",", " ");
            	titleAndBodyContainer = titleAndBodyContainer.toLowerCase();
            	
            		        	 
            	//Create a word list without duplicated words
            	StringBuilder result = new StringBuilder();
            	
            	
            	HashSet<String> set = new HashSet<String>();
            	for(String s : titleAndBodyContainer.split(" ")) {
            	    if (!set.contains(s)) {
            	        result.append(s);
            	        result.append(" ");
            	        set.add(s);
            	    }
            	}
            	//System.out.println(result.toString());
            	
            	
            	String[]finalWordHolder = (result.toString()).split(" ");
            	Arrays.sort(finalWordHolder);
            	
            	
            	//Navigate through text and create the Hash
            	for(int arrayCount=0;arrayCount<finalWordHolder.length;arrayCount++)
            	{
            		
            		        		
            		if(wordMap.containsKey(finalWordHolder[arrayCount]))
            		{
            			hashIndex.append((String)wordMap.get(finalWordHolder[arrayCount]));
            		}
            		
            	}
            	
            	//System.out.println(hashIndex.toString().trim());
            	
            	jsonObject1.addProperty("hash_index", hashIndex.toString().trim()); 
            	jsonObject1.addProperty("primary_key", gsonVal); 
            	jsonObjectHolder.add(jsonObject1); //Add the JSON Object to the JSON collection
            	
            	jsonHashHolder.add(hashIndex.toString().trim());
            	
            	System.out.println("Primary Key: "+jsonObject1.get("primary_key"));
            	
            	if(gsonVal==100)
            	{
            		System.out.println("10000 Reached. Writing now....");
            		
            		writeNewJsonFile(jsonObjectHolder,gsonVal);
            		
            		File hashFile = new File("Hash.txt");
            		FileWriter hashFileWriter = new FileWriter(hashFile,true);
            		
            		String str = "";
                	
                	for(int i=0;i<jsonHashHolder.size();i++)
                	{
                		hashFileWriter.write(jsonHashHolder.get(i));
                		hashFileWriter.write("\r\n");
                	}
                	
            		
                	hashFileWriter.close();
                	
        	        break;
            	}
            	
            	gsonVal++;
            }
            
            System.out.println("Hash Creation Completed");
        
    	}
    	catch(Exception e)
    	{
    		e.printStackTrace();
    	}
    }
    
    
    
      
    
    //Write the new JSON File
    private void writeNewJsonFile(List><JsonObject> jsonObjectHolder, int number) throws IOException
    {
    	System.out.println("Starting to write the JSON File");
    	//Add everything into a JSONArray
    	JsonArray jsonArrayNew = new JsonArray();
    	
    	for(int i=0;i<jsonObjectHolder.size();i++)
    	{
    		System.out.println("inside array: "+i);
    		jsonArrayNew.add(jsonObjectHolder.get(i));
    	}
    	
    	System.out.println("outside array");
    	
    	ByteArrayInputStream input = new ByteArrayInputStream(jsonArrayNew.toString().getBytes());
    	
    	Long contentLength = Long.valueOf(jsonArrayNew.toString().getBytes().length);
    	
    	ObjectMetadata metaData = new ObjectMetadata();
    	metaData.setContentLength(contentLength);
    	
    	s3.putObject(outputBucket,outputFile+"_"+number,input,metaData);
    	
    	
    }

As you can see, I need to generate a .json file after 100 `JsonObjects` are collected. But my code generate invalid Json. It simply generate the following Json for the above code.


[ { "_type": "ArticleItem", "body": "NEW DELHI: The Aam Aadmi Party's (AAP's) ", "title": "GR Gopinath slams AAP's stand on FDI in retail - The Economic Times", "last_crawl_date": "2014-01-16", "url": "http://economictimes.", "hash_index": "00101v0ji0k9", "primary_key": 0 }, { "_type": "ArticleItem", "body": "Japan’s ties with India do n", "title": "Politics is now | The Indian Express", "last_crawl_date": "2014-01-16", "url": "http://indianexpress.com", "hash_index": "00102x05q0cc0ji0uw0vc", "primary_key": 1 }, { "_type": "ArticleItem", "body": "SAGITTARIUS NOV 23-DEC 22. ", "title": "Daily | Astrotype | The Indian Express", "last_crawl_date": "2014-01-16", "url": "http://indianexpress.com/astrotype/daily/", "hash_index": "00102x0ae0b10fs0h~0i20k80r&", "primary_key": 2 }, { "_type": "ArticleItem", "body": "Bad corporate news sends stocks lower.", "title": "Bad corporate news sends stocks lower - Jan. 16, 2014", "last_crawl_date": "2014-01-16", "url": "http://money.cnn.com", "hash_index": "00102x0b00ji0j#0k90r", "primary_key": 3 }, { "_type": "ArticleItem", "body": "Related. 2008 Batla House encounter ", "title": "Batla House encounter: HC denies bail", "last_crawl_date": "2014-01-16", "url": "http://indianexpress.com", "hash_index": "0010ae0k90t#0vc", "primary_key": 4 }, { "_type": "ArticleItem", "body": "Related. A three-judge bench ", "title": "Supreme Court reserves order on scrapping of coal blocks allocations", "last_crawl_date": "2014-01-16", "url": "http://indianexpress.com", "hash_index": "00106j0aj0f$0ji0md0r&", "primary_key": 5 }, { "_type": "ArticleItem", "body": "Lew: Debt ceiling ", "title": "Lew: Debt ceiling crunch to come sooner than thought - Jan. 16, 2014", "last_crawl_date": "2014-01-16", "url": "http://money.cnn.com", "hash_index": "00106v0ae0cc0", "primary_key": 6 }, { "_type": "ArticleItem", "body": "Because apparently Americans do", "title": "Hershey's to make 3-D chocolate printer - Jan. 16, 2014", "last_crawl_date": "2014-01-16", "url": "http://money.cnn.com", "hash_index": "00102x05y0", "primary_key": 7 }, { "_type": "ArticleItem", "body": "Time Warner to sell HQ for $1.3 billion.", "title": "Time Warner to sell HQ for $1.3 billion - Jan. 16, 2014", "last_crawl_date": "2014-01-16", "url": "http://money.cnn.com", "hash_index": "00102102x09i", "primary_key": 8 }, { "_type": "ArticleItem", "body": "5 things we want for Windows 9.", "title": "5 things we want for Windows 9 - Jan. 16, 2014", "last_crawl_date": "2014-01-16", "url": "http://money.cnn.com", "hash_index": "00101v021", "primary_key": 9 }, { "_type": "ArticleItem", "body": "It was a rough morning for actors in ", "title": "The 6 biggest Oscar nomination snubs | New York Post", "last_crawl_date": "2014-01-16", "url": "http://nypost.com", "hash_index": "0010", "primary_key": 10 }, { "_type": "ArticleItem", "body": "Sen. Jay Rockefeller", "title": "‘Zero evidence’ of ‘legitimate traffic study’ in Bridgegate | New York Post", "last_crawl_date": "2014-01-16", "url": "http://nypost.com", "hash_index": "00105

As you can see, it is invalid and it does not have 100 `JsonObject`s. What have I done wrong here? Please help. It is invalid because it does not close the JsonObject and JsonArray properly with brackets.

Is there no one in SO to help this problem?

PS: Please note that I made this question in here as well, but I did not get any valuable answer.>

[Paul Clapham]

I don't see any code which writes JSON, so it's kind of hard to comment on that code. But if your problem is that the end of the expected data is missing, chances are that the code writing the data failed to close the file (or output stream or whatever).

[Yohan Weerasinghe]

Thanks for the reply. I can post the complete code if you can give some hand

[Paul Clapham]

I don't understand what kind of help you need to post the appropriate code.

Actually... what you need to do is to explain the existing code. What's a "JsonArray"? What is this mysterious "s3" variable which seems to be in charge of everything? If you're going to use classes which aren't part of the standard Java API then the least you can do is to explain them. Or post a link to their documentation. Or something; don't just assume that somebody is going to know all about them.