How to check how much memory is being used

[Craig Oconnor]

I wrote a program (link below) that is really simple. reads a "map" and if it is a 1 it is land, and if it is 2 it is the water. I was supposed to do something simple for a challenge. However, I did not get an accept because there is a 1024MB limit for the application during runtime. You can see in my code I added a printout of how much memory is being used. I guess garbage collection is making this process more illusive.


TL;DR - How can I check how much memory is being used and where?


http://pastebin.com/ghNT4ND8

[Campbell Ritchie]

Welcome to the Ranch

Many people are reluctant to open pastebin files, so please post your code here. The 1024MB limit on heap size can be changed, if you have enough available memory. Garbage collection is very efficient, but you may suffer problems with Lists, Maps or similar data structures based on arrays. When the capacity is full, the array is replaced by a larger array. During the replacement process there will be two arrays which cannot (yet) undergo garbage collection, and that may overwhelm the heap memory capacity. I don't know whether that is the cause of your problem.

[Craig Oconnor]

Campbell Ritchie wrote:Welcome to the Ranch

Many people are reluctant to open pastebin files, so please post your code here. The 1024MB limit on heap size can be changed, if you have enough available memory. Garbage collection is very efficient, but you may suffer problems with Lists, Maps or similar data structures based on arrays. When the capacity is full, the array is replaced by a larger array. During the replacement process there will be two arrays which cannot (yet) undergo garbage collection, and that may overwhelm the heap memory capacity. I don't know whether that is the cause of your problem.

Thanks for the info! The problem is I want to optimize my code to fit within the 1024 heap size, and I am using byte arrays to keep the size down. I don't know how to make it less....


(edit) - I guess adding the arraylists did not help as they take a definitive chunk of memory from the start right? I will use byte arrays instead with a defined size...

Ok, here is the code:

import java.io.*;
import java.util.*;
import java.lang.*;
import java.lang.Runtime; 


public class coast {

   public static int coast = 0;
   public static boolean edge_found;


   static void waterCheck(int y, int x,byte[][] ar,boolean[][] bar,ArrayList<Integer> y_axis,ArrayList<Integer> x_axis){
      //tru through, if any exceptions found, this is not a river or lake.
     bar[y][x] = true;
     y_axis.add(y);
     x_axis.add(x);
     try {
         //top
         if ((ar[y-1][x] == 0) && (bar[y-1][x] != true)){
            waterCheck(y-1,x,ar,bar,y_axis,x_axis);
         }
     }
     catch(Exception e){
      edge_found = true;
     }

     try {
         //left
         if ((ar[y][x-1] == 0) && (bar[y][x-1] != true)){
            waterCheck(y,x-1,ar,bar,y_axis,x_axis);
         }
     }
     catch(Exception e){
      edge_found = true;
     }
     try {
         //right
         if ((ar[y][x+1] == 0) && (bar[y][x+1] != true)){
            waterCheck(y,x+1,ar,bar,y_axis,x_axis);
         }
     }
     catch(Exception e){
      edge_found = true;
     }

     try {
         //bottom
         if ((ar[y+1][x] == 0) && (bar[y+1][x] != true)){
            waterCheck(y+1,x,ar,bar,y_axis,x_axis);
         }
     }
     catch(Exception e){
      edge_found = true;
     }
   }



   static void landCheck(int y, int x,byte[][] ar,boolean[][] bar){
      //tru through, if any exceptions found, this is not a river or lake.
     bar[y][x] = true;
     try {
         //top
         if ((ar[y-1][x] == 1) && (bar[y-1][x] != true)){
            landCheck(y-1,x,ar,bar);
         }
         if (ar[y-1][x] == 0){
            coast++;
         }
     }
     catch(Exception e){
      coast++;
     }

     try {
         //left
         if ((ar[y][x-1] == 1) && (bar[y][x-1] != true)){
            landCheck(y,x-1,ar,bar);
         }
         if (ar[y][x-1] == 0){
            coast++;
         }
     }
     catch(Exception e){
      coast++;
     }
     try {
         //right
         if ((ar[y][x+1] == 1) && (bar[y][x+1] != true)){
            landCheck(y,x+1,ar,bar);
         }
         if (ar[y][x+1] == 0){
            coast++;
         }
     }
     catch(Exception e){
      coast++;
     }

     try {
         //bottom
         if ((ar[y+1][x] == 1) && (bar[y+1][x] != true)){
            landCheck(y+1,x,ar,bar);
         }
         if (ar[y+1][x] == 0){
            coast++;
         }
     }
     catch(Exception e){
      coast++;
     }
   }






   public static void main(String[] args) {
      try{

      	int mb = 1024 * 1024; 

      	Runtime instance = Runtime.getRuntime();
 
		System.out.println("***** Heap utilization statistics [MB] *****\n");
 
		// available memory
		System.out.println("Total Memory: " + instance.totalMemory() / mb);
 
		// free memory
		System.out.println("Free Memory: " + instance.freeMemory() / mb);
 
		// used memory
		System.out.println("Used Memory: "
				+ (instance.totalMemory() - instance.freeMemory()) / mb);
 
		// Maximum available memory
		System.out.println("Max Memory: " + instance.maxMemory() / mb);

         Scanner in = new Scanner(new File("text2.txt"));
         int n = in.nextInt();
         int m = in.nextInt();
         in.nextLine();

         String s;
         byte[][] ar = new byte[n][m];
         boolean[][] bar = new boolean[n][m];

         for (int y = 0; y < n; y++){
            s = in.nextLine();
            for (int x = 0; x < s.length(); x++) {
               ar[y][x] = (s.charAt(x) == '1') ? (byte) 1 : (byte) 0;
            }
         }
         //System.out.println(Arrays.deepToString(ar));

         //Begin test~
         for (int y = 0; y < n; y++){
            for (int x = 0; x < m; x++){
               if ((ar[y][x] == 0) && (bar[y][x] == false)){
                  //if water: if lake or river, convert to land
                  edge_found = false;
                  ArrayList<Integer> y_axis = new ArrayList<Integer>();
                  ArrayList<Integer> x_axis = new ArrayList<Integer>();

                  waterCheck(y,x,ar,bar,y_axis,x_axis);
                  if (edge_found == false){
                     //for all x and y values recorded, make them 1s
                     for (int k = 0; k < y_axis.size(); k++){
                        ar[y_axis.get(k)][x_axis.get(k)] = 1;
                     }
                  }

               }
            }
         }

         for (int y = 0; y < n; y++){
            for (int x = 0; x < m; x++){
               if ((ar[y][x] == 1) && (bar[y][x] == false)){
                  //if water: if lake or river, convert to land

                  landCheck(y,x,ar,bar);
               }
            }
         }

         System.out.println(coast);

         System.out.println("***** Heap utilization statistics [MB] *****\n");
 
		// available memory
		System.out.println("Total Memory: " + instance.totalMemory() / mb);
 
		// free memory
		System.out.println("Free Memory: " + instance.freeMemory() / mb);
 
		// used memory
		System.out.println("Used Memory: "
				+ (instance.totalMemory() - instance.freeMemory()) / mb);
 
		// Maximum available memory
		System.out.println("Max Memory: " + instance.maxMemory() / mb);
         //System.out.println("start total memory: "+tm1);
         //System.out.println("start free memory: "+fm2);
         //System.out.println("end total memory: "+tm1);
         //System.out.println("end free memory: "+fm2);
      } catch (IOException e){
         System.out.println(e);
      }
   }
}

[Junilu Lacar]

Well, a few things that seem off in that code:
1. There's a lot of try-catch but nothing that throws an Exception as far as I can tell.
2. Lots of recursive calls. If you recurse deep enough, you'll get a StackOverflowException.

[Junilu Lacar]

BTW, if I were evaluating this code, I'd be highly put off by your use of try-catch blocks in those two methods. Those don't look like a proper use of try-catch blocks to me, even if there was something that would throw an Exception. Why are you catching Exception and not something more specific? Is the condition you're protecting against truly an exceptional condition? It seems to me you're using try-catch and exceptions to control the flow and logic of your code. That's a very bad practice.

[Dave Tolls]

What are all these things?
That is, what are 'bar' and 'ar'?
What values do 'n' and 'm' have from the file?

Just trying to figure out how big all these lists and arrays are likely to be.

[Craig Oconnor]

HAHA! I find it really interesting the responses. I thought it to so clever what I did. I am counting edges. so when i am traversing the 'map' notice I check each squares, "above","below","left","right". if I check left or up at x = 0 or y = 0, I get an out of bounds exception, so this is an EDGE. which means that if I am studying the water (the 0's) I know I have an ocean I am studying but if I am studying a lake, I will never get an exception.

So, instead of catching it in code (which I originally did), I just used catches instead, if you hit the edge of the map (for instance x = -1, y = 0), than you have an ocean, if you don't, you have a lake or river.


I didn't foresee showing people this code, so I just do ar for array and bar for boolean-array. If I write code I know someone is going to read I am more thorough with my naming conventions. It's about writting a FAST algorithm, not clean code in this case.


- edit, I do appreciate the help given though. I will use iteration instead of recursion just to see.

[Junilu Lacar]

Craig Oconnor wrote:I didn't foresee showing people this code, so I just do ar for array and bar for boolean-array. If I write code I know someone is going to read I am more thorough with my naming conventions. It's about writting a FAST algorithm, not clean code in this case.

The most important person reading that code is yourself. Why put all that cognitive weight on yourself? That's like switching all the keys on your keyboard around and reasoning that you're the only one using it and you already know how to touch type anyway, so it doesn't really matter that the key marked "A" is actually for the letter "F" and the key marked "?" is really the ";" key. I never skimp on cleanliness in code, especially when I'm still figuring out how to make the program work. The more the program reflects how you're thinking about the problem, the easier it is to figure out a solution.

As far as making it fast, all those try-catch blocks won't help. Try-Catch blocks are expensive to set up and slow down the execution of your program considerably.

[Junilu Lacar]

And if you're not showing this code to anyone else, what did you mean by "I did not get an accept"? From whom did you not get acceptance of your program?

[Craig Oconnor]

Junilu Lacar wrote:And if you're not showing this code to anyone else, what did you mean by "I did not get an accept"? From whom did you not get acceptance of your program?

Thanks for the advice guys. I will take these lessons to heart. As for the the "accept". A computer generates a response. If it gives all the answers within an allotted period of time and within the memory limit you win. At first I really was always doing clean nice code, but I noticed as i hacked my own code in different ways my results were getting better and better. Honestly. I did a version without the catches and it took .05 seconds longer. Which going from 1000 data points to 100,000 data points that makes a huge difference.

I wrote this problem in JS too, and personally I find it REALLY easy to give clean posh code with JS cause it has push and pop and map and sort and NaN and JSON and etc.... also i rarely use for loops but stick to lambdas but JS is much slower than java.

Plus I genuinely want to improve Java.

Anyways, thanks for you response....

[Campbell Ritchie]

Craig Oconnor wrote:. . . with JS cause it has push and pop and map and sort and NaN and JSON and etc.... also i rarely use for loops but stick to lambdas . . . ....

Java has all those things, too.

[Dave Tolls]

OK, but what are 'ar' and 'bar' supposed to represent?
To me you have a 'map' of 1's and 0's...so what is the byte array for? What does it show?
And what are those two ArrayLists?