JNI issue with ByteArray

[Dennis Quelch]

Hi,
I was wondering if someone could tell me what the following might be a symptom of. I'm making a JNI call from Java to C. Inside the C routine I allocate a ByteArray and populate it with a series of Floats. On the return from the JNI call I go through and parse through the byte array doing a endian swap on all the floats. Nearly all of the floats are correct, but around the middle of the array I start getting NAN returned from my readFloat() call. All the floats before and after this certain section are correct. I've verified on the C side that the byte array is being populated correctly so I'm confident it's not on that end. Note, this doesn't happen on every call to the routine either. I'm using latitude/longitude coordinates for the floats and in some instances the numbers are not reading correctly by the DataInputStream.

Any ideas? If the memory was off in the byte array then wouldn't the floats before or after the trouble section be wrong as well? What would cause readFloat to return NaN?

Code snippet from how I'm reading from the byte array on the java side

public static void some_routine(byte[] data) {
  
    DataInputStream str = new DataInputStream(new ByteArrayInputStream(data));
   try {  
     int cnt = EndianUtil.swapInteger(str.readInt());   // Returns the number of floats
     for (int i=0;i < cnt; i++) {
           float value = EndianUtil.swapFloat(str.readFloat());

           System.out.println("Float value: " + value);  //  Sometimes is 6.9055E-41  (NaN swapped)
      }
    } catch( IOException e) {}

}

Thanks for any help diagnosing!

[Ernest Friedman-Hill]

Hi,

Welcome to JavaRanch!

Probably it's the way the byte array is being created, populated, and/or returned from the C side -- let's see how you're doing it.

[Dennis Quelch]

Sure, here is what I'm doing on the C side:

typedef struct {
float lat;
float lon;
} LatLonType;

JNIEXPORT jbyteArray JNICALL Java.....buildGridRequest(JNIEnv *env, jclass cls, jbyteArray gridInfo)
{
   LatLonType *rsp = NULL;
   jbyteArray result;
   char *buffer;
   char *p;
   int count;

    rsp = generateLatLonPoints(....., &count);   // Returns an array of LatLonType's

   // Get needed size of buffer
   int size = sizeof(int) + (count * sizeof(LatLonType));

  // Allocate memory 
  buffer = (char*) calloc(size, sizeof(char));
  p = buffer;

  memcpy(p, &count, sizeof(int));
  p+= sizeof(int);

  for(int i=0;i < count; i++)
   {
     memcpy(p, &rsp[i], sizeof(LatLonType));
     p+= sizeof(LatLonType);
   }

  result = env->NewByteArray(size);
  env->SetByteArrayRegion(result, 0, size, (jbyte*)buffer);

  free(buffer);
  free(rsp);

  return(result);
}

Like I said in my previous post. 95% of the time all the floats pass through just fine, then every so often on the java side the readFloat() call returns NaN. Even though the data going into the byte array looked fine.

Thanks so much for your time helping me figure this out!

Dennis

[Ernest Friedman-Hill]

Unfortunately I don't see anything obvious here.

Is there possibly anything inside generateLatLonPoints() which is not threadsafe? Might the problems occur when two calls happen almost simultaneously?

[Dennis Quelch]

Unfortunately I've added some extra code to dump out the data to a file so that I can see what the exact data is when the NaN is encountered. It always looks correct to me, and nothing obvious. There is definitely something happening between the shared library and JNI call when certain floats are sent in. I thought that maybe my compiler was producing a shared library that wasn't quite compatible with what the virtual machine was expecting with regards to a floating point number. Though I would expect this behavior to show up more often than it does though.

[Rob Spoor]

One thing I find a bit dangerous is your assumption that one LatLon struct is always going to be 8 bytes (the size of a Java float). Whereas a Java float is always 4 bytes, there is no such requirement for C programs / libraries. I would definitely change that part of the writing, to ensure you write 8 bytes each time. Maybe something like this:

memcpy(p, &count, sizeof(int));   
p+= sizeof(int);

for(int i=0;i < count; i++)
{
    jfloat lat = (jfloat)rsp[i]->lat;
    jfloat lon = (jfloat)rsp[i]->lon;
    memcpy(p, &lat, sizeof(jfloat));
    p += sizeof(jfloat);
    memcpy(p, &lon, sizeof(jfloat));
    p += sizeof(jfloat);
}

sizeof(jfloat) will return 4, so this code is no longer dependent on any compiler specific settings.

Note that in your reading loop you are reading only one float whereas two are written.

[Dennis Quelch]

Thanks! I'll give that a try. And yes, my java example was missing that extra readFloat(). I had just copied over some code to illustrate what I was doing. The original has both reads in it. I'll post back after testing. Thanks!

[Dennis Quelch]

Hey guys, just wanted to give you an update. I tracked down the problem and it was in the EndianUtil.swapFloat() routine. The standard EndianUtil library that you can find anywhere on the web for swapping a float has issues in about 1% of floats that are sent into it. By reading a float using the readFloat() routine of DataInputStream, converting it to bits using Float.floatToIntBits(), swapping the Integer, then converting it back to float using Float.intBitsToFloat() causes a precision error in certain cases resulting in junk being returned.

The solution is to write a new set of routines that read the bits from the bytestream directly into integers, then perform the byte swapping and converting it back to a float.

Problem solved! Moral of the story, always be wary of seemingly common utilities found on the web.

Thanks

Dennis

[Ernest Friedman-Hill]

That's great! Thanks very much for the update; this was a real stumper.