• Post Reply Bookmark Topic Watch Topic
  • New Topic

Zip file produced by Java can't be opened in OS/390 (Mainframe)

 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

I have to zip a file and send it to a mainframe over ftp.

The mainframe receives the file, but it can't open it, Pkunzip throws an error.

The produced Zip file contains only 1 file.

Now, when I open the file in WinRar, rename the filename, rename it back to the original filename again and try to open the file (created by Java program and now processed by WinRar) it opens.

I've included 2 pictures of the first few characters in hexadecimal view of both files.

and there's a difference. the file rezipped by WinRar contains a few more characters in the header section.

I don't know, if it's of any importance at all, but the only difference I could find was that descibed above.


1. Question :
There's a zipping method in the java API, ZipOutputStream.DEFLATED.

Should I use this one ? The Javadoc says "Compression method for compressed (DEFLATED) entries". What does that mean ? to compress already compressed file like mp3, jpeg etc ?


2. Question :
I couldn't find anything that could affect the header of the file to be produced by Java.
Does anybody have a suggestion to this problem or even a solution (anybody encountered this before) ?

Finally, the code that zips the file looks like this :


final int BUFFER = 2048;
byte data[] = new byte[BUFFER];

ZipOutputStream out = null;
BufferedInputStream origin = null;
try {
out = new ZipOutputStream(new BufferedOutputStream(new FileOutputStream(destination)));
out.setMethod(ZipOutputStream.DEFLATED);
out.putNextEntry(new ZipEntry(source));

origin = new BufferedInputStream(new FileInputStream(sourceFilename), BUFFER);
int count;
while ((count = origin.read(data, 0, BUFFER)) != -1)
out.write(data, 0, count);
} catch (Exception e) {
e.printStackTrace();
} finally {
close(origin);
close(out);
}


Any help is appreciated.



ProcessedBy_WinRar.gif
[Thumbnail for ProcessedBy_WinRar.gif]
Zipped by WinRar
CreatedBy_Java_program_and_untouched.gif
[Thumbnail for CreatedBy_Java_program_and_untouched.gif]
Zipped by my Java program (untouched)
 
James Sabre
Ranch Hand
Posts: 781
Java Netbeans IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Make sure you have transferred the file using ftp in 'binary' mode.
 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
James Sabre wrote:Make sure you have sent the file using ftp 'binary' mode.


Hi James,

Thanks for the quick response.

It's actually an SFTP server, I use the JSch library at http://www.jcraft.com/jsch/, and it doesn't have an option to set the mode to 'text'.

(says this page : http://osdir.com/ml/java.jsch/2008-06/msg00024.html and I can confirm that by looking at the API)

However, I took the screenshots attached without going through the sftp phase at all.

And, during our tests we transfered the file to mainframe using WinSCP (we tried it manually).

So again, the only difference I could find here between the Java created file and the WinRar-processed file are these few hex characters at the beginning (header ?) of the zip file.

I still need help on this.


 
James Sabre
Ranch Hand
Posts: 781
Java Netbeans IDE Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In your position I would still be concerned that I had corrupted the file during the transfer and would generate a message digest (MD5, SHA1, SHA2 etc) on both machines. Only if the digests are exactly the same can you be pretty sure that you have not corrupted the file.

 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
James Sabre wrote:In your position I would still be concerned that I had corrupted the file during the transfer and would generate a message digest (MD5, SHA1, SHA2 etc) on both machines. Only if the digests are exactly the same can you be pretty sure that you have not corrupted the file.



Thanks for the advice, I will do that.

However, I'm pretty much sure that it is not a transfer problem, because, we ran the file transfer program on the mainframe, which picked the file from the ftp server,ran pkunzip. It gave an error.

then we opened it via WinRar, renamed the file in the zip file and renamed it back to the original filename. Then ran the mainframe program again, the file got transfered, and we ran pkunzip again. and it worked.

I think, pkunzip on the mainframe doesn't understand some 'zip-versions', WinRar does though.

So I think we narrowed the problem to the zip structure of the file created by my program.

I also tried Apache Commons library for creating zip files, but the result was the same, no success. The created zip file was the same as the one created by the Java API.


 
Joe Ess
Bartender
Posts: 9361
11
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Mehmet Gunacti wrote:
I think, pkunzip on the mainframe doesn't understand some 'zip-versions', WinRar does though.


This is quite possible, though it is interesting that WinRar understand the Java-generated one, and the mainframe understands the WinRar-altered one. The specifications the Java API uses for Zip files are at the bottom of this page. Maybe that will shed some light.
 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Joe, that helped.

Here's the zip format spec. : http://www.pkware.com/documents/casestudies/APPNOTE.TXT

according to the spec I tried to find the differences of both files.

the result is as in the picture attached.

I used a file "a.txt" which content is simply "1234567890".

the crc of a.txt is :
DEC : 639479525
HEX : 261DAEE5

the zip file created by my Java program includes 4 sections (I've marked those with a green underscore) :
0x04034b50 <- A. Local file header:
0x02014b50 <- F. Central directory structure:
0x08074b50 <- C. Data descriptor:
0x06054b50 <- I. End of central directory record:

a zip file created by right-clicking a.txt and choosing "add to archive..." has fewer sections.
But that's not important right now, since I know that PKUNZIP on the mainframe does accept archives created by Java and rezipped by WinRar.


The zip files created by java doesn't include
1 - the crc
2 - the compressed file size
3 - the uncompressed file size

in the "local file header". Those 3 are present and set in other parts of the file, though, e.g. in the "central file header signature" section.

And the 'Version made by' flag changes from "14 00" to "17 0b", but I don't think that's the problem PKUNZIP is complaining about.


I've also tried to set the crc and the size explicitly, but the result is the same as described above :




I'm stuck at this point. Does anybody have a clue how to solve this ?


rezippedBy_WinRar.jpg
[Thumbnail for rezippedBy_WinRar.jpg]
Re-zipped by WinRar
createdBy_Java.gif
[Thumbnail for createdBy_Java.gif]
created by Java
 
Joe Ess
Bartender
Posts: 9361
11
Linux Mac OS X Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Does the mainframe have the JAR executable? That should conform to the same spec as the JDK/JRE on the machine.
 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Joe Ess wrote:Does the mainframe have the JAR executable? That should conform to the same spec as the JDK/JRE on the machine.


Don't know, but I doubt that they will accept that as a solution. I will ask next week for that.

This guy here had a similar (same) problem in .net : http://dotnetzip.codeplex.com/Thread/View.aspx?ThreadId=57007

I'll have to do some more tests with the mainframe guy next week.

 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I got an answer here : http://forums.oracle.com/forums/thread.jspa?messageID=9213231
by malcolmmc : I've had problems with certain versions of zip utility in the past. The problem is down to the way that the ZipOutputStream writes zip files in a forward-only direction, and doesn't know the size of the entries until they have been written. The ZIP specification make allowance for this kind of thing by allowing the size, compressed size and CRC for an entry to be deferred from the entry's header to a special footer item which follows the entry's data.

However there have been some ZIP utilities that don't implement this feature of the ZIP specification. I wound up writing my own zip writer which required a file as an output, so it could skip back and write the necessary info into the header. Don't have it to hand, I'm afraid, though you might still find it somewhere on the archives of the old Sun site.

 
Graham Weatherup
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am very interested to know if you found the code malcolmmc mentioned that writes the header to the beginning of the file or could point me to where it would be on the old sun site he speaks of. Or can anyone else tell me how this is done.

I also have a thread open here about my problems

It is also very interesting that this person could not get odb files to open correctly but did get them to open at all. as I don't see that he is doing this.

Any other thoughts of what might be the problem?
 
Mehmet Gunacti
Ranch Hand
Posts: 34
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Graham Weatherup wrote:I am very interested to know if you found the code malcolmmc mentioned that writes the header to the beginning of the file or could point me to where it would be on the old sun site he speaks of. Or can anyone else tell me how this is done.


Sorry, but I went native.
Never worked with ODB files either.

Good luck with finding a solution. Btw. consider going native
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!