problem retrieving a zip file from FTP server using org.apache.commons.net.ftp.

[esha sharma]

here is the problem
I am trying to retrieve a zip file from FTP server and after retriveing few MB of data (aprx 22MB) it hangs and doesnt move further... the same code works fine for another files except zip file...
i m using ftpclient.retrieveFile(filename, fos)
where filename is name of the file that i m trying to retrieve...n fos is FileOutputStream....
any help would be appreciated..
[ January 02, 2009: Message edited by: esha sharma ]

[esha sharma]

I am still waiting for answer...

[Joe Ess]

From our FAQ, PatienceIsAVirtue:

And if you finally decide to bump the post, please try to do so by posting more information! After all, if nobody answered, that's probably because nobody did know an answer. You had 24 hours to do your own research in the mean time - show us that you used them, and give us new things to think about. If nothing else, work harder to TellTheDetails.

[esha sharma]

thank you for the reply.
Joe, before coming to this forum i googled for this problem...but could not get any answer to solve this problem..
So I thought i might get some answer here...
I would like to elaborate the problem...
I am trying to retrieve a zip file from FTP server and the size of zip file is apx 67MB. For that I am using org.apache.commons.net.ftp.FTPClient class. Everytime it fetches that file till 22MB and stuck there...it does not do anything and program stuck there...
please let me know if i am doing something wrong...the same code works fine if i retrieve any othere file...
one more question, is there any limitation for file size?
please let me know, is there anything that i need to mention here?
thank you

[Joe Ess]

There is no limitation on the size of a file. Can you show us your code?

[Steve Luke]

Perhaps you can show the code you are using to retrieve the file and what you do with it after it is downloaded.

As for limits, do you control the FTP server hosting the file? What do you know about it's configuration? Can you access the FTP sites logs to see if there is an error, warning, or message? What happens when you try to download the file with a traditional FTP client?

Do you catch exceptions and log them? Have you checked the logs to see if an exception occurs? Is it just one particular ZIP file that causes the problem or all ZIP files above a certain size? When you say it works on other file types did you test with other files that where just as large as the ZIP file?

You don't actually have to answer all these questions on this thread, but they are the sort of questions you should test to try to get yourself closer to an answer. Many of them will be relevant to the problem and when you address them you are likely to find a key answer that helps solve the issue. So please do go through the steps of checking all of these things. When you post back please show the code and any answers to the questions above that may be relevant.

Steve

[esha sharma]

************************source code***********************

    
public static void retrieveZip () throws Exception
{
    String fileName="data.zip";

        String DBFolder = myServer.getProp("rootFolder") + File.separator + "zipFile";
        File file = new File(DBFolder + File.separator + fileName );
        FileOutputStream fos = new FileOutputStream( file );
        FTPClient f=null;
        int type = FTP.ASCII_FILE_TYPE;
		
	String username = myServer.getProp("ftpServer.username");
	String password = myServer.getProp("ftpServer.password");;
	String server =myServer.getProp("ftpServer.host");
	String folder = myServer.getProp("ftpServer.catalogFolder");
    try{   
        lg.info(server+":"+username+":"+password+":"+folder);
	f = new FTPClient();
	f.connect(server);
	if(!FTPReply.isPositiveCompletion(f.getReplyCode())) throw new RuntimeException("Unable to connect:" + f.getReplyString());
	f.login(username, password);
	if(!FTPReply.isPositiveCompletion(f.getReplyCode())) throw new RuntimeException("Unable to login:" + f.getReplyString());
	f.changeWorkingDirectory(folder);
//        f.cwd(folder);
       if(!FTPReply.isPositiveCompletion(f.getReplyCode())) throw new RuntimeException("Unable to chg dir:" + f.getReplyString());
            f.setFileType(type);
        if(!f.isConnected()) throw new RuntimeException("Unable to connect");
         f.enterLocalPassiveMode();
//        f.enterLocalActiveMode();
        
        
        // commented it for the timebeing
 //       f.setFileType(FTP.BINARY_FILE_TYPE);
//        f.setFileTransferMode(f.BINARY_FILE_TYPE);
        System.out.println("isRemoteVerificationEnabled()==="+ f.isRemoteVerificationEnabled());
        f.setRemoteVerificationEnabled(false);
 //       f.setFileTransferMode(FTPClient.COMPRESSED_TRANSFER_MODE);
//       f.setFileType(FTPClient.BINARY_FILE_TYPE);
 //     f.setFileTransferMode(FTP.ASCII_FILE_TYPE);
//       f.setFileTransferMode(FTPClient.BINARY_FILE_TYPE);
//
//        f.sendNoOp();
 
       FTPFile[] files = f.listFiles();
        for(int i=0; i<files.length; i++)
        {
         System.out.println((i+1)+" ) File name== "+ files[i].getName()); 
         System.out.println("File Type== "+ files[i].getType());  
         System.out.println("File is file == "+  files[i].isFile());
         if(files[i].getName().equalsIgnoreCase(fileName))
         {
         System.out.println("before retrieving file");    
         f.retrieveFile(fileName, fos ); 
         System.out.println("after retrieving file");

         }  
            
        }
//        boolean commandOK=f.completePendingCommand();
        
    }catch(Exception e){e.printStackTrace(); System.out.println("exception in retrieveZip() method  ");}
    finally{ // close input stream
            if(fos != null)
                fos.close();
            f.disconnect();
            System.out.println("FTP connection closed");
}

}

************************end of source code***********************


1) The zip contains files, using these files I am updating some tables.

2) do you control the FTP server hosting the file?
No I do not.

3) What do you know about it's configuration?
I do not have other information except host name, user name, password.

4) Can you access the FTP sites logs to see if there is an error, warning, or message?
No I cannot access.


5) What happens when you try to download the file with a traditional FTP client?
I have tried downloading that file through FTP Client software; I can download everything without any problem. I have noticed that file size is different while using both procedures (using program and FTP Client software).

6) Do you catch exceptions and log them?
Yes, I do. No exception no error message. It does not do anything. Like stuck in infinite loop.

7) Is it just one particular ZIP file that causes the problem or all ZIP files above a certain size?
When I try to download another zip file with smaller size, first, it does not download complete file. Second, when I try to open that zip file, it gives error message, “Error: Unexpected end of archive file”. And when I try to unzip that file I get another error message “file is corrupted.”
All the other files (which are not zip files) on FTP server are below 22MB, so I am not sure about that.

Thank you!

JCE: Please Use Code Tags

[esha sharma]

i have tried using retrieveFileStream()....and then copied the stream using copyStream() method....but i hv noticed that its blocked and doing nothing (not even giving any error message) because it is waiting to read next byte from InputStream. As it is not getting anything from inputstream so its waiting....Please let me know, if anything could be done in this situation...
Thank you!

[hvt kg]

Hi, esha sharm.

I am new bie.
I had problems the same. Can you post your code. which is tries?

Regrads
hvt_kg

[Amit Ghorpade]

"hvt kg" welcome to Javaranch
please check your private messages. You can check them by clicking the My Private Messages link above.
Also take a look at NotACodeMill

[Bernie Barnard]

Today I was having the same problem. I was downloading zip files from a site and some files were coming back smaller than those on the FTP server. Curiously, it was working fine on my Windows desktop, but not on my linux server. Solved the problem by FTPClient.setFileType(FTP.BINARY...). Try it!.

[Achyuth Nambiar]

Hi,

I am facing the same problem.
While downloading the zip file from an ftp site, it just goes on into infinite loop.

The code is same as what Esha has.

The below code

setFileType(FTP.BINARY_FILE_TYPE)

did not work for me.

Could you please let me know how to approach this issue?

Thank you
Achyuth

[Aniruddh Joshi]

So when it 'hangs' , does your entire flow stop.
I mean you never come out of the try block ?

And does it happen with this precise file or with files > size of this one ?
Trying some other larger file might help understand if its an issue with the file size.

[Achyuth Nambiar]

Hi Ani,

Thank you for your reply.

When it hangs it does NOT stop the flow. It just continues as it in an infinite loop. It never comes out of the try block.

The similar file with the same size and name works some times. This issue is happening randomly.
I don't think this issue is because of the files size or so. Because it works for the same file when we try again later.

Thank you
Achyuth