• Post Reply Bookmark Topic Watch Topic
  • New Topic

Design suggestion  RSS feed

 
nitinram agarwal
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I have a framework using Core java and Spring 2.5 which invokes some sftp script in unix environment to upload the files to destination server. As of now, it supports only 1 file upload per invocation. I am in charge of enhancing the framework so that it supports multiple file upload. However for multiple files to be sftped, if for some reason , the script fails after sftping less number of files then the next invocation of the program should only try to sftp the remaining files (as an added feature rather than retrying all the files). For ex. assuming that on the first invocation, the program is supposed to sftp 5 files and it failed after sftping 2 files then there should be an option to sftp only the remaining 3 files in the next invocation. As a possible solution, I have multiple options like updating some cache entries or updating db tables but that is not allowed as a solution (and I have not spend much time arguing why as of now). I can think of another solution - writing to a file , the name of the files which were successfully sftped and proceed with the remaining files. However this seems to be a more crude kind of solution and I am thinking of a better solution which should be generic enough. Can you please advice on some better design for this situation? Please note that the destination server is not sending any information back to the source server for all this sftp.
 
Campbell Ritchie
Marshal
Posts: 56553
172
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This discussion is too difficult for “beginning”, so I shall move it.
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
nitinram agarwal wrote:I have a framework using Core java and Spring 2.5 which invokes some sftp script in unix environment to upload the files to destination server. Can you please advice on some better design for this situation? Please note that the destination server is not sending any information back to the source server for all this sftp.

Well that sounds like a problem right there. And if you're not in control of the server end of things, the chances are you won't be able to move the file (or delete it) after upload, which is what I'd probably try first. Does the server allow you to run the 'ls' command from an ftp session?

My suggestion: rather than run a script, why not do it in Java? this page lists a few ways of doing it, but I don't know if any of them support 'globbing' (which is what you want).

Winston
 
Richard Tookey
Bartender
Posts: 1166
17
Java Linux Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
nitinram agarwal wrote:I have a framework using Core java and Spring 2.5 which invokes some sftp script in unix environment to upload the files to destination server.


Presumably this uses ProcessBuilder or Runtime.exec() to invoke the script but this indirect approach is always going to have a problem so why not use one of the pure Java SFTP client libraries. This will allow you much finer control of the process and you will be able to get a direct feedback as to the success or failure of each file transfer.
 
nitinram agarwal
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well the framework actually uses RunTime.exec and it is some thing like "sftpScript <arg List>". the requirement is to invoke the same script multiple times with different arg list (which seems to be bit trivial). however I am trying to figure out on how to know which files the program should try to sftp as part of subsequent run (for ex originally supposed to transfer 5 files, could transfer only 3 files and failed. In the next run should try to sftp 2 more files).
I cannot rewrite the application using relevant Java libraries as the whole application is currently working using this framework and I need to enhance it.
 
Richard Tookey
Bartender
Posts: 1166
17
Java Linux Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
nitinram agarwal wrote:I cannot rewrite the application using relevant Java libraries as the whole application is currently working using this framework and I need to enhance it.


So changing to us a Java SFTP library is not an enhancement! Interesting concept.
 
Pat Farrell
Rancher
Posts: 4686
7
Linux Mac OS X VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
nitinram agarwal wrote:I cannot rewrite the application using relevant Java libraries as the whole application is currently working using this framework and I need to enhance it.


I'm sorry. While the Ranch here is a very friendly place, this statement is ridiculous. Perhaps your native language is not English, so the translation is losing a lot.

You need to "enhance" aka fix the application. So do so. You can keep the overweight framework that is in place, simply don't use the shell scripts and do it all with existing, proven, documented and debugged Java libraries.
 
nitinram agarwal
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am sorry if I have unintentionally caused something. When I said, I cannot.. I mean.. as a consultant.. I have constraints. The customer already has the framework in place and I don't have the liberty of replacing the framework. The replacement will be a time consuming activity which requires budgeting and other procedural activities. I am trying to find out some optimal solution for enhancing the framework and am trying to avoid the file usage as a tracking tool for files which have been transferred successfully. I have turned to Javaranch multiple times in the past seeking suggestion and always had a healthy discussion. I never intent to hurt anyone and always find the folks to be very helpful.
 
Richard Tookey
Bartender
Posts: 1166
17
Java Linux Netbeans IDE
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I really don't understand. Unless the framework is purely there to invoke Runtime.exec() I am not expecting you to replace the framework; just to extend it. As a consultant I would expect you to advise your customer of the fragility of the just modifying the script and the advantages of replacing it with a pure Java solution. If the framework is there to just invoke Runtime.exec() then if well designed it should just be a matter of plugging in the replacement.
 
nitinram agarwal
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I agree with your comments. In fact, I suggested the customer to use a new framework all together with all pure java part and migrate the older one in phases (many teams are using the existing framework) to the new one. but it will take time and hence I am trying to figure out what best solution can be provided in the current scenario.
 
Winston Gutkowski
Bartender
Posts: 10575
66
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
nitinram agarwal wrote:I can think of another solution - writing to a file , the name of the files which were successfully sftped and proceed with the remaining files. However this seems to be a more crude kind of solution and I am thinking of a better solution which should be generic enough.

Well, it seems to me that your stuck between a rock and a hard place.
1. The server won't provide you with any information.
2. Your users want you to band-aid the existing (bad) system, rather than fix it properly. See my quote below.

Also: you never did answer my question about 'ls'.

Winston
 
nitinram agarwal
Ranch Hand
Posts: 90
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Does the server allow you to run the 'ls' command from an ftp session?

I need to check this with the deployment guy who typically has the privilege of checking this detail. If this command can be executed then I will know the files which were successfully transmitted. In this case, the solution will be simpler and I don't need to use any secondary storage for file tracking. I hope I got what you meant? I missed this part earlier and hence could not respond.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!