Win a copy of Terraform in Action this week in the Cloud forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Rob Spoor
  • Bear Bibeault
Saloon Keepers:
  • Jesse Silverman
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Al Hobbs
  • salvin francis

updating files from one repository to another

 
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi,

I am a Java beginner and am working on a project which needs to be run once a week, what its supposed to do is updates file from one repository to another.

Lets say we have source(repository new) and destination(repository old). Now the destination needs to have all the new files copied to it that have been added to the source this week and we need to delete all the files from destination that no longer exist in source. So its basically like copying the entire source to destination but since its a large repository we need to avoid copying the files that already existed. NOTE: existing Files are never edited, only new ones are added or old ones are deleted.

I need help with the back end only for now which i want to do in java. Just to get a hang of it i tried working on a program that does the same on a local directory. this works:




now i need to edit it for it to implement on webDAV. the problem is I am not familiar with any of the functions in java.net . can anyone tell me how to edit this to work on webDAV.

Thans.

 
Rancher
Posts: 1776
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Is webDav a code maintenance tool. Can you check it it has API to support check out and check in using java?
 
Saloon Keeper
Posts: 7178
166
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Why are you trying to reinvent the wheel? That's what Rsync does, not the WebDAV part, but the file syncing part.

Are both WebDAV servers in place? Do you have their URLs, usernames and passwords?

The Java class libraries do not have a WebDAV client, but searching for "java webdav library" or some such term will find several.
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Tim Moores wrote:Why are you trying to reinvent the wheel? That's what Rsync does, not the WebDAV part, but the file syncing part.

Are both WebDAV servers in place? Do you have their URLs, usernames and passwords?

The Java class libraries do not have a WebDAV client, but searching for "java webdav library" or some such term will find several.


----------------------------------------------------------------------------------------

sorry for the late reply.
Since I am learning I kind of want to do it from scratch, and we are just working on one server where we have two folders. We just need to update one from the other and I have the user name and password. I am able to connect and get the entire file system the only part I am having trouble now is with copying a file from one folder to another. The below code works perfectly the first time but when it finds a second file that needs to copied it throws an exception.



This is the console:
reading before inp: /filesys1/Copy (2)/Copy (2) (3)/doc - Copy (2) (3) 3 - Copy.txt
reading after inp: null/doc - Copy (2) (3) 3 - Copy.txt
write success: true, /filesys2/Copy (2)/Copy (2) (3)/doc - Copy (2) (3) 3 - Copy.txt
reading before inp: /filesys1/Copy (2)/Copy (2) (3)/doc - Copy (2) (3) 4 - Copy.txt
reading after inp: null/doc - Copy (2) (3) 4 - Copy.txt
Exception in thread "main" java.lang.NullPointerException
at org.apache.commons.net.io.Util.copyStream(Util.java:100)
at org.apache.commons.net.ftp.FTPClient.__storeFile(FTPClient.java:385)
at org.apache.commons.net.ftp.FTPClient.storeFile(FTPClient.java:1379)
at FtpConnectTry.gotoDirectory(FtpConnectTry.java:12)
at FtpConnectTry.gotoDirectory(FtpConnectTry.java:89) //the only code line 89 has is a recursive call to he method
at FtpConnectTry.gotoDirectory(FtpConnectTry.java:89) //the only code line 89 has is a recursive call to he method
at FtpConnectTry.main(FtpConnectTry.java:34) //the only code line 34 has is a recursive call to he method

Also notice how it returns null in the console after the inputStream inp = new.....
Say if there are 20 files that need to be copied from source to destination my program will do exactly what it needs to if we run it 20 times, so again the problem is whenever we come into this if loop the second time it gives an error. any suggestions will be greatly appreciated.
 
Marshal
Posts: 74387
334
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Moving thread as too difficult for "beginning".
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

John Jai wrote:Is webDav a code maintenance tool. Can you check it it has API to support check out and check in using java?


I totally missed your comment, webdav is actually like web directory. If you access a certain url it gives you the entire directory. And there are API's for webdav, it is org.apache.webdav.ant.. But the code I wrote here is for any server to server transfer. I am probably gonna start studying the webdav API today onwards. Still, since I spent so much time on this it would be nice to know what I am doing wrong. Thanks for you interest.
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Campbell Ritchie wrote:Moving thread as too difficult for "beginning".


Haha.. would you help me if I told you I was an expert. I just took this as project to get comfortable with java. I mean I have done calculator level programs on java before but nothing this big so far.
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I found [FTPClient].completePendingCommand();
I am sure it has got something to do with this. If anyone knows about this please help me out. when i insert the above method after inp.close(); my program copies 2 files at a time instead of just one that it was doing earlier. I just need to figure out how to use it right.
 
Bartender
Posts: 10780
71
Hibernate Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Rish Gupta wrote:I found [FTPClient].completePendingCommand();
I am sure it has got something to do with this. If anyone knows about this please help me out. when i insert the above method after inp.close(); my program copies 2 files at a time instead of just one that it was doing earlier. I just need to figure out how to use it right.


Seems to me you're piling an awful lot of things on top of one another: directory reading, comparison, file syncing and, to top it all, FTP to do the copying.

My suggestion: back up and get each stage working before you go onto the next; and I would also use simple File I/O first and get it working with that (maybe with local directories); then only once everything else is OK, tackle the FTP part.

Also, if I were doing this, I think I'd use Files for my comparisons rather than Strings - maybe a subclass I created that can compare by names relative to a 'start directory'. That way you could use Java Sets rather than ploughing through arrays.

Winston
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I did work with local directories. My first post was working perfectly, it did every thing.

For the FTP part I started from scratch with simply printing the directory of the source folder first.
Next step was to create blank directories which were found missing.
Then came the actual FTP file transfers into those directories and bam I hit a stone wall with no way around it.
 
Rish Gupta
Greenhorn
Posts: 29
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Here is the entire code I am working on in case someone in the future needs to refer to some other part other than where i got stuck.

PS: the if(!fileExists) {....} part may be a lil different from what I posted earlier.
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic