• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Bear Bibeault
  • Liutauras Vilda
  • Devaka Cooray
Sheriffs:
  • Knute Snortum
  • Junilu Lacar
  • Henry Wong
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
  • Tim Holloway
Bartenders:
  • salvin francis
  • Frits Walraven
  • Piet Souris

Uploading file to remote server

 
Ranch Hand
Posts: 138
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So far in order to upload a file to my existing server, I'm using the following piece of code in Java :



So right now my above piece of code resides on the same server RHEL Server where files(inside `/srv/regular_users/` are getting uploaded. Let's say if I have a different file storage server, and if it's domain is https://newfilestorageserver.com

Then, is following path going to work?


 
Sheriff
Posts: 21863
105
Eclipse IDE Spring VI Editor Chrome Java Ubuntu Windows
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you want to upload something through HTTP, then your server needs to expose an endpoint that allows you to upload files. You'd then have to send a POST request to that endpoint with the file. Using Path won't work.

An alternative is to not use HTTP but instead FTP(S) or SFTP, both widely used file transfer protocols. Both require a separate service to be installed on the server; if it's a Linux server than sshd is probably already installed and you can already use SFTP. I've created projects that allow you to use FTP(S) or SFTP in combination with Path.
 
Jack Tauson
Ranch Hand
Posts: 138
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rob Spoor wrote:An alternative is to not use HTTP but instead FTP(S) or SFTP, both widely used file transfer protocols. Both require a separate service to be installed on the server.



Thanks. When you talk about FTP or SFTP, I have few questions regarding this:

1)Does FTP/SFTP needs to be installed on the File Storage server only or the server from where I'm initiating the file uploading process? Or is it required at both the places?

2) When would you use FTP over SFTP in this scenario?

 
Marshal
Posts: 25143
64
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you're using FTP, or SFTP, or basically any file transfer mechanism, you need to have a server for that mechanism running on the machine you are transferring files TO. On the machine you are transferring files FROM, you need to have a client for that mechanism.
 
Rob Spoor
Sheriff
Posts: 21863
105
Eclipse IDE Spring VI Editor Chrome Java Ubuntu Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
But the client can be written in Java. For instance, my FTP file system project uses Apache Commons Net. The SFTP file system project uses JSch. Both of these libraries can be used directly without any problems.

Jack Tauson wrote:2) When would you use FTP over SFTP in this scenario?


To be honest, never. SFTP is a better defined protocol, and IMHO provides a better security mechanism. FTP itself should be forbidden because it's fully unsecured. FTPS has security, but is still terrible. I've had quite some issues with the FTP fle system project because FTP simply gives too much freedom to the implementation. Sure, there are common commands, but what exactly is returned in LIST commands is not officially specified. Each server can do its own thing.
 
Saloon Keeper
Posts: 21710
148
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

it's domain is https://newfilestorageserver.com



That's not a domain. That's a URL. The domain is newfilestorageserver.com. The rest of the URL is the protocol specifier, and as Rob mentioned, that only works if there is an HTTP server such as Apache, nginx, or maybe even a custom server written in nodejs - AND if the server contains a webapp that accepts and stores uploaded files at that URL (which internal to the webapp server is the context path "/".

scp is a good bet, since you don't have to have a separate server for it. As long as the target machine is running an ssh server, the firewall and ssh configs allow your client machine to connect, and you have a user account, you're all set. You can either login/copy via userid/password or via a client certificate. Most Linux servers have ssh servers running, since otherwise in a large shop the sysadmins would have to run around between machines all day long. Plus, devops provisioning services like Ansible use ssh/scp as well.

FTP is a VERY bad bet, because it was designed to transmit userid and password as unencrypted text, so very easy to sniff out and abuse.

Another option for general file transfer and sync is the rsync service. It can use scp or it can set up its own server with its own protocols. It's especially good for uploading whole directory trees worth of files, plus it only transfers data that the recipient doesn't already have, so overhead is less.
 
Tim Holloway
Saloon Keeper
Posts: 21710
148
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Incidentally, you don't talk to domains, you talk to hosts. There's a popular shorthand that is built into a lot of systems where when you submit an http(s) URL to "newfilestorageserver.com", then it actually tries to talk to "www.newfilestorageserver.com", where "www" is the host name within the newfilestorageserver.com domain.

I have many hosts at mousetech.com, so if you want to upload to a particular machine, you have to indicate exactly which machine you want to upload to. For example, "sqlserver2.mousetech.com" would be the Fully-Qualified Domain Name (FQDN), for the "sqlserver2" host machine. Note also that a single host might have several hostnames. That same (possibly virtual) box might also host "kerberos31.mousetech.com" and half a dozen other services. By giving each service its own hostname, I can re-distribute services fairly easily.

Most machines within a domain are configured to know their own domain name. So if my desktop system is officially "desktop327.mousetech.com" and I want to scp to filehost5.mousetech.com, I can simply refer to it as filehost5, and the "mousetech.com" will be assumed by default. Note also that not all hosts have to be defined to DNS or an /etc/hosts file, so "desktop327". might simply be the name I slapped on my desktop machine, but not referenced by name from any other machine inside or outside of the LAN (that is, over the open Internet).
 
Jack Tauson
Ranch Hand
Posts: 138
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Paul Clapham wrote:If you're using FTP, or SFTP, or basically any file transfer mechanism, you need to have a server for that mechanism running on the machine you are transferring files TO. On the machine you are transferring files FROM, you need to have a client for that mechanism.



Thanks everyone for your detailed responses. I have few more questions.

Q1) Since I'll be connecting to a different server using FTP/SFTP client ( getting more inclined towards SFTP after reading everyone's responses), I would have to put my user credentials in the code. Is it wise to have dedicated user credentials for this specific connection because I don't want to put my user credentials in the code.?

Q2) Secondly, I found FTP client related libarries online in java like https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html
However, I couldn't find anything similar to that when I tried for SFTP. The thing I found is this:

https://www.baeldung.com/java-file-sftp

Which option would be good if I have to go with SFTP option mentioned in the above link? Thanks again !
 
Tim Holloway
Saloon Keeper
Posts: 21710
148
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To talk to an FTP server you have to have an account on the FTP server machine. That account is different from the login on your local machine and can have a completely different password and/or user ID.
 
Saloon Keeper
Posts: 3021
407
Android Eclipse IDE Angular Framework MySQL Database TypeScript Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jack Tauson wrote:Let's say if I have a different file storage server ...


What exactly is the file storage server?  Is it an storage appliance, a general purpose server running Linux/Windows, ???

Do you have the option of mounting the remote file system from your application platform using NFS/CIFS/WebDAV/SSHFS?  If you do, then you might be able to use your application as-is, and eliminate the effort of implementing a client for SFTP or some other file transfer mechanism.
 
Rob Spoor
Sheriff
Posts: 21863
105
Eclipse IDE Spring VI Editor Chrome Java Ubuntu Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jack Tauson wrote:Q1) Since I'll be connecting to a different server using FTP/SFTP client ( getting more inclined towards SFTP after reading everyone's responses), I would have to put my user credentials in the code. Is it wise to have dedicated user credentials for this specific connection because I don't want to put my user credentials in the code.?


Definitely. That way, you can also further limit what this user can do.

By the way, SFTP allows you to authenticate through a public key, if the server side is setup to allow this. That way you don't have to store a password (however, you do need to store the key pair). See https://www.ssh.com/ssh/key for more information.

Q2) Secondly, I found FTP client related libarries online in java like https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html
However, I couldn't find anything similar to that when I tried for SFTP. The thing I found is this:

https://www.baeldung.com/java-file-sftp

Which option would be good if I have to go with SFTP option mentioned in the above link? Thanks again !


I already mentioned JSch, which is also the first entry on that page. Apache Commons VFS uses JSch so I'd only use it to provide a more easy API. I don't know SSHJ so I can't tell you anything about it.
 
Jack Tauson
Ranch Hand
Posts: 138
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rob Spoor wrote:

Jack Tauson wrote:Q1) Since I'll be connecting to a different server using FTP/SFTP client ( getting more inclined towards SFTP after reading everyone's responses), I would have to put my user credentials in the code. Is it wise to have dedicated user credentials for this specific connection because I don't want to put my user credentials in the code.?


Definitely. That way, you can also further limit what this user can do.

By the way, SFTP allows you to authenticate through a public key, if the server side is setup to allow this. That way you don't have to store a password (however, you do need to store the key pair). See https://www.ssh.com/ssh/key for more information.

Q2) Secondly, I found FTP client related libarries online in java like https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html
However, I couldn't find anything similar to that when I tried for SFTP. The thing I found is this:

https://www.baeldung.com/java-file-sftp

Which option would be good if I have to go with SFTP option mentioned in the above link? Thanks again !


I already mentioned JSch, which is also the first entry on that page. Apache Commons VFS uses JSch so I'd only use it to provide a more easy API. I don't know SSHJ so I can't tell you anything about it.



Thanks again.

Could anyone tell me what file servers I would need in order to sftp /ftp to the server where I'm planning to put the files? Thanks !
 
Tim Holloway
Saloon Keeper
Posts: 21710
148
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It depends on the platform. There are many FTP servers. Probably the one I'd use for Linux these days is ProFTP.

Note: TFTP is a completely different protocol. It helps with PXE boot systems, for example. Also GridFTP is an extended FTP designed for grid computing. You'll just want plain ordinary FTP.
 
Rob Spoor
Sheriff
Posts: 21863
105
Eclipse IDE Spring VI Editor Chrome Java Ubuntu Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The chances are high that your RHEL server already runs SSHD, so that means you have to do nothing to already be able to use SFTP with your own account. All you'd need to do is set up another account just for these uploads.
 
Ruth Stout was famous for gardening naked. Just like this tiny ad:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!