• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Deploying Java Web Crawler Application to Server

 
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I have written a simple web crawler type java application that periodically parses a website that I would like to have running 24x7 or at least something more regularly than when my laptop is not asleep. Is there a simple way to do this without diving into the intricacies of maintaining linux servers and/or rewriting the code as a web app. The application works perfectly fine as is on my computer, simply spits out a CSV text file; I just need this to be running on a more stable "always up" server. I have done quite a bit of digging but when I try to find server options to do this I get overwhelmed by the complexities of web app development and setting up a linux server via the command line (which I'm not really interested in doing). I would just like to "copy" the code to another machine, have it run and periodically be able to download the resulting text file. Any suggestions? Thanks for the help
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Welcome to the Ranch.

Been there, done exactly that (in fact, a lot more than mere downloading, data analysis and presentation as well).

You don't need to make it into a web app. In fact, that wouldn't make sense because this app does not have a GUI. You can invoke the jar file at regular intervals via a cron job (or Windows Scheduler), but yes - if you want it to run regularly then you'd want it to run on a server that's always on and online, not a desktop or laptop. You'd want to set aside a dedicated directory in which to store output files with regular names like crawl_2014_07_08_22_59.csv so it's easy to find specific ones later.
 
Henry Greenwood
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply, do you have any suggestions on an easy to use/cheap server solution where I have to do minimal setup and maintenance? Or if you can't suggest one in particular, at least the search terms I should use to find a server that will meet my needs. Thanks.
 
Ulf Dittmer
Rancher
Posts: 43081
77
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
See https://coderanch.com/how-to/java/ServletsFaq#hosting for a list of Java hosting providers. $10 per month or so should get you in the game - that's what low-end Java hosting typically costs. Look out for a "cron job" feature or "scheduled job" feature - not all hosting plans have that.

Or if cron jobs aren't available, you can turn it into a really basic web app that consists of a ServletContextListener that sets up a java.util.Timer and TimerTask to run the scheduler every so often. That should be very little work.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic