• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • paul wheaton
  • Ron McLeod
  • Jeanne Boyarsky
Sheriffs:
  • Paul Clapham
Saloon Keepers:
  • Tim Holloway
  • Roland Mueller
Bartenders:

Fetching Webpage

 
Greenhorn
Posts: 7
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The following program fetches webpage using "URLConnection"
Usage: java FetchPage [-u validURL] [-f Outputfile]
Note : -u : used for giving URL ; -f : used for storing fetched content to a local file.
For Example 1) java FetchPage -u https://www.paypal.com/
2) java FetchPage -u https://www.paypal.com/ -f outputfile.html
-----------------------------------------------------------------------------
import java.net.*;
import java.io.*;

public class FetchPage
{

static URL my_url;
static URLConnection my_urlconn = null;
static BufferedWriter bout =null;
static BufferedReader br = null;
static String outputfilename =" ", strurl = " ";
static int uflag = 0, fflag = 0;
// Deals with Usage: java FetchPage -u validURL -f outputfilename
public static void check(String[] args) throws FileNotFoundException,MalformedURLException,Exception
{

int len = args.length;
if (len == 2 || len == 4)
{
for (int i=0; i<args.length; i++)
{

if (args[i].equals("-u") && i+1<len) { strurl = args[i+1] ; uflag = 1;}

if (args[i].equals("-f") && i+1<len) { outputfilename = args[i+1]; fflag = 1; }
}

}
else
{
System.out.println("USAGE: java FetchPage [-u validURL] [-f outputfilename] ");

System.exit(-1);
}
}


//Fetching the webpage [& storing to local file when -f filename Option is used]
public static void main(String[] args)
{
int cnt=0;
try
{
check(args);
System.out.println("URL---> " + strurl);
my_url = new URL(strurl);
my_urlconn = my_url.openConnection();
br = new BufferedReader(new InputStreamReader(my_urlconn.getInputStream()));
if (fflag==1) bout = new BufferedWriter(new FileWriter(outputfilename)); // for -f Option
String tmp;
tmp = br.readLine();
while(tmp != null)
{
cnt++;
System.out.println("" + tmp);
if (fflag==1) bout.write(tmp); // for -f Option
tmp = br.readLine();
}
System.out.println("---------------------------------------------------------------");
System.out.println(" Lines Fetched ->>: " + cnt);
System.out.println(" URL = " + strurl);
System.out.println(" File Flag Value = " + fflag + " URL Flag Value = " + uflag);
System.out.println("---------------------------------------------------------------");
}catch (FileNotFoundException e) {System.out.println("ERROR: Invalid URL Exception" + e.getMessage()) ; }
catch (MalformedURLException m){
System.out.println( "ERROR: Malformed URL Exception " + m.getMessage());
}catch (Exception e) {e.printStackTrace();
}
finally {
try
{
if (bout != null)
bout.close();
if (br != null)
br.close();
} catch (Exception e) {};
}
}
}
[ August 10, 2008: Message edited by: Basheer Yunus ]
 
permaculture is a more symbiotic relationship with nature so I can be even lazier. Read tiny ad:
Clean our rivers and oceans from home
https://www.kickstarter.com/projects/paulwheaton/willow-feeders
reply
    Bookmark Topic Watch Topic
  • New Topic