• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

problem with search engines

 
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello All,

I am having a problem with search engines.
I am able to connect to google and yahoo thro java and retrieve the html code.

say,for google the url is http://www.google.com/search?q=java

I am getting the html code when "java" is the query.

But when I click next in google(for next set of results) , the url is ..


http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

When I try to connect to this url , i am getting malformed url exceptions.

It is the same with yahoo.

Hope u understood my problem.
How can i get the html code from this url???

Can anyone help me??
 
Ranch Hand
Posts: 547
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
perhaps you can show what you coded so far ?

pascal
 
Ranch Hand
Posts: 5093
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
read up on URL encoding.
 
author and iconoclast
Posts: 24207
46
Mac OS X Eclipse IDE Chrome
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
We have a whole forum devoted to Sockets and Internet Protocols which I think is a god fit for your problem. I'm moving this thread over there.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi, this is my code.I am trying to display the html code of google search results.I am using command line arguments to give input.

i have executed like

java Demo http://www.google.com/search?q=java&hl=en&lr=&start=10&sa=N

import java.net.*;
import java.io.*;

class Demo {

public static void main(String[] args) throws Exception {
URL url = new URL(args[0]);
URLConnection conn = url.openConnection();
conn.setRequestProperty("User-Agent","");
conn.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = in.readLine()) != null)
{
System.out.print(line);
}
}
}



Why this is not working?But when i substitute the query in place of args[0] in the code,it is working.Is there anything to do with encoding?

Can anyone help me?
 
Ranch Hand
Posts: 1646
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Try printing out args[0] before you create the URL from it. Perhaps your command shell is munging the &s or some other characters.
 
vas vas
Greenhorn
Posts: 19
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
What you said is exactly right.

when I execute like java Demo "url" it is working.May be the problem is with "&".

thank you so much.I appreciate your help.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic