• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Bear Bibeault
  • Liutauras Vilda
  • Devaka Cooray
Sheriffs:
  • Knute Snortum
  • Junilu Lacar
  • Henry Wong
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Carey Brown
  • Tim Holloway
Bartenders:
  • salvin francis
  • Frits Walraven
  • Piet Souris

web crawler compilation problem

 
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hi all,

i was looking at the web crawler in hereand source code here. i tried running it but got following error:

Exception in thread "main" java.lang.Error: Unresolved compilation problem:

at crawler.WebCrawler.main(WebCrawler.java:32)

i corrected the old main to "public static void main(String[] args) {" but still got error.

please advice. i am using eclipse.




 
author
Posts: 23867
141
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Exception in thread "main" java.lang.Error: Unresolved compilation problem:

at crawler.WebCrawler.main(WebCrawler.java:32)



Eclipse has the capability to run something that didn't compile right. This just means that you have a compile error, tried to run it anyway, and ran right into the code that didn't compile. Basically, the solution is to fix the compile errors before you run it.


i corrected the old main to "public static void main(String[] args) {" but still got error.



What was wrong with the old main() ? It looks fine to me. And all you seem to have done was change the name of the argument -- what is that suppose to do?

please advice. i am using eclipse.



I would suggest that you fix the code that is labelled in red. Eclipse marks all error in red, and on the left of the editor, provides a ballon, with a tooltip, telling what is wrong with the line.

Henry
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you for your reply. but there is no red line under the line giving error and

i only got this message as an error.

Exception in thread "main" java.lang.Error: Unresolved compilation problem:

at crawler.WebCrawler.main(WebCrawler.java:32)

 
Henry Wong
author
Posts: 23867
141
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

thank you for your reply. but there is no red line under the line giving error and



If I wasn't clear before -- ignore the error that is printed when the program is ran. It is useless here. Scroll down the whole file. Find the lines marked in red. And fixed them all.

Do *not* run the program again until all the red marks are gone!! Running a program with compile errors don't give you any errors that are usable -- fix all the compile errors.

Henry
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you for your prompt reply again.

is there anything wrong with the importing statement(referring to the print screen)? because eclipses never underline any line and i don think there is any problem with the code as well.

i got the code from java tutorial here

Picture-1.png
[Thumbnail for Picture-1.png]
 
Henry Wong
author
Posts: 23867
141
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, when you put the pointer on the left ballon, what does the tooltip say? And how do you suppose to fix that?


BTW, after you fix this, you should get five red errors appearing... which can be all fixed together by simply fixing the first one.... so don't panic.

Henry
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ok! thank you henry for your time. really appreciate it and the code is working now.

just added package crawler in .
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
but i have another question.

the program don't seems to be working. i typed a few url and it never show any result.

anybody knows why?

i have included the code here. thank you

 
Marshal
Posts: 67930
258
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That is really old code, before Java 1.2. You have to change both references to List to java.awt.List; java.util.List hadn't been introduced yet.
I suggest you find a newer web crawler with a decent explanation of how it works.
 
Henry Wong
author
Posts: 23867
141
jQuery Eclipse IDE Firefox Browser VI Editor C++ Chrome Java Linux Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This code looks to be a testing webcrawler, for a particular location, and for particular sites. It doesn't look like it was meant to be a general purpose crawler.

First of all, this code...



Sets an http proxy -- which implies that the code is supposed to run from a location that is behind a particular firewall.

Second of all, this code...



Checks and confirms a robot.txt file to double check if it is allowed to crawl the site and how to crawl the site (I guess this was a growing standard years ago). Regardless, many sites, like yahoo, doesn't have this file, so the crawling will abort. You should check to see if this may be the cause in your case.

My guess is, this code was probably used for training, and to be run only from the training site, and only to target specific testing sites.

Henry
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you henry and Campbell Ritchie !

does anybody has a site that teach robots or web crawler to recommend? thank you again.
 
Rancher
Posts: 43011
76
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Some of these may fit the bill: http://java-source.net/open-source/crawlers
 
bryan lim
Ranch Hand
Posts: 140
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
thank you!! i learn so much from coderanch.!!

i am wondering can i ask gwt question in coderanch?? why don't the forum open a section on gwt?? it will be great !!
 
author
Posts: 3281
8
Mac OS X Eclipse IDE Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
We basically open new forums once we get several questions on a new topic. A this stage we haven't seen enough Q's on GWT for it to warrant it's own forum. So in short please post GWT questions to the "Application Frameworks" forum
 
Beware the other head of science - it bites! Nibble on this message:
Java file APIs (DOC, XLS, PDF, and many more)
https://products.aspose.com/total/java
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!