Win a copy of The Journey To Enterprise Agility this week in the Agile and Other Processes forum! And see the welcome thread for 20% off.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Jeanne Boyarsky
  • Liutauras Vilda
  • Campbell Ritchie
  • Tim Cooke
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Junilu Lacar
  • Knute Snortum
Saloon Keepers:
  • Ron McLeod
  • Ganesh Patekar
  • Tim Moores
  • Pete Letkeman
  • Stephan van Hulst
Bartenders:
  • Carey Brown
  • Tim Holloway
  • Joe Ess

Breadth first search  RSS feed

 
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello Friends,

I have to implement a breadth-first web crawler. I have to stop the search based on the level which will be given by user. Not sure how to implement this logic? Please advice if you have any thoughts on this.

Thanks in advance.

Regards
~KK
 
Marshal
Posts: 59684
187
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You would do well to write down the algorithm in simple words on paper. The pseudo-code method. Also draw a picture of a tree, and go over it with a pencil showing the order you are visiting the nodes.
 
Johny Kk
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I am doing it now. But somehow I am not able to catch the logic.

Thanks
 
Campbell Ritchie
Marshal
Posts: 59684
187
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This is what I found on Google: 1 2 3. And lots more.
 
Sheriff
Posts: 21421
94
Chrome Eclipse IDE Java Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If you need to find out at which level a URL is, just store it with the URL. Create a custom class that holds both the level of the URL and the URL itself.

One thing to think about: what will you do when a URL at level 2 links back to a URL on level 1? Do you skip it (as you've already searched it), or do you check it again as level 3?
 
Johny Kk
Greenhorn
Posts: 8
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks Campbell and Rob.

@ Rob... I will skip it. I have to store only unique URL's. I have a custom class which does this for me.

Thanks
~KK
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!