• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Spiders????

 
Ranch Hand
Posts: 92
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Everybody!
Can anybody throw some light on what is 'spider' all about?I know they r used in the concept of search engine.
Thanx for any reply.
Preethi.
 
Sheriff
Posts: 7001
6
Eclipse IDE Python C++ Debian Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
"spider" is effectively a synonym for "webcrawler". Both describe a piece of software which finds links in a web page and follows them to other web pages.
In the context of a search engine, imagine that you wish to set up a search engine for your own web site. For any kind of decent performance you really need an index, but there are several ways of creating that index. You could manually run some program on each web page, but you might forget a page, or forget to regenerate it when a page changes. You might also accidentally index a page that should not be publicly visible.
A better way might be to set some software pointing at your "home" page which indexes all the words on that page, then looks for all the links to other pages on your site and repeats the process. When it has finished it will have indexed all the publicly visible pages on your site.
You can then set this "indexing spider" to run from time to time (say first thing every monrning) so that whatever changes you make to the structure or content of your web site, the search index will always be up to date.
 
Preethi Suryam
Ranch Hand
Posts: 92
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Frank!
Thanx for ur reply.now i got a vague idea about spiders. what i understood about it is it's a program which tracks all the links and stores them, and finally we use those results for our search engine.i am sorry i was not able to get a more clear picture about this.anyway thanx once again.
Have a Nice Day!
Preethi.
 
I'm sure glad that he's gone. Now I can read this tiny ad in peace!
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic