<world-wide web> (Or "robot", "crawler") A program that automatically explores the World-Wide Web by retrieving a document and recursively retrieving some or all the documents that are referenced in it. This is in contrast with a normal web browser operated by a human that doesn't automatically follow links other than inline images and URL redirection.

The algorithm used to pick which references to follow strongly depends on the program's purpose. Index-building spiders usually retrieve a significant proportion of the references. The other extreme is spiders that try to validate the references in a set of documents; these usually do not retrieve any of the links apart from redirections.

The standard for robot exclusion is designed to avoid some problems with spiders.

Early examples were Lycos and WebCrawler.


(01 Jun 2001)

spiculispongiae, spiculum, spicy, SPID < Prev | Next > spider, spider angioma, spider angiomas

Bookmark with: icon icon icon icon iconword visualiser Go and visit our forums Community Forums

1. <zoology> Any one of numerous species of arachnids comprising the order Araneina. Spiders have the mandibles converted into poison fangs, or falcers. The abdomen is large and not segmented, with two or three pairs of spinnerets near the end, by means of which they spin threads of silk to form cocoons, or nests, to protect their eggs and young. Many species spin also complex webs to entrap the insects upon which they prey. The eyes are usually eight in number (rarely six), and are situated on the back of the cephalothorax.

Spiders are divided into two principal groups: the Dipneumona, having two lungs: and the Tetrapneumona, having four lungs. See Mygale. The former group includes several tribes; as, the jumping spiders (see Saltigradae), the wolf spiders, or Citigradae (see under Wolf), the crab spiders, or Laterigradae (see under Crab), the garden, or geometric, spiders, or Orbitellae (see under Geometrical, and Garden), and others. See Bird spider, Grass spider, House spider, Silk spider.

2. <zoology> Any one of various other arachnids resembling the true spiders, especially certain mites, as the red spider (see under Red).

3. An iron pan with a long handle, used as a kitchen utensil in frying food. Originally, it had long legs, and was used over coals on the hearth.

4. A trevet to support pans or pots over a fire.

5. <machinery> A skeleton, or frame, having radiating arms or members, often connected by crosspieces; as, a casting forming the hub and spokes to which the rim of a fly wheel or large gear is bolted; the body of a piston head; a frame for strengthening a core or mold for a casting, etc. Spider ant.

Any one of numerous species of South American monkeys of the genus Ateles, having very long legs and a long prehensile tail.

<botany> Spider orchis, any shell of the genus Pteroceras. See Pteroceras.

Origin: OE. Spire, fr. AS. Spinnan to spin; so named from spinning its web; cf. D. Spin a spider, G. Spinne, Sw. Spindel. Seee Spin.

(01 Mar 1998)

spiculispongiae, spiculum, spicy, SPID, spider < Prev | Next > spider angioma, spider angiomas, spider-burst

Bookmark with: icon icon icon icon iconword visualiser Go and visit our forums Community Forums