Maverick Nod

In our effort to track down information on the Web, we employ an arsenal of tools. We may browse the handpicked sites of a directory like Yahoo!  or the results of a search engine like Google . Some cast an even wider net by using metasearch tools like Search.com  or ixquick . And yet we still don't always find what we need.

This material has been referred to as the Hidden Internet, the Deep Web or the Invisible Web. The Invisible Web is comprised of information stored in databases, according to Chris Sherman, Webmaster of About.com's Web Search . Spiders and robots cannot enter these databases.

"It's as if they've run smack into the entrance of a massive library with securely bolted doors," Sherman said. "Spiders can record the library's address, but can tell you nothing about the books, magazines or other documents."

What else makes up the Invisible Web?

• Non-HTML files (PDF files, etc.)
• Webbed databases
• Sites requiring registration or login
• Archives (newspapers and magazines, etc.)
• Dynamically created Web pages
• Interactive tools (calculators, etc.)

Thankfully, gateways have been developed to mine these hidden troves. Think of them as database or search tool finders. Here are a few of the best:

• Direct Search
  Maintained by librarian Gary Price, this site provides direct access to the search interfaces of Invisible Web resources.
  
  Invisible Web 
• A Yahoo!-like directory of more than 10,000 specialized databases (business, legal, travel, etc.). Includes free- and fee-based resources.
• Internet Oracle
  Search forms and direct links to hundreds of search engines, from general purpose directories to niche topic indexes. Categories of databases include Reference and Research, Medical, Maps, Legal and Government.