Effective Web Searching
HomeLibrary › Effective Web Searching FaceBookTwitterYouTube
Page Outline

Search Engines
Subject Directories
Invisible Web
Evaluating Web Information Resources
Finding Dead Links
More Information

Search Engines

A search engine is a searchable database of Internet files collected by a computer program (called a crawler, robot, worm, spider). Indexing is automatically created from the collected files. There are no selection criteria for the collection of files.

Google seems to be everyone’s search engine of choice. Be sure to take advantage of Google’s search features and advanced search capabilities to enhance your searching. Check these links on Google's web site for more information:

Google Web Search Features
Google Advanced Search
Google Advanced Search Operators

Use Google Scholar to search across many disciplines and sources for peer-reviewed papers, theses, books, abstracts, and articles available from academic publishers, professional societies, preprint repositories, universities, and other scholarly organizations.

Also check out the Google Guide. Produced by Nancy Blachman, this is a handy web guide to using Google. The site is designed for searchers who want to understand how Google actually does its job. Coverage focuses on functionality and features.

However, Google is not the only game in town. Search engines are more different than most people think. There is actually not a large overlap in results. Check it out for yourself using these search engine comparison tools:  Langreiter.com and Thumbshots Ranking. They search across various search engines and provide a graphical comparison of where the results fall in each search engine’s ranking. Gooyaglehoo compares search results from Google and Yahoo in a split screen display.

The comparison tools listed above provide interesting information about various search engines. The lesson to be learned is: For the most effective searching, use more than one search engine. Directly search each one for best results rather than using one of the meta-search engines. And don’t forget to explore and use the advanced features of the search engines that you use.

Here are some other search engines to try:


Subject Directories

Subject directories are collections of many links on a topic. They are compiled with human input and can help locate "the best" on a topic. You are probably already familiar with two popular subject directories for law: Findlaw and Cornell Law School's Legal Information Institute. The UC Berkeley Teaching Library provides a good list of academic-oriented subject directories. The Cowles Library web page also has a list of sources (most of which are subject directories) for finding more scholarly web sources: Internet Searching for Academic Content.

Invisible or Deep Web

The "visible web" is what you can retrieve or "see" in the search results of search engines. The "invisible (or deep) web" is the information that you cannot retrieve ("see"). Searchable databases make up a large part of the invisible web. The invisible web is estimated to offer two to three times as many pages as the visible web.

Searchable databases and other invisible web material can be located by searching in most general web subject directories. Adding "database" to your search terms in Google may help you find searchable databases in your area of interest. These are a few useful sites for finding the searchable databases that the invisible web has to offer:
OAIster (academically-oriented resources)
direct search
For more information about the invisible web, consult the Invisible or Deep Web section of UC Berkeley Teaching Library’s online tutorial on Finding Information on the Internet.

Finding Dead Links

Web pages come and go and addresses change. Multiple studies have found that a high percentage of web sites cited in academic articles are no longer available within four years. (See discussion in Nicholas Tomaiuol, DOIs, URLs, LoCKSS, and Missing Links, 14 Searcher 18 (July/Aug 2006). Other studies have documented similarly high rates of "link rot" of web sites cited in law reviews (Mary Rumsey, Runaway Train: Problems of Permanence, Accessibility, and Stability in the Use of Web Sources in Law Review Citations, 94 L. Lib. J. 27 (2002)) and even judicial opinions (Coleen M. Barger, On the Internet, Nobody Knows You’re a Judge: Appellate Courts’ Use of Internet Materials, 4 J. App. Prac. & Process 417 (2002)).

Here are some tips for accessing web pages that no longer exist at the linked or cited location:
If you can no longer access a cited URL, try the following:
  • Double-check the URL. Make sure you typed the address as cited, including using the same case (upper or lower).
  • Search the root page. Truncate the URL by deleting information up to each back slash and see if the page has been moved elsewhere on the site by browsing or using the site index or site search feature, if available. (In the Google advanced search and Yahoo! Advanced search options, you can also specify the site to search.)
  • Try again later. Check the URL as cited in a day or two. The site may just be down temporarily.
  • Check Google cache. Google stores an older version of most web pages, and these can be retrieved by using the following search string: cache:WebURL.
  • Check the Internet Archive Wayback Machine. This site provides access to archived versions of web sites.
  • Check the CyberCemetery. This site provides "permanent public access to the web sites and publications of defunct U.S. government agencies and commissions."
  • Use Internet search tools. Simply using a search engine to try to find the new web site address sometimes works. As noted above, different search engines can yield different results, so start with your favorite search tool but also try others if needed.  In addition to those listed above, Search Engine Watch offers annotated links to a number of search engines. Also, be sure to explore advanced search features and read the search tips for greater effectiveness and efficiency.
  • For additional tips on searching for missing pages, see the Living Internet's page Find a Missing Web Link.
Evaluating Web Information

Searching the web is a quick and easy way to find information. However, when researching for class, co-curricular or work projects, it is important to make sure the information you find is current, factual, and authoritative. This is not as easy to do with web resources as it is with traditionally published materials. The sites listed below provide many helpful tips and techniques to help you evaluate the information you find on the web.

Guidelines for Evaluating Government Information on the Web
Developed by an American Association of Law Libraries committee, these guidelines are specific to legal and other governmental information on the Web.

Evaluating Web Pages: Techniques to Apply & Questions to Ask
This very helpful site is organized to combine two techniques into a process that begins with looking at your search results from a search engine or other source, follows through by investigating the content of page, and extends beyond the page to what others may say about the page or its author(s). (Prepared by UC Berkeley Teaching Library)

Evaluating Web Resources
Provides evaluation checklists for advocacy web pages, business marketing web pages, news web pages, informational web pages and personal web pages, with examples of actual web pages. Includes a bibliography for web evaluation techniques and links to additional web evaluation sites. Prepared by Jan Alexander & Marsha Tate, Reference Librarians, Wolfgram Memorial Library of Widener University, Chester, Pennsylvania.

How to Evaluate a Web Page
Provides a list of what to look for in evaluating web pages. Prepared by Colorado State University Libraries. 

More Information

More information about choosing and using web search tools is available at these sites:

Recommended Search Engines
This resource assists users in choosing the best tools to use for different kinds of searches. It defines search engines, search directories, and the "invisible web" and offers tips on when to use what. Highlight of the site is a detailed features table that explains Boolean syntax, phrase searching, truncation, and field searching, among other features, for several services. (Prepared by UC Berkeley Teaching Library)

Searching the Internet: Recommended Sites and Search Techniques
Written by Laura Cohen (librarian at the State University of New York at Albany), this tutorial explores search engines, subject directories, and the "invisible web" to help you gain skills in conducting research on the Internet.

Prepared by Sue Lerdal, Reference Librarian
with assistance from
Karen Wallace, Circulation/Reference Librarian
Updated by, Sara Lowe, Reference Librarian
Last Revised: July 2009, SL
Please feel free to e-mail the author with suggestions for improving this guide.

Last Modified: 11/18/2009 2:48:00 PM by Karen Wallace