Why can't Google and other search engines find many articles useful for college-level research?

There are two reasons why Web search engines are unable to find many of the journal, magazine and newspaper articles available in the library's subscription databases:

1. They are locked behind publisher firewalls that block the “crawler” programs used by search engines to gather information about Web pages so that they can be found by their users.



2. The articles in many databases and Web sites accessible only by paid subscription do not have permanent URLs. Instead, URLs that include information about the password used to access the database/site or other information are generated each time a user views an article. These URLs expire when the user logs out and cannot be reused. Search engines are generally only capable of recording and pointing users to stable URLs.

The vast amount of information available in the so-called “invisible Web” (Web content that cannot be found by search engines) cannot be located directly by users using Google, Yahoo and similar sites at present.