Are pay for performance search engines relevant?

Pay for performance (PFP) search engines, like their 'traditional' counterparts (e.g. Google), provide search services for documents on the World Wide Web. These search engines, however, rank documents not on content characteristics but according to the amount of money a vendor is willing to pay when a user visits a web site appearing in the search results page. A study was conducted to compare the retrieval effectiveness of Overture (formerly GoTo, a PFP search engine) and Google (a traditional search engine) from an academic perspective. Thirty-one queries from different graduate-level subject areas were submitted to each of these search services and the first 20 documents returned were retrieved and analysed for precision and distribution of relevant documents using different relevancy criteria. Results indicate that Google outperformed Overture in both categories. Implications of this study are also discussed.