Assignment adapted from James Allan's CMPSCI 646 course (Fall, 2004) at U. Mass and Doug Oard's LBSC 796/INFM 718R (Spring 2011) at UMD.
Last updated: August 15 2012.
The purpose of this exercise is to gain some "hands-on" experience in the process of evaluating information retrieval systems. You will be assessing documents that are retrieved in response to two "topics" (statements of information needs). The two search engines we'll be comparing are Google and Bing.
online degrees benefits tuition quality legitimacy
Many universities are now offering online degrees in addition to campus degrees.
What are the benefits of getting online degrees? Relevant Web pages must address tuition, quality,
and legitimacy (i.e., acceptability by employees) of online-degree programs.
[Google results] [Bing results]
one-day tour in New Orleans in summer
A French friend of yours is coming to
attend a conference in New Orleans for his first time next summer. He wishes to spend one
day in New Orleans to tour the city. What is a good plan for his one-day tour in New Orleans next summer?
[Google results] [Bing results]
To ensure that everyone evaluates the same hits, results from each search engine have been cached for you; follow the above links. You will evaluate the relevance of a subset of these hits. If your social security number ends in an even digit, evaluate the first 15 hits of each query. If you social security number ends in an odd digit, evaluate the last 15 hits of each query.
Use this Excel spreadsheet to keep track of your relevance judgments. Note that there are 4 spreadsheets in the file, one for each query. In the column marked "Relevance", enter "R" if you think the document is relevant. Enter "N" if you think the document is not relevant. Unaccessible Web pages are considered non-relevant. Add the spreadsheet to your homework Web page. Change the name of the spreadsheet to your last name followed by your first initial so that we don't wind up with a dozen files with the same filename.