Recommendation system

While studing recommendation system Here, the question raised how to compare recommenders????

Based on a set of properties that are relevant for the application.We have three types of experiments.

 Offline Setting
An offline setting, where recommendation compared without user interaction, then reviewing user studies, where a small experiments done with the system and report on the experience, and finally describe large scale online experiments, where real user populations interact with the system
Often it is easiest to perform of?ine experiments using existing data sets and a protocolthatmodelsuserbehaviortoestimaterecommenderperformancemeasures such as prediction accuracy.

 OfflineExperiments
An offline experiment is performed by using a pre-collected dataset of users choosing or rating items. Using this data set we can try to simulate the behavior of users that interact with a recommendation system. In doing so, we assume that the user behavior when the data was collected will be similar enough to the user behavior when the recommender system is deployed, so that we can make reliable decisions based on the simulation the goal of the offline experiments is to alter out inappropriate approaches, leaving a relatively small set of candidate algorithms to be tested by the more costly user studies or online experiments.

 UserStudies
A user study is conducted by recruiting a set of test subject, and asking them to perform several tasks requiring an interaction with the recommendation system. While the subjects perform the tasks, we observe and record their behavior, collecting any number of quantitative measurements, such as what portion of the task was completed, the accuracy of the task results, or the time taken to perform the task.