Home > Publications
Home University of Twente
Prospective Students
Intranet (internal)

EEMCS EPrints Service

15904 The Combination and Evaluation of Query Performance Prediction Methods
Home Policy Brochure Browse Search User Area Contact Help

Hauff, C. and Azzopardi, L. and Hiemstra, D. (2009) The Combination and Evaluation of Query Performance Prediction Methods. In: Proceedings of the 31st European Conference on IR Research on Advances in Information Retrieval, 6-9 April 2009, Toulouse, France. pp. 301-312. Lecture Notes In Computer Science 5478. Springer Verlag. ISBN 978-3-642-00957-0

Full text available as:

- Univ. of Twente only
237 Kb

Official URL:

Exported to Metis


In this paper, we examine a number of newly applied methods for combining pre-retrieval query performance predictors in order to obtain a better prediction of the query's performance. However, in order to adequately and appropriately compare such techniques, we critically examine the current evaluation methodology and show how using linear correlation coefficients (i) do not provide an intuitive measure indicative of a method's quality, (ii) can provide a misleading indication of performance, and (iii) overstate the performance of combined methods. To address this, we extend the current evaluation methodology to include cross validation, report a more intuitive and descriptive statistic, and apply statistical testing to determine significant differences. During the course of a comprehensive empirical study over several TREC collections, we evaluate nineteen pre-retrieval predictors and three combination methods.

Item Type:Conference or Workshop Paper (Full Paper, Talk)
Research Group:EWI-HMI: Human Media Interaction, EWI-DB: Databases
Research Program:CTIT-NICE: Natural Interaction in Computer-mediated Environments
Research Project:MultimediaN/N5: Semantic access
ID Code:15904
Deposited On:31 August 2009
More Information:statisticsmetis

Export this item as:

To request a copy of the PDF please email us request copy

To correct this item please ask your editor

Repository Staff Only: edit this item