The Wayback Machine - https://web.archive.org/web/20180613203647/https://link.springer.com/chapter/10.1007%2F978-1-4471-2099-5_18

SIGIR ’94 pp 173-181 | Cite as

Automatic Combination of Multiple Ranked Retrieval Systems

  • Brian T. Bartell
  • Garrison W. Cottrell
  • Richard K. Belew
Conference paper

Abstract

Retrieval performance can often be improved significantly by using a number of different retrieval algorithms and combining the results, in contrast to using just a single retrieval algorithm. This is because different retrieval algorithms, or retrieval experts, often emphasize different document and query features when determining relevance and therefore retrieve different sets of documents. However, it is unclear how the different experts are to be combined, in general, to yield a superior overall estimate. We propose a method by which the relevance estimates made by different experts can be automatically combined to result in superior retrieval performance. We apply the method to two expert combination tasks. The applications demonstrate that the method can identify high performance combinations of experts and also is a novel means for determining the combined effectiveness of experts.

Keywords

Average Precision Retrieval Performance Retrieval Algorithm Relevance Judgement Relevance Estimate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Brian T. Bartell. Optimizing Ranking Functions: A Connectionist Approach to Adaptive Information Retrieval. PhD thesis, Department of Computer Science & Engineering, The University of California, San Diego, 1994.Google Scholar
  2. 2.
    Brian T. Bartell, Garrison W. Cottrell, and Richard K. Belew. Learning the optimal parameters in a ranked retrieval system using multi-query relevance feedback. In Proceedings of the Symposium on Document Analysis and Information Retrieval,Las Vegas, 1994. in press.Google Scholar
  3. 3.
    Nicholas J. Belkin, C. Cool, W. Bruce Croft, and James P. Callan. Effect of multiple query representations on information retrieval system performance. In Proc. SIGIR 1993, pages 339–346, Pittsburgh, PA, June 1993.CrossRefGoogle Scholar
  4. 4.
    I. Borg and J. Lingoes. Multidimensional Similarity Structure Analysis. Springer-Verlag, New York, 1987.CrossRefGoogle Scholar
  5. 5.
    Edward A. Fox, M. Prabhakar Koushik, Joseph Shaw, Russell Modlin, and Durgesh Rao. Combining evidence from multiple searches. In Donna K. Harman, editor, The First Text REtrieval Conference (TREC-1), pages 319–328, March 1993. NIST Special Publication 500–207.Google Scholar
  6. 6.
    Norbert Fuhr and Chris Buckley. A probabilistic learning approach for document indexing. ACM Transactions on Information Systems, 9 (3): 223–248, 1991.CrossRefGoogle Scholar
  7. 7.
    Michael Gordon. Probabilistic and genetic algorithms in document retrieval. Communications of the ACM, 31 (10), October 1988.Google Scholar
  8. 8.
    L. Guttman. What is not what in statistics. The Statistician, 26: 81–107, 1978.CrossRefGoogle Scholar
  9. 9.
    Donna Harman. Overview of the first Text REtrieval Conference. In Proceedings of the ACM SIGIR, pages 36–48, Pittsburgh, PA, June 1993.Google Scholar
  10. 10.
    J. Katzer„M. J. McGill, J. A. Tessier, W. Frakes, and P. DasGupta. A study of the overlap among document representations. Information Technology: Research and Development, 1 (4): 261–274, Oct 1982.Google Scholar
  11. 11.
    David D. Lewis and W. Bruce Croft. Term clustering of syntactic phrases. In Proceedings of the ACM SIGIR, Brussels, Sept 1990.Google Scholar
  12. 12.
    William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling. Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, 1988.Google Scholar
  13. 13.
    T. Saracevic and P. Kantor. A study of information seeking and retrieving. III. Searchers, searches, overlap. Journal of the ASIS, 39 (3): 197–216, 1988.Google Scholar
  14. 14.
    Paul Thompson. A combination of expert opinion approach to probabilistic information retrieval, part 1: The conceptual model. Information Processing El Management, 26 (3): 371–382, 1990.CrossRefGoogle Scholar
  15. 15.
    Paul Thompson. Description of the PRC CEO algorithm for TREC. In Donna K. Harman, editor, The First Text REtrieval Conference (TREC-1), pages 337–342, March 1993. NIST Special Publication 500–207.Google Scholar
  16. 16.
    Howard Turtle and W. Bruce Croft. Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems, 9(3):187–222, july 1991Google Scholar
  17. 17.
    S. K. M. Wong, Y. J. Cai, and Y. Y. Yao. Computation of term associations by a neural network. In Proceedings of SIGIR, Pittsburgh, PA, June 1993.Google Scholar

Copyright information

© Springer-Verlag London Limited 1994

Authors and Affiliations

  • Brian T. Bartell
    • 1
  • Garrison W. Cottrell
    • 2
  • Richard K. Belew
    • 2
  1. 1.Advanced Technology GroupEncylopædia Britannica, Inc.La JollaUSA
  2. 2.Department of Computer Science & Engineering-0114University of California, San DiegoLa JollaUSA

Personalised recommendations