LuceneQE

Query Expansion Module for Lucene based on BM25 ranking function and an extension for query clauses

We have implemented the state of the art query expansion techniques combined with a new family of ranking functions which can take into account some semantic structure in the query.

You can downloaded the code from the next links:

• LuceneQE.tar.gz (Source Code)
• LuceneQE.jar or Sourceforge project LuceneQE
• Javadoc

This code has been implemented by José Ramón Pérez Agüera and Hugo Zaragoza This implementation is only for research and experimental projects. If you are looking for a robust BM25 implementation for Lucene, we recommend you the project leaded by Joaquin Perez-Iglesias, which had been the reference for our Lucene-BM25 implementation.

If you are interested in this work, you can find a scientific description of our approach in:

• Jose R. Perez-Aguera, Hugo Zaragoza, Lourdes Araujo. Exploiting morphological query structure using Genetic Optimization. NLDB 2008 13th International Conference on Applications of Natural Language to Information Systems. Proceedings will be published in Lecture Notes in Computer Science (LNCS) Springer Verlag. [PDF]
• Jose R. Perez-Aguera Hugo Zaragoza. UCM-Y!R at CLEF 2008 Robust and WSD tasks. CLEF 2008 Workshop, 17-19 September, Aarhus, Denmark. (Aceptado como Selecter Paper en Lecture Notes in Computer Science. Springer) Disponible en http://www.clef-campaign.org/2008/working_notes/aguera-paperCLEF2008.pdf.

You can also visit LuceneQE wiki (documentation,FAQ,discussion,...) to obtain more information about this project.

You will need the following libraries to use our implementation:

• Lucene-core-2.3.0.jar
• Lucene-highlighter-2.3.0.jar
• Lucene-memory-2.3.0.jar
• snowball-1.0.jar
• wn.jar

Query Expansion

• BM25
• Kullback-Liebler Divergence for Query Expansion
• Divergence From Randomness for Query Expansion
• Rocchio like reweigthing method

Clauses based Query Representation

• BM25 with clauses for structured queries

How to use our code

• BM25 Example
• Query Expansion without structure Example
• Query Expansion with structure Example