( Lucene can also be used to index database table rows. The advantage for having Lucene search instead of (database software’s) in-build full-text search engine is Lucene ranks search results based on their relevancy. For example assume you have a product table with <title , description> fields and you want to give higher rank to ‘title’ over ‘description’. )

A stopword is a language-word that has no significance meaning in a keyword based search system (e.g. Google). Lucene also has a set of such words for English language and these are simply ignored while analyzing/tokenizing text. You can find them inside org/apache/lucene/analysis/StopAnalyzer.java file declared as StopAnalyzer.ENGLISH_STOP_WORDS constant.

public static final String[] ENGLISH_STOP_WORDS = {
"a", "an", "and", "are", "as", "at", "be", "but", "by",
"for", "if", "in", "into", "is", "it",
"no", "not", "of", "on", "or", "such",
"that", "the", "their", "then", "there", "these",
"they", "this", "to", "was", "will", "with"
};

You can also specify your own stopwords while indexing text. Use StandardAnalyzer’s constructor and pass a set of words as agrument. These will be ignored while indexing.

- ankit