Field Detail

• DEFAULT_MAX_TOKEN_LENGTH

  public static final int DEFAULT_MAX_TOKEN_LENGTH

  Default maximum allowed token length

  See Also:

  Constant Field Values

• STOP_WORDS_SET

  public static final Set<?> STOP_WORDS_SET

  An unmodifiable set containing some common English words that are usually not useful for searching.

Constructor Detail

• StandardAnalyzer

  public StandardAnalyzer​(Version matchVersion,
                          Set<?> stopWords)

  Builds an analyzer with the given stop words.

  Parameters:

  matchVersion - Lucene version to match See {@link above}

  stopWords - stop words

• StandardAnalyzer

  public StandardAnalyzer​(Version matchVersion)

  Builds an analyzer with the default stop words (STOP_WORDS_SET).

  Parameters:

  matchVersion - Lucene version to match See {@link above}

• StandardAnalyzer

  @Deprecated
  public StandardAnalyzer​(Version matchVersion,
                          File stopwords)
                   throws IOException

  Deprecated.

  Use StandardAnalyzer(Version, Reader) instead.
  Builds an analyzer with the stop words from the given file.

  Parameters:

  matchVersion - Lucene version to match See {@link above}

  stopwords - File to read stop words from

  Throws:

  IOException

  See Also:

  WordlistLoader.getWordSet(Reader, Version)

• StandardAnalyzer

  public StandardAnalyzer​(Version matchVersion,
                          Reader stopwords)
                   throws IOException

  Builds an analyzer with the stop words from the given reader.

  Parameters:

  matchVersion - Lucene version to match See {@link above}

  stopwords - Reader to read stop words from

  Throws:

  IOException

  See Also:

  WordlistLoader.getWordSet(Reader, Version)

Method Detail

• setMaxTokenLength

  public void setMaxTokenLength​(int length)

  Set maximum allowed token length. If a token is seen that exceeds this length then it is discarded. This setting only takes effect the next time tokenStream or reusableTokenStream is called.

• getMaxTokenLength

  public int getMaxTokenLength()

  See Also:

  setMaxTokenLength(int)

• createComponents

  protected ReusableAnalyzerBase.TokenStreamComponents createComponents​(String fieldName,
                                                                        Reader reader)

  Description copied from class: ReusableAnalyzerBase
  Creates a new ReusableAnalyzerBase.TokenStreamComponents instance for this analyzer.

  Specified by:

  createComponents in class ReusableAnalyzerBase

  Parameters:

  fieldName - the name of the fields content passed to the ReusableAnalyzerBase.TokenStreamComponents sink as a reader

  reader - the reader passed to the Tokenizer constructor

  Returns:

  the ReusableAnalyzerBase.TokenStreamComponents for this analyzer.