java.lang.Object
- org.junit.Assert
- - org.apache.lucene.util.LuceneTestCase
  - - org.apache.lucene.analysis.BaseTokenStreamTestCase

```
public abstract class BaseTokenStreamTestCase
extends LuceneTestCase
```
Base class for all Lucene unit tests that use TokenStreams.
When writing unit tests for analysis components, its highly recommended to use the helper methods here (especially in conjunction with MockAnalyzer or MockTokenizer), as they contain many assertions and checks to catch bugs.

See Also:

MockAnalyzer, MockTokenizer

Nested Class Summary

Nested Classes
Modifier and Type	Class	Description
`static interface`	`BaseTokenStreamTestCase.CheckClearAttributesAttribute`	Attribute that records if it was cleared or not.
`static class`	`BaseTokenStreamTestCase.CheckClearAttributesAttributeImpl`	Attribute that records if it was cleared or not.

Nested classes/interfaces inherited from class org.apache.lucene.util.LuceneTestCase
LuceneTestCase.Nightly

Field Summary
- Fields inherited from class org.apache.lucene.util.LuceneTestCase
  classRules, random, RANDOM_MULTIPLIER, ruleChain, stores, TEMP_DIR, TEST_CLEAN_THREADS, TEST_DIRECTORY, TEST_ITER, TEST_ITER_MIN, TEST_LINE_DOCS_FILE, TEST_LOCALE, TEST_NIGHTLY, TEST_SEED, TEST_TIMEZONE, TEST_VERSION_CURRENT, testsFailed, VERBOSE

Constructor Summary

Constructors
Constructor Description

BaseTokenStreamTestCase()

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method	Description
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] posIncrements)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, boolean offsetsAreCorrect)`
`static void`	`assertAnalyzesTo(Analyzer a, String input, String[] output, String[] types)`
`static void`	`assertAnalyzesToPositions(Analyzer a, String input, String[] output, int[] posIncrements, int[] posLengths)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output, int[] posIncrements)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void`	`assertAnalyzesToReuse(Analyzer a, String input, String[] output, String[] types)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] posIncrements)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements, int[] posLengths, Integer finalOffset)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, int[] posIncrements, Integer finalOffset)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, Integer finalOffset)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, Integer finalOffset)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, int[] posLengths, Integer finalOffset, boolean offsetsAreCorrect)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, int[] startOffsets, int[] endOffsets, String[] types, int[] posIncrements, Integer finalOffset)`
`static void`	`assertTokenStreamContents(TokenStream ts, String[] output, String[] types)`
`static void`	`checkAnalysisConsistency(Random random, Analyzer a, boolean useCharFilter, String text)`
`static void`	`checkAnalysisConsistency(Random random, Analyzer a, boolean useCharFilter, String text, boolean offsetsAreCorrect)`
`static void`	`checkOneTerm(Analyzer a, String input, String expected)`
`static void`	`checkOneTermReuse(Analyzer a, String input, String expected)`
`static void`	`checkRandomData(Random random, Analyzer a, int iterations)`	utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void`	`checkRandomData(Random random, Analyzer a, int iterations, boolean simple)`	utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void`	`checkRandomData(Random random, Analyzer a, int iterations, int maxWordLength)`	utility method for blasting tokenstreams with data to make sure they don't do anything crazy
`static void`	`checkRandomData(Random random, Analyzer a, int iterations, int maxWordLength, boolean simple)`
`static void`	`checkRandomData(Random random, Analyzer a, int iterations, int maxWordLength, boolean simple, boolean offsetsAreCorrect)`
`static String`	`escape(String s)`
`protected String`	`toDot(Analyzer a, String inputText)`
`protected void`	`toDotFile(Analyzer a, String inputText, String localFileName)`

Methods inherited from class org.apache.lucene.util.LuceneTestCase
afterClassLuceneTestCaseJ4, alwaysIgnoredTestMethod, assertEquals, assertEquals, assertEquals, assertEquals, assertSaneFieldCaches, asSet, assumeFalse, assumeNoException, assumeTrue, atLeast, atLeast, beforeClassLuceneTestCaseJ4, dumpArray, dumpIterator, getDataFile, getName, getTestClass, getTestLabel, isTestThread, localeForName, newDirectory, newDirectory, newDirectory, newDirectory, newField, newField, newField, newField, newField, newField, newFSDirectory, newFSDirectory, newIndexWriterConfig, newIndexWriterConfig, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newLogMergePolicy, newSearcher, newSearcher, newTieredMergePolicy, newTieredMergePolicy, purgeFieldCache, randomDirectory, randomLocale, randomTimeZone, rarely, rarely, reportAdditionalFailureInfo, reportPartialFailureInfo, setMergeFactor, setUp, setUseCompoundFile, tearDown, usually, usually

Methods inherited from class org.junit.Assert
assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertArrayEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertEquals, assertFalse, assertFalse, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotEquals, assertNotNull, assertNotNull, assertNotSame, assertNotSame, assertNull, assertNull, assertSame, assertSame, assertThat, assertThat, assertTrue, assertTrue, fail, fail

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail
- BaseTokenStreamTestCase
```
public BaseTokenStreamTestCase()
```

Method Detail

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             String[] types,
                                             int[] posIncrements,
                                             int[] posLengths,
                                             Integer finalOffset,
                                             boolean offsetsAreCorrect)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             String[] types,
                                             int[] posIncrements,
                                             int[] posLengths,
                                             Integer finalOffset)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             String[] types,
                                             int[] posIncrements,
                                             Integer finalOffset)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             String[] types,
                                             int[] posIncrements)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             String[] types)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] posIncrements)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             Integer finalOffset)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             int[] posIncrements)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             int[] posIncrements,
                                             Integer finalOffset)
                                      throws IOException

Throws:: IOException

assertTokenStreamContents

public static void assertTokenStreamContents(TokenStream ts,
                                             String[] output,
                                             int[] startOffsets,
                                             int[] endOffsets,
                                             int[] posIncrements,
                                             int[] posLengths,
                                             Integer finalOffset)
                                      throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] startOffsets,
                                    int[] endOffsets,
                                    String[] types,
                                    int[] posIncrements)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] startOffsets,
                                    int[] endOffsets,
                                    String[] types,
                                    int[] posIncrements,
                                    int[] posLengths)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] startOffsets,
                                    int[] endOffsets,
                                    String[] types,
                                    int[] posIncrements,
                                    int[] posLengths,
                                    boolean offsetsAreCorrect)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    String[] types)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] posIncrements)
                             throws IOException

Throws:: IOException

assertAnalyzesToPositions

public static void assertAnalyzesToPositions(Analyzer a,
                                             String input,
                                             String[] output,
                                             int[] posIncrements,
                                             int[] posLengths)
                                      throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] startOffsets,
                                    int[] endOffsets)
                             throws IOException

Throws:: IOException

assertAnalyzesTo

public static void assertAnalyzesTo(Analyzer a,
                                    String input,
                                    String[] output,
                                    int[] startOffsets,
                                    int[] endOffsets,
                                    int[] posIncrements)
                             throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output,
                                         int[] startOffsets,
                                         int[] endOffsets,
                                         String[] types,
                                         int[] posIncrements)
                                  throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output)
                                  throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output,
                                         String[] types)
                                  throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output,
                                         int[] posIncrements)
                                  throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output,
                                         int[] startOffsets,
                                         int[] endOffsets)
                                  throws IOException

Throws:: IOException

assertAnalyzesToReuse

public static void assertAnalyzesToReuse(Analyzer a,
                                         String input,
                                         String[] output,
                                         int[] startOffsets,
                                         int[] endOffsets,
                                         int[] posIncrements)
                                  throws IOException

Throws:: IOException

checkOneTerm

public static void checkOneTerm(Analyzer a,
                                String input,
                                String expected)
                         throws IOException

Throws:: IOException

checkOneTermReuse

public static void checkOneTermReuse(Analyzer a,
                                     String input,
                                     String expected)
                              throws IOException

Throws:: IOException

checkRandomData

public static void checkRandomData(Random random,
                                   Analyzer a,
                                   int iterations)
                            throws IOException

utility method for blasting tokenstreams with data to make sure they don't do anything crazy

Throws:: IOException

checkRandomData

public static void checkRandomData(Random random,
                                   Analyzer a,
                                   int iterations,
                                   int maxWordLength)
                            throws IOException

utility method for blasting tokenstreams with data to make sure they don't do anything crazy

Throws:: IOException

checkRandomData

public static void checkRandomData(Random random,
                                   Analyzer a,
                                   int iterations,
                                   boolean simple)
                            throws IOException

utility method for blasting tokenstreams with data to make sure they don't do anything crazy

Parameters:: simple - true if only ascii strings will be used (try to avoid)
Throws:: IOException

checkRandomData

public static void checkRandomData(Random random,
                                   Analyzer a,
                                   int iterations,
                                   int maxWordLength,
                                   boolean simple)
                            throws IOException

Throws:: IOException

checkRandomData

public static void checkRandomData(Random random,
                                   Analyzer a,
                                   int iterations,
                                   int maxWordLength,
                                   boolean simple,
                                   boolean offsetsAreCorrect)
                            throws IOException

Throws:: IOException

escape

public static String escape(String s)

checkAnalysisConsistency

public static void checkAnalysisConsistency(Random random,
                                            Analyzer a,
                                            boolean useCharFilter,
                                            String text)
                                     throws IOException

Throws:: IOException

checkAnalysisConsistency

public static void checkAnalysisConsistency(Random random,
                                            Analyzer a,
                                            boolean useCharFilter,
                                            String text,
                                            boolean offsetsAreCorrect)
                                     throws IOException

Throws:: IOException

toDot

protected String toDot(Analyzer a,
                       String inputText)
                throws IOException

Throws:: IOException

toDotFile

protected void toDotFile(Analyzer a,
                         String inputText,
                         String localFileName)
                  throws IOException

Throws:: IOException

Class BaseTokenStreamTestCase

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.util.LuceneTestCase

Field Summary

Fields inherited from class org.apache.lucene.util.LuceneTestCase

Constructor Summary

Method Summary

Methods inherited from class org.apache.lucene.util.LuceneTestCase

Methods inherited from class org.junit.Assert

Methods inherited from class java.lang.Object

Constructor Detail

BaseTokenStreamTestCase

Method Detail

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertTokenStreamContents

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesToPositions

assertAnalyzesTo

assertAnalyzesTo

assertAnalyzesToReuse

assertAnalyzesToReuse

assertAnalyzesToReuse

assertAnalyzesToReuse

assertAnalyzesToReuse

assertAnalyzesToReuse

checkOneTerm

checkOneTermReuse

checkRandomData

checkRandomData

checkRandomData

checkRandomData

checkRandomData

escape

checkAnalysisConsistency

checkAnalysisConsistency

toDot

toDotFile