Langlab 1.3.0 API documentation

Convenience function alias for lg-split-sentences-bi for English.

Convenience function alias for lg-split-sentences-icu-bi for English.

Convenience function alias for lg-split-tokens-bi for English.

Convenience function alias for lg-split-tokens-icu-bi for English.

Split s into seq of sentences using standard BreakIterator class. Sets language to lang.

Note. It is not clear to me how the locale is used by BreakIterator.

Split s into seq of sentences using ICU BreakIterator class. Sets language to lang.

Note. It is not clear to me how the locale is used by BreakIterator.

Split s into seq of words using standard BreakIterator class. Sets language to lang.

Note. It is not clear to me how the locale is used by BreakIterator.

Split s into seq of words using ICU BreakIterator class. Sets language to lang.

Note. It is not clear to me how the locale is used by BreakIterator.

Creates Open NLP sentence splitter using model from model-fname.

Creates Open NLP token splitter using model from file model-fname.

Splits s on regexp re, but as opposed to string/split keeps the regexp matches in a resulting seq.

Similar effect can be achieved by using look-arounds, but its clumsy, see

http://stackoverflow.com/questions/19951850/split-string-with-regex-but-keep-delimeters-in-match-array

Splits s on whitespace and removes punctuation. Splitter based on Simple Analyzer from Lucene.

Splits s into tokens on whitespace (using regexp \s+). Inverse of merge-tokens-with-space.