Tokenizer can take a text string and split it in tokens separated by text strings or one or more characters.
Tokenizer supports marker strings that define the beginning and end of a token that may have separator strings inside.
Operators which can be used to combine multiple tokens are also supported.
PHP 5.0 or higher
More popular Text Processing
- 10.0 KB
- 02/13/2010 12:27:24