<tokenize>
Purpose/Function
If the whitespace attribute is true
, calls the org.apache.common.lang.StringUtils.split
method; else it calls a utility method which cycles through the chain (list) of delimiters until a delimiter is not found. If paired is true, it processes the delimiters in pairs, treating odd tokens as "start" tags and even ones as "end" tags.
Attributes
NAME |
TYPE |
DEFAULT VALUE |
DESCRIPTION |
---|---|---|---|
string |
|
required |
string to be tokenized |
whitespace |
|
false |
tokenize on all whitespace characters |
paired |
|
false |
process delimiters in pairs, returning token between them |
includeDelimiter |
|
false |
include the delimiter with the token |
asList |
|
true |
if |
chain |
|
|
the chain of delimiters |
Elements
TAG |
TYPE |
COUNT |
DESCRIPTION |
---|---|---|---|
|
|
0:N (only if whitespace is |
add a delimiter to the chain |
In the case of a single delimiter, this can also be set as an attribute (cf. 5. Interchanging Attributes and Tags).
ReturnValues
DEFAULT NAME |
TYPE |
DESCRIPTION |
---|---|---|
tokens |
either |
the tokenized string |
Examples
<ogrescript> <declare name="s0" string="abcdefghijklmnop qrstuvwxyzab,cdefghijklm nopqrstuvwxyzlmnopqrstuvwxyz"/> <declare name="tokens"/> <tokenize string="${s0}" whitespace="true"> <return-value defaultName="tokens"/> </tokenize> <echo message="${tokens}"/> <tokenize string="${s0}" delimiter="," asList="false"> <return-value defaultName="tokens"/> </tokenize> <echo message="${tokens}"/> <tokenize string="${s0}" includeDelimiter="true"> <delimiter>ghijk</delimiter> <delimiter> </delimiter> <delimiter>vw</delimiter> <return-value defaultName="tokens"/> </tokenize> <echo message="${tokens}"/> </ogrescript>
Paired tokens:
<ogrescript> <declare name="s0" string="[OgreScript:abcdefghijklmnop]NOTINCLUDED[OgreScript:qrstuvwxyzab]NOTINCLUDED[OgreScript:cdefghijklm]NOTINCLUDED[OgreScript:nopqrstuvwxyzlmnopqrstuvwxyz]"/> <declare name="tokens"/> <tokenize string="${s0}" paired="true" includeDelimiter="true"> <delimiter>[OgreScript:</delimiter> <delimiter>]</delimiter> <return-value defaultName="tokens"/> </tokenize> <echo message="${tokens}"/> </ogrescript>