SAVETOKEN()
Save the global token environment
- Syntax
-
- SAVETOKEN () -> cStaticTokenEnvironment
- Returns
-
- <cStaticTokenEnvironment> a binary string encoding the global TE
- Description
-
- The SAVETOKEN() function can be used to store the global TE for future use or when two or more incremental tokenizers must the nested. Note however that the latter can now be solved with locally stored token environments.
- Status
-
- Ready
- Compliance
-
- SAVETOKEN() is compatible with CTIII's SAVETOKEN(),
- Platforms
-
- All
- Files
-
- Source is token2.c, library is libct.
- See Also