Commit 687f1165 authored by Bjorn Bringert's avatar Bjorn Bringert
Browse files

Add token_index column to _TOKENIZE table.

If the new optional fifth argument to _TOKENIZE is
non-zero, the _TOKENIZE function now uses a third column,
token_index, in addition to the earlier token and source
columns. The new column stores the index of the token within
the string. This is useful for distingusihing between
matches that are prefixes of the original string,
and matches that are prefixes of one of the tokens
inside the original string.

This change is required to fix http://b/issue?id=1847321
parent 2da78c08
Loading
Loading
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment