Skip to content

*: support Local MATCH AGAINST boolean expr | pd=release-8.5 tikv=release-8.5 tidb=release-8.5 ticdc=release-8.5 #11111

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
solotzg wants to merge 32 commits into
base: feature/release-8.5-fts
Choose a base branch
from
+3,493 −14
Open

*: support Local MATCH AGAINST boolean expr | pd=release-8.5 tikv=release-8.5 tidb=release-8.5 ticdc=release-8.5 #11111

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
4992c92
fts: support MATCH AGAINST boolean pushdown on release-8.5
solotzg Sep 21, 2026
55c2b68
fts: support nullable MATCH columns in TiFlash
solotzg Sep 22, 2026
b6f1477
docs: update TiFlash MATCH AGAINST design
solotzg Sep 22, 2026
6405b45
docs: refresh MATCH AGAINST E2E status
solotzg Sep 22, 2026
0ca2f3f
fts: support NGRAM boolean pushdown
solotzg Sep 23, 2026
2d7d7c8
test: cover FTS collation matrix on TiFlash
solotzg Sep 23, 2026
a3fca18
build: include direct RaftLogManager dependencies
solotzg Sep 23, 2026
2145aac
fts: fix TiFlash MATCH evaluation edge cases
solotzg Sep 23, 2026
2913916
fts: align NGRAM prefix evaluation
solotzg Sep 28, 2026
50aea27
fts: forward STANDARD_V1 analyzer settings
solotzg Sep 28, 2026
ea7f96c
build: sync tipb default-value comments
solotzg Sep 28, 2026
f8a84f4
fts: read Boolean query from structured Tipb expression
solotzg Sep 30, 2026
3a504cb
fts: decode versioned metadata from Expr.val
solotzg Sep 30, 2026
22bdbe4
fts: remove TiCI scan-level MATCH path
solotzg Oct 8, 2026
7f0cac2
tiflash: execute Local MATCH through scalar evaluation
solotzg Oct 8, 2026
f7d3ee2
tiflash: short-circuit Local MATCH evaluation
solotzg Oct 9, 2026
6281c56
tiflash: sync Local MATCH signature ID 5250
solotzg Oct 9, 2026
cc51474
tiflash: align Local MATCH split-word and Unicode semantics
solotzg Oct 9, 2026
30160fe
tiflash: verify Local MATCH Unicode classification fingerprint
solotzg Oct 9, 2026
184f580
fix: align Local MATCH semantics and cache phrase verification
solotzg Oct 10, 2026
febfdb4
perf: reduce Local MATCH analyzer work and token copies
solotzg Oct 10, 2026
3028a5c
perf: slide Local MATCH ngram stopword checks
solotzg Oct 10, 2026
f82ab25
refactor: simplify Local MATCH cold-path setup
solotzg Oct 10, 2026
9bbf1a7
test: remove Local MATCH microbenchmark target
solotzg Oct 12, 2026
b0ea7e6
refactor: remove legacy Local MATCH scan integration
solotzg Oct 12, 2026
96cc7fb
test: cover Local MATCH DAG and block correctness
solotzg Oct 12, 2026
3e73ec0
format
solotzg Oct 12, 2026
cd0b66d
fix: resolve Local MATCH clang-tidy errors
solotzg Oct 13, 2026
ade972a
fix: accept untyped NULL Local MATCH searches
solotzg Oct 13, 2026
c55c74c
fix: update image references to require environment variables for Loc…
solotzg Oct 13, 2026
7c71fbe
deps: use official Local MATCH tipb protocol
solotzg Oct 14, 2026
71904a2
fix: reject invalid Local MATCH Boolean node metadata
solotzg Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion contrib/tipb
Open in desktop
Submodule tipb updated 11 files
+3 −2 .github/workflows/unit-test.yml
+7 −0 go-tipb/analyze.pb.go
+1,248 −423 go-tipb/executor.pb.go
+1,164 −371 go-tipb/expression.pb.go
+617 −79 go-tipb/select.pb.go
+124 −51 go-tipb/topsql_agent.pb.go
+24 −0 proto/executor.proto
+63 −0 proto/expression.proto
+40 −1 proto/select.proto
+2 −0 proto/topsql_agent.proto
+8,671 −0 scripts/proto.lock
9 changes: 8 additions & 1 deletion dbms/src/Flash/Coprocessor/DAGExpressionAnalyzerHelper.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ String DAGExpressionAnalyzerHelper::buildSingleParamJsonRelatedFunctions(
function_json_unquote)
{
bool valid_check
= !(isScalarFunctionExpr(input_expr) && input_expr.sig() == tipb::ScalarFuncSig::CastJsonAsString);
= !isScalarFunctionExpr(input_expr) || input_expr.sig() != tipb::ScalarFuncSig::CastJsonAsString;

@AilinKid AilinKid Oct 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same meaning, unnecessary?

@solotzg solotzg Oct 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the expressions are logically equivalent. This rewrite was made in commit cd0b66d to address a readability-simplify-boolean-expr diagnostic from clang-tidy during CI. The repository treats clang-tidy warnings as errors. It does not change JSON behavior; we retained it to satisfy the lint check.

function_json_unquote->setNeedValidCheck(valid_check);
}
else if (auto * function_cast_json_as_string = dynamic_cast<FunctionCastJsonAsString *>(function_impl);
Expand Down Expand Up @@ -520,6 +520,13 @@ String DAGExpressionAnalyzerHelper::buildDefaultFunction(
String name = analyzer->getActions(child, actions);
argument_names.push_back(name);
}
if (expr.sig() == tipb::ScalarFuncSig::LocalMatchAgainstBoolean)
{
// Expr.val carries Local MATCH metadata, not an SQL argument. Adapt it
// to the runtime function's required final constant argument.
const auto metadata_expr = constructStringLiteralTiExpr(expr.val());
argument_names.push_back(analyzer->getActions(metadata_expr, actions));
}
return analyzer->applyFunction(func_name, argument_names, actions, getCollatorFromExpr(expr));
}

Expand Down
1 change: 1 addition & 0 deletions dbms/src/Flash/Coprocessor/DAGUtils.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -455,6 +455,7 @@ const std::unordered_map<tipb::ScalarFuncSig, String> scalar_func_map({
{tipb::ScalarFuncSig::RegexpInStrSig, "regexp_instr"},
{tipb::ScalarFuncSig::RegexpReplaceSig, "regexp_replace"},
{tipb::ScalarFuncSig::RegexpSubstrSig, "regexp_substr"},
{tipb::ScalarFuncSig::LocalMatchAgainstBoolean, "local_match_against_boolean"},

{tipb::ScalarFuncSig::JsonExtractSig, "json_extract"},
{tipb::ScalarFuncSig::JsonUnquoteSig, "json_unquote"},
Expand Down
Loading