For two ASCII characters, rows 0 and 1 of the score matrix collapse to
scalar running state, so Phase 2 and Phase 3 fuse into one pass with no
score arrays. withPos stores the two rows for the backtrace. Up to 1.4x
on two-char queries, the most common multi-char length.
Verify both fast paths against the general algorithm with exhaustive
(every short string over a class-complete alphabet) and fuzz tests,
runnable via the new make fuzz target.
Phase 3 writes C rows only from column F[r], but tie-breaking in the
backtrace could read row i+1 at column j+1 left of F[i+1]. With reused
slab, that cell holds data from previously processed item, so highlight
positions in equal-score ties depended on processing order.
A non-word character (e.g. '.') used to receive a flat bonusNonWord
regardless of context. Now it gets bonusBoundaryWhite at the start of
input and bonusBoundaryDelimiter right after a delimiter, matching the
treatment of word characters at the same boundaries.
Without this, '.completion' matching '.completion' lost to
'bash_completion.d/completions/X' because the consecutive chunk anchor
in the long path (the 'c' after '/') received bonusBoundaryDelimiter
while the exact match's '.' was capped at bonusNonWord.
Fix#4795
There is an edge-case in FuzzyMatchV1 during backward scan, related to
normalization: if string is initially denormalized (e.g. Unicode symbol),
backward scan will proceed further to the next char; however, when the
score is computed, the string is normalized first, then scanned based on
the pattern. This leads to accessing pattern index increment, which
itself leads to out-of-bound index access, resulting in a panic.
To illustrate the process, here's the sequence of operations when search
is perfored:
1. during backward scan by "minim" pattern
```
xxxxx Minímal example
^^^^^^^^^^^^
||||||||||||
miniiiiiiiim <- compute score for this substring
```
2. during compute score by "minim" pattern
```
Minímal exam
minimal exam <- normalize chars before computing the score
^^^^^^
||||||
minim <- at this point the pattern is already fully scanned and index
is out-of-the-bound
```
In this commit the char is normalized during backward scan, to detect
properly the boundaries for the pattern.
- Prefix matcher will trim leading whitespaces only when the pattern
doesn't start with a whitespace
- Suffix matcher will trim trailing whitespaces only when the pattern
doesn't end with a whitespace
- Equal matcher will trim leading whitespaces only when the pattern
doesn't start with a whitespace, and trim trailing whitespaces only
when the pattern doesn't end with a whitespace
Previously, only suffix matcher would trim whitespaces unconditionally.
Fix#1894
- Make structs smaller
- Introduce Result struct and use it to represent matched items instead of
reusing Item struct for that purpose
- Avoid unnecessary memory allocation
- Avoid growing slice from the initial capacity
- Code cleanup
In the best case (all ascii), this reduces the memory footprint by 60%
and the response time by 15% to 20%. In the worst case (every line has
non-ascii characters), 3 to 4% overhead is observed.
Based on the patch by Matt Westcott (@mjwestcott).
But with a more conservative approach:
- Does not use linearly increasing penalties; It is agreed upon that we
should prefer matching characters at the beginnings of the words, but
it's not always clear that the relevance is inversely proportional to
the distance from the beginning.
- The approach here is more conservative in that the bonus is never
large enough to override the matchlen, so it can be thought of as the
first implicit tiebreak criterion.
- One may argue the change breaks the contract of --tiebreak, but the
judgement depends on the definition of "tie".
I profiled fzf and it turned out that it was spending significant amount
of time repeatedly converting character arrays into Unicode codepoints.
This commit greatly improves search performance after the initial scan
by memoizing the converted results.
This commit also addresses the problem of unbounded memory usage of fzf.
fzf is a short-lived process that usually processes small input, so it
was implemented to cache the intermediate results very aggressively with
no notion of cache expiration/eviction. I still think a proper
implementation of caching scheme is definitely an overkill. Instead this
commit introduces limits to the maximum size (or minimum selectivity) of
the intermediate results that can be cached.
Junegunn Choi