Shotnest · Library
Recognize equal word sets in a different order
“red blue” and “blue red” reach a token overlap of 1 in the Shotnest compare tool, yet they are not equal even after normalization.
Order is outside the measure
Token overlap is the number of shared distinct tokens divided by the number of distinct tokens in both texts together. Position plays no part. “red blue” and “blue red” produce the same set, so the overlap is 2 ÷ 2 = 1. Normalized equality compares the whole normalized strings, so it still reports No: the words are in a different order.
That gap between the two lines is the useful signal. Full overlap with unequal normalized text means the same vocabulary has been rearranged, which can be harmless in a list and decisive in a sentence.
Normalization does not reorder anything: it lowercases, folds accents and collapses spaces, then compares the strings as they stand.
Try the original example
- First text
- red blue
- Second text
- blue red
- Exactly equal
- No
- Equal after normalization
- No
- Token overlap
- 1
Read the three results together
- Exactly equal: No.
- Equal after normalization: No, because the order differs.
- Token overlap: 1, with empty first-only and second-only lists.
- Read both texts aloud in order; if the meaning changes, treat them as different records.
- Shared tokens
- 2
- Distinct tokens in total
- 2
- Token overlap
- 1
When word order matters
“Gate 12 seat 5” and “seat 12 gate 5” share every token and swap two numbers between fields. “Not before Friday” and “before Friday, not” mean different things. In the Shotnest app, search also ignores order: every typed word must appear in the screenshot text, in any position. That helps you find a screenshot, and it is one more reason to read the result before acting on it.
- Calling two screenshots duplicates because their text overlap is 1.
- Skipping the normalized-equality line, which is the only one that notices order.
- Assuming a rearranged list is identical when its items are tied to positions, such as seats or ranks.
The swapped-field example gives exactly that result: “Gate 12 seat 5” and “seat 12 gate 5” are not equal after normalization, and their overlap is 1.
No verdict on images
The compare tool linked below reports that its result is text overlap only, not proof that two images or documents are identical. Shotnest’s own duplicate check needs matching image dimensions and visual fingerprints in addition to shared words.