Runs in your browser — no install. (Works once the repo is public.)
# --- Colab setup (skipped when running locally) ---
import os, sys
if 'google.colab' in sys.modules and not os.path.exists('data/sample_corpus.json'):
!git clone -q https://github.com/mdhabibi/llm-search-handbook.git
%cd llm-search-handbook
!pip -q install -r requirements.txt
Chapter 1 — Words vs. Meaning¶
A five-minute, hands-on feel for the single most important idea in this course: counting shared words is not the same as matching meaning.
We'll (1) build a naïve word-overlap 'search', (2) watch it work on an easy query, and (3) watch it break on a query where the answer is phrased differently. That failure motivates everything from Chapter 3 onward.
# Bootstrap: find the repo root (folder containing data/sample_corpus.json) and import helpers
import sys, os
d = os.getcwd()
while d != os.path.dirname(d) and not os.path.exists(os.path.join(d, 'data', 'sample_corpus.json')):
d = os.path.dirname(d)
ROOT = d
sys.path.insert(0, os.path.join(ROOT, 'src'))
from corpus import load_corpus, tokenize
docs = load_corpus()
print(f'Loaded {len(docs)} documents from', os.path.join(ROOT, 'data', 'sample_corpus.json'))
1. A tiny toy archive¶
First, the clean textbook example. Five short 'documents', and a query. We score each document by how many distinct words it shares with the query.
toy = [
'the grass is green',
'the sky is blue',
'the capital of canada is ottawa',
'a whale is a mammal',
'tomorrow is saturday',
]
def word_overlap(query, corpus):
q = set(tokenize(query))
scored = [(i, len(q & set(tokenize(doc)))) for i, doc in enumerate(corpus)]
return sorted(scored, key=lambda x: -x[1])
query = 'what color is the grass'
for i, s in word_overlap(query, toy):
print(f'score={s} D{i}: {toy[i]}')
D0 wins — it literally repeats grass, is, the. Keyword matching nails the easy case. So far so good.
2. The same idea on our real corpus — and a surprise¶
Watch what happens when documents don't conveniently repeat the query's words. Run overlap for the grass question on the real 16-document corpus.
texts = [d['text'] for d in docs]
print('Query: what color is the grass')
for i, s in word_overlap('what color is the grass', texts)[:4]:
print(f' score={s} D{i}: {docs[i]["title"]}')
Notice the ranking is now wrong: passages about whales or capitals can outrank The color of grass, because they happen to share filler words like the and is while the grass passage phrases things differently.
Two lessons already: (a) common words pollute word-overlap — we'll fix that with IDF in Chapter 2; (b) even after that, matching words is fragile.
3. The failure that defines the course¶
Now a query whose perfect answer uses completely different words. Our corpus has a migraine passage: 'a sharp, throbbing sensation localized to one temple'. Ask about it in plain language:
query = 'strong pain in the side of the head'
print('Query:', query)
ranking = word_overlap(query, texts)
for i, s in ranking[:5]:
print(f' score={s} D{i}: {docs[i]["title"]}')
migraine_id = next(d['id'] for d in docs if d['title'] == 'Migraine symptoms')
rank_pos = [i for i,_ in ranking].index(migraine_id)
print(f'\nThe truly relevant migraine passage (D{migraine_id}) lands at rank {rank_pos+1} of {len(docs)}.')
The migraine passage — the actual answer — is buried, because it shares almost no literal words with the query. 'Temple' means 'side of the head'; 'sensation' means 'pain'. A word-counter is blind to synonyms.
This is the vocabulary-mismatch problem. Solving it — teaching machines to match meaning — is what Chapters 3–6 are about.
Exercises¶
- Re-run word-overlap with stop words removed (
tokenize(text, remove_stopwords=True)). Does the grass ranking improve? Why? - Find another query on our corpus where overlap fails. What synonyms trip it up?
- Predict which chapter's technique would rescue each failing query.