Runs in your browser — no install. (Works once the repo is public.)

In [ ]:

# --- Colab setup (skipped when running locally) ---
import os, sys
if 'google.colab' in sys.modules and not os.path.exists('data/sample_corpus.json'):
    !git clone -q https://github.com/mdhabibi/llm-search-handbook.git
    %cd llm-search-handbook
    !pip -q install -r requirements.txt

# --- Colab setup (skipped when running locally) --- import os, sys if 'google.colab' in sys.modules and not os.path.exists('data/sample_corpus.json'): !git clone -q https://github.com/mdhabibi/llm-search-handbook.git %cd llm-search-handbook !pip -q install -r requirements.txt

Chapter 12 — Advanced Topics¶

A menu of techniques beyond the core pipeline. The MaxSim (ColBERT) and query-expansion demos run offline; HyDE and CLIP plug into a model on your machine.

In [ ]:

# Bootstrap: locate repo root and import shared helpers
import sys, os, json
d = os.getcwd()
while d != os.path.dirname(d) and not os.path.exists(os.path.join(d, 'data', 'sample_corpus.json')):
    d = os.path.dirname(d)
ROOT = d; sys.path.insert(0, os.path.join(ROOT, 'src'))
import numpy as np
from corpus import load_corpus, tokenize

# Bootstrap: locate repo root and import shared helpers import sys, os, json d = os.getcwd() while d != os.path.dirname(d) and not os.path.exists(os.path.join(d, 'data', 'sample_corpus.json')): d = os.path.dirname(d) ROOT = d; sys.path.insert(0, os.path.join(ROOT, 'src')) import numpy as np from corpus import load_corpus, tokenize

1. Query expansion¶

Add related terms so retrieval matches more surface forms.

In [ ]:

SYNONYMS = {'headache': ['migraine', 'temple', 'head pain'],
            'ocean': ['sea', 'marine'], 'car': ['automobile', 'vehicle']}
def expand(query):
    extra = []
    for w in query.lower().split():
        extra += SYNONYMS.get(w, [])
    return query + (' ' + ' '.join(extra) if extra else '')

print(expand('bad headache'))
print(expand('a large ocean animal'))

SYNONYMS = {'headache': ['migraine', 'temple', 'head pain'], 'ocean': ['sea', 'marine'], 'car': ['automobile', 'vehicle']} def expand(query): extra = [] for w in query.lower().split(): extra += SYNONYMS.get(w, []) return query + (' ' + ' '.join(extra) if extra else '') print(expand('bad headache')) print(expand('a large ocean animal'))

2. HyDE — retrieve with a hypothetical answer¶

Ask an LLM to draft an answer, embed that, and use it to retrieve. Here a mock generator stands in so the flow runs anywhere.

In [ ]:

def mock_generate(prompt):
    # a real LLM writes a passage-like hypothetical answer here
    return 'The blue whale is the largest animal ever, feeding on krill in the ocean.'

def hyde_query(question, generate):
    hypothetical = generate(f'Write a short factual passage answering: {question}')
    print('hypothetical answer used for retrieval ->', hypothetical)
    return hypothetical  # embed THIS instead of the short question

_ = hyde_query('what is the biggest animal?', mock_generate)
print('\n(Embed the hypothetical answer; it often sits closer to real answer passages.)')

def mock_generate(prompt): # a real LLM writes a passage-like hypothetical answer here return 'The blue whale is the largest animal ever, feeding on krill in the ocean.' def hyde_query(question, generate): hypothetical = generate(f'Write a short factual passage answering: {question}') print('hypothetical answer used for retrieval ->', hypothetical) return hypothetical # embed THIS instead of the short question _ = hyde_query('what is the biggest animal?', mock_generate) print('\n(Embed the hypothetical answer; it often sits closer to real answer passages.)')

3. ColBERT-style late interaction (MaxSim) from scratch¶

Represent each text as one vector per token. Score with MaxSim: for each query token, take its best-matching document token, then sum. Keeps token detail a single vector loses.

In [ ]:

rng = np.random.default_rng(0)
def unit(v): return v/np.linalg.norm(v)

# Toy token embeddings (in reality these come from a ColBERT model)
vocab = {w: unit(rng.standard_normal(8)) for w in
         ['whale','ocean','krill','capital','city','canada','the','a']}
def embed_tokens(text): return np.array([vocab[w] for w in text.split() if w in vocab])

def maxsim(q_text, d_text):
    Q, D = embed_tokens(q_text), embed_tokens(d_text)
    sims = Q @ D.T                 # (|q| tokens) x (|d| tokens) cosine matrix
    return float(sims.max(axis=1).sum())   # best doc-token per query-token, summed

query = 'whale ocean'
docA = 'the whale ocean krill'      # on-topic
docB = 'the capital city canada'    # off-topic
print(f'MaxSim(query, on-topic doc)  = {maxsim(query, docA):.3f}')
print(f'MaxSim(query, off-topic doc) = {maxsim(query, docB):.3f}')
assert maxsim(query, docA) > maxsim(query, docB)
print('Late interaction rewards fine-grained token matches. ✓')

rng = np.random.default_rng(0) def unit(v): return v/np.linalg.norm(v) # Toy token embeddings (in reality these come from a ColBERT model) vocab = {w: unit(rng.standard_normal(8)) for w in ['whale','ocean','krill','capital','city','canada','the','a']} def embed_tokens(text): return np.array([vocab[w] for w in text.split() if w in vocab]) def maxsim(q_text, d_text): Q, D = embed_tokens(q_text), embed_tokens(d_text) sims = Q @ D.T # (|q| tokens) x (|d| tokens) cosine matrix return float(sims.max(axis=1).sum()) # best doc-token per query-token, summed query = 'whale ocean' docA = 'the whale ocean krill' # on-topic docB = 'the capital city canada' # off-topic print(f'MaxSim(query, on-topic doc) = {maxsim(query, docA):.3f}') print(f'MaxSim(query, off-topic doc) = {maxsim(query, docB):.3f}') assert maxsim(query, docA) > maxsim(query, docB) print('Late interaction rewards fine-grained token matches. ✓')

4. Multimodal retrieval (sketch)¶

Models like CLIP embed images and text into one shared space, so a text query retrieves images with the same nearest-neighbor search from Chapter 5.

In [ ]:

print('Pseudocode (plug in CLIP on your machine):')
print('  img_vecs  = clip.encode_image(images)      # shared space')
print('  q_vec     = clip.encode_text("a photo of a dog")')
print('  results   = nearest(q_vec, img_vecs)        # same machinery as Ch5')

print('Pseudocode (plug in CLIP on your machine):') print(' img_vecs = clip.encode_image(images) # shared space') print(' q_vec = clip.encode_text("a photo of a dog")') print(' results = nearest(q_vec, img_vecs) # same machinery as Ch5')

Takeaway & exercises¶

These techniques earn their place only by improving Chapter 9 metrics on your data. Add complexity deliberately.

Exercises

1. Wire real HyDE: use the Chapter 10 generate() + Chapter 5 retriever. Does it beat plain retrieval?
2. Extend MaxSim to rank several documents; compare its order to single-vector cosine.
3. Try a multilingual model: query in English, retrieve a passage you added in another language.
4. Sketch an agentic multi-hop flow for: 'Which is older, the maker of X or of Y?'