How TalkToMySites answers from your pages, and shows its sources
A step-by-step look at crawling, section-aware chunking, two-stage retrieval and streaming answers, all running on Cloudflare.
An AI assistant is only useful if people believe it. The fastest way to lose that trust is an answer that sounds confident and turns out to be invented. So TalkToMySites is built around one rule: answer from your pages, and show which ones.
Here's what happens between a visitor typing a question and the answer appearing.
1. Crawling
When you add a site, we fetch your sitemap. Sitemap indexes and gzipped sitemaps both work. Every URL goes into a queue, and a background job works through it every minute.
For each page we pull out the main content and drop the parts that repeat on every page: navigation, headers, footers, cookie banners. If we kept those, every search would match your menu.
2. Chunking by section
A whole page is too big to search well, and a fixed-size slice cuts paragraphs in half. We split each page along its headings instead, into chunks of roughly 350 tokens, and keep each heading attached to the text under it. The page title and description go along with every chunk, so a section about "Billing" still knows it belongs to your pricing page.
3. Embeddings and a search index per site
Each chunk is turned into an embedding with bge-small-en-v1.5 on Workers AI and stored in Cloudflare Vectorize. Every site gets its own namespace, so one customer's search can never return another customer's pages.
4. Two-stage retrieval
When a question comes in, we embed it and pull the 20 closest chunks from your index. Vector search is fast but a little blunt, so a second model, bge-reranker-base, reads the question against each candidate and re-orders them by how well they actually answer it.
We keep the top four, with no more than two from any single page. That keeps the sources varied: if three pages each cover part of the answer, you'll see all three.
5. Writing the answer
The chosen sections go to llama-3.1-8b-instruct along with the question. The instructions are short and strict: answer only from these excerpts, say so when they don't cover the question, and reply in the language the visitor used. The answer streams back to the widget as it's written, and the source pages show up underneath.
6. Guard rails
A widget on a public page is an open door, so a few things sit in front of the model:
- Each widget only accepts requests from the domain you registered.
- Each visitor is limited to 20 questions a minute, and each site to 300.
- Monthly question limits are tracked per organization, so you always know where you stand.
Why Cloudflare
Every step above runs on Cloudflare: Workers for the API and crawler, Workers AI for the three models, Vectorize for search, D1 for everything else. There's no separate vector database, no GPU server, and no OpenAI key. That's what lets us offer a free plan, and it's why answers start streaming quickly wherever your visitors are.
