Choosing a model and controlling cost

You pay your AI provider per token (roughly per word) sent and received. A few choices keep that bill small and predictable.

Pick a small model first#

Product search is a short, structured task and the small models handle it well. GPT-4o Mini (OpenAI or OpenRouter) and Claude Haiku 5.5 (Anthropic or OpenRouter) are the recommended starting points and the cheapest in their lists. Move to GPT-4o, Claude Sonnet or Claude Opus only if you want longer, more nuanced answers to policy questions.

What the plugin does to limit spend#

• Message length cap. Each shopper message is limited to 1,000 characters; longer messages are rejected before anything is sent to the provider.
• Short memory. Only the shopper’s current message and up to five earlier messages from the same conversation are sent, not the whole history.
• Rate limit. The per-IP limit under Connections stops one visitor (or script) from sending more than the configured number of messages per minute.
• Reply length. Replies are capped at the provider level (1,024 tokens for OpenAI and OpenRouter, 4,096 for Anthropic with low reasoning effort), so a single answer cannot run away.
• Search caching. Product search results are cached for five minutes, so repeated searches do not hit the database again (the AI call still happens for each message).

Watch the numbers#

The Dashboard shows Tokens Used Today and, when the model is GPT-4o Mini, an estimated API cost for today. For every other model, check the usage page of your provider: OpenAI’s Usage dashboard, OpenRouter’s Activity page or the Anthropic Console.