Lessons from building an in-house regulation search tool with a small local LLM

A technical retrospective documents the limitations and unresolved challenges of building an in-house regulation search tool using a small local language model. Following five instalments that focused on initial development successes, the author outlines which approaches failed in practice. The account highlights ongoing technical hurdles encountered when implementing local models for specialised internal search.

The findings show that initial development milestones for local search tools do not guarantee that all edge cases will be resolved. Teams building on-premises compliance tools may encounter persistent model constraints that require alternative design approaches.

  • A six-part development retrospective concluded by detailing failed approaches and unresolved problems in an in-house search project.
  • The project evaluated the feasibility of deploying a small local language model to navigate internal regulations.
  • The author noted that earlier stages achieved success, but critical functional gaps remained unsolved at the project's conclusion.
AI Apps & Platforms Generative AI Sovereign AI
All AI news

More AI news

Models

Tether AI Research releases QVAC BotPsy-0.8B for on-device tool use

Tether AI Research announced the release of QVAC BotPsy-0.8B on 11 November 2026. The release is an open-weight, 0.8-billion-parameter model built to support tool-using artificial intelligence on personal devices.

Industry

Anthropic introduces Claude workplace tools to compete in enterprise productivity

Anthropic has announced an expansion of its AI platform into workplace productivity software with the launch of Claude Sheets, Whiteboards, Meetings, and Data. The announcement prompted an immediate market reaction, with Microsoft shares falling by more than 2 per cent.

Research

Google releases Gemini 4 Carbon for general availability

According to reporting on note.com, Google has moved its Gemini 4 Carbon artificial intelligence model into general availability. The change comes roughly two weeks after a 30 October report stated that the system was not yet accessible for general use.

Build it with us, or have us check yours

In 15 minutes we talk through what you want to build, who builds it and what it needs to go live.

Talk directly with our AI specialists

15 minutes, no obligation
No sales pressure
Prototype in 7 days