LocalAI

Provider Guide

LocalAI

LocalAI is a free, open-source, self-hosted drop-in replacement for the OpenAI API that runs entirely on your own hardware — no GPU required. It supports a wide range of model families and backends (llama.cpp/GGUF, whisper.cpp, stable diffusion, and more) behind one OpenAI-compatible server.

Because LocalAI speaks the OpenAI API standard, Hrida.ai connects to it the same way it connects to any OpenAI-compatible provider — just point it at your LocalAI server's URL.

localai.io →OpenAI-Compatible Guide →

1 Install & Run LocalAI

The fastest way to get started is with Docker:

bash

docker run -p 8080:8080 --name local-ai -ti localai/localai:latest-cpu

If you have an NVIDIA GPU, use one of the CUDA-tagged images instead for hardware acceleration. See the LocalAI installation guide for GPU images, binaries, and Kubernetes/Helm options.

Model Gallery

LocalAI ships with a built-in model gallery — once running, you can browse and one-click install models (LLMs, embeddings, TTS, image generation) directly from its web UI at http://localhost:8080, without hand-editing config files.

2 Add the Connection in Hrida.ai

1. 1Open Hrida.ai in your browser.
2. 2Go to ⚙️ Admin Settings → Connections → OpenAI.
3. 3Click ➕ Add Connection.
4. 4Enter the following:

Setting	Value
URL	http://localhost:8080/v1
API Key	Leave blank (unless you've configured one)

💡 5. Click Save.

Running Hrida.ai in Docker too? Use http://host.docker.internal:8080/v1 instead of localhost so the container can reach LocalAI on the host.

3 Start Chatting!

That's it! Hrida.ai fetches the model list from your LocalAI server's /v1/models endpoint automatically. Select any installed model from the dropdown and start chatting.

Chat completions and streaming work out of the box. Support for other features (tool calling, embeddings, vision) depends on the specific backend/model you've loaded in LocalAI.

Setting Up Industry-Specific Models

Rather than exposing raw model filenames, LocalAI lets you register each model under its own model config — giving it a custom name, system prompt, and default parameters tuned to a specific job. That name is what shows up in Hrida.ai's model dropdown, so end users pick a task, not a checkpoint. A typical setup for a business deployment groups models by use case:

Model Name	Use Case
Agents-A1	General-purpose agent model — tool calling and multi-step task execution across workflows.
IndustryBased	Deep research on domain/industry-specific questions — longer context, tuned for analysis over breadth.
Daily	Day-to-day reasoning — notes, meeting summaries, and email drafting.

How This Works

Each entry above is a separate .yaml file in LocalAI's models/ directory (or defined via the model gallery UI), pointing at whichever backend model/weights you want — they don't need to be different checkpoints, just different configs. Once saved, LocalAI lists each name at /v1/models, and Hrida.ai's model dropdown shows them exactly as named. See LocalAI's documentation for the full YAML reference (prompt templates, context size, function-calling settings, and more).

ℹ️The names above are examples — adapt them to whatever taxonomy fits your organization (by department, by task, by data sensitivity level, etc.). Hrida.ai doesn't care about the naming scheme; it just lists whatever LocalAI reports.

You're connected to LocalAI!

Fully local, fully private inference — no API keys, no cloud dependency. If you run into connection issues, see the connection troubleshooting guide.

Learn More

OpenAI-Compatible Providers

Protocol design philosophy and setup for other providers.

→

LocalAI Documentation

Full installation options, GPU backends, and the model gallery.

→