Skip to main content

Web Search

Web search providers, result limits, the web loader and the YouTube loader.

These variables are set on the Hrida.ai container. Variables marked ConfigVar are saved to the database on first start — see how ConfigVar works before changing them later.

  • Type: bool
  • Default: False
  • Description: Enable web search toggle.
  • Persistence: This environment variable is a ConfigVar variable.

ENABLE_SEARCH_QUERY_GENERATION​

  • Type: bool
  • Default: True
  • Description: Only applies to Default Function Calling mode, which is legacy and no longer supported. If True: an LLM generates optimized, distilled search queries from the conversation context. If False: the user's last message is used verbatim as the web search query. Native Mode (the supported mode) uses the model's own search_web tool call and does not consult this setting.
  • Persistence: This environment variable is a ConfigVar variable.

WEB_SEARCH_TRUST_ENV​

  • Type: bool
  • Default: True
  • Description: Routes the web-page content fetcher (the stage that scrapes result pages after a web search, and the general website/URL loader) through the proxy defined by the http_proxy / https_proxy environment variables, also honoring no_proxy and .netrc. Needed because the default fetcher uses aiohttp, which — unlike requests — ignores these proxy variables unless told to trust the environment. This does not affect the search-engine query request itself (that already respects proxy env vars), and it never overrides an explicitly configured proxy; for the Firecrawl/Tavily/Playwright loaders it only acts as a fallback when no proxy is otherwise set.
  • Persistence: This environment variable is a ConfigVar variable.

WEB_FETCH_FILTER_LIST​

  • Type: string (comma-separated list)
  • Default: "" (empty, but default blocklist is always applied)
  • Description: Configures additional URL filtering rules for web fetch operations to prevent Server-Side Request Forgery (SSRF) attacks. The system includes a default blocklist that protects against access to cloud metadata endpoints (AWS, Google Cloud, Azure, Alibaba Cloud). Entries without a ! prefix are treated as an allow list (only these domains are permitted), while entries with a ! prefix are added to the block list (these domains are always denied). The default blocklist includes !169.254.169.254, !fd00:ec2::254, !metadata.google.internal, !metadata.azure.com, and !100.100.100.200. Custom entries are merged with the default blocklist.
info

Example:

Block additional domains: WEB_FETCH_FILTER_LIST="!internal.company.com,!192.168.1.1" Allow only specific domains: WEB_FETCH_FILTER_LIST="example.com,trusted-site.org"

WEB_SEARCH_DOMAIN_FILTER_LIST​

  • Type: list of str
  • Default: []
  • Description: Comma-separated list of domains to filter web search results. Domains prefixed with ! are blocked; domains without prefix create an allowlist (only those domains permitted).
  • Persistence: This environment variable is a ConfigVar variable.

WEB_SEARCH_RESULT_COUNT​

  • Type: int
  • Default: 3
  • Description: Maximum number of web search results to crawl. In Native/Agentic tool calling, this is also the default search_web result count when the model omits count, and the maximum cap when the model provides count.
  • Persistence: This environment variable is a ConfigVar variable.

WEB_SEARCH_CONCURRENT_REQUESTS​

  • Type: int
  • Default: 0
  • Description: Limits the number of concurrent search requests to the search engine provider. Set to 0 for unlimited concurrency (default). Set to 1 for sequential run to prevent rate limiting errors (e.g., Brave Free Tier).
  • Persistence: This environment variable is a ConfigVar variable.

WEB_FETCH_MAX_CONTENT_LENGTH​

  • Type: int
  • Default: None (no limit)
  • Description: Maximum number of characters to return from fetched URLs. When set, content exceeding this limit is truncated. Previously hardcoded at 50,000 characters. Leave empty or unset to return full content without truncation. Useful for controlling context window usage with large web pages.
  • Persistence: This environment variable is a ConfigVar variable.

WEB_LOADER_CONCURRENT_REQUESTS​

  • Type: int
  • Default: 10
  • Description: Specifies the number of concurrent requests used by the web loader to fetch content from web pages returned by search results. This directly impacts how many pages can be crawled simultaneously.
  • Persistence: This environment variable is a ConfigVar variable.
info

"WEB_LOADER_CONCURRENT_REQUESTS" was previously named "WEB_SEARCH_CONCURRENT_REQUESTS". The variable "WEB_SEARCH_CONCURRENT_REQUESTS" has been repurposed to control the concurrency of the search engine requests (see above). To control the web loader concurrency (fetching content from results), you MUST use "WEB_LOADER_CONCURRENT_REQUESTS".

WEB_SEARCH_ENGINE​

  • Type: str
  • Options:
    • searxng - Uses the SearXNG search engine.
    • google_pse - Uses the Google Programmable Search Engine.
    • brave - Uses the Brave search engine.
    • brave_llm_context - Uses Brave's LLM Context API endpoint (/res/v1/llm/context). Returns pre-extracted, relevance-scored page passages instead of short snippets, so Hrida.ai skips the post-search scraping step. Same API key and rate limits as brave.
    • kagi - Uses the Kagi search engine.
    • mojeek - Uses the Mojeek search engine.
    • bocha - Uses the Bocha search engine.
    • serpstack - Uses the Serpstack search engine.
    • serper - Uses the Serper search engine.
    • serply - Uses the Serply search engine.
    • searchapi - Uses the SearchAPI search engine.
    • serpapi - Uses the SerpApi search engine.
    • duckduckgo - Uses the DuckDuckGo search engine.
    • tavily - Uses the Tavily search engine.
    • jina - Uses the Jina search engine.
    • bing - Uses the Bing search engine.
    • exa - Uses the Exa search engine.
    • perplexity - Uses the Perplexity API to access perplexity's AI models. Calls their AI models, which execute a search and also return a full response.
    • perplexity_search - Uses the Perplexity Search API search engine. In contrast to the perplexity option, this uses Perplexity's web search API for searching the web and retrieving results.
    • sougou - Uses the Sougou search engine.
    • ollama_cloud - Uses the Ollama Cloud search engine.
    • azure_ai_search
    • yacy
    • yandex - Uses the Yandex Search API.
    • youcom - Uses the You.com YDC Index API for web search.
  • Persistence: This environment variable is a ConfigVar variable.

DDGS_BACKEND​

  • Type: str
  • Default: auto
  • Options: auto (Random), bing, brave, duckduckgo, google, grokipedia, mojeek, wikipedia, yahoo, yandex.
  • Description: Specifies the backend to be used by the DDGS engine.
  • Persistence: This environment variable is a ConfigVar variable. It can be configured in the Admin Panel > Settings > Web Search > DDGS Backend when DDGS is selected as the search engine.

BYPASS_WEB_SEARCH_EMBEDDING_AND_RETRIEVAL​

  • Type: bool
  • Default: False
  • Description: Bypasses the web search embedding and retrieval process.
  • Persistence: This environment variable is a ConfigVar variable.

BYPASS_WEB_SEARCH_WEB_LOADER​

  • Type: bool
  • Default: False
  • Description: Bypasses the web loader when performing web search. When enabled, only snippets from the search engine are used, and the full page content is not fetched.
  • Persistence: This environment variable is a ConfigVar variable.

SEARXNG_QUERY_URL​

  • Type: str
  • Description: The SearXNG search API URL supporting JSON output. {'<query>'} is replaced with the search query. Example: http://searxng.local/search?q={'<query>'}
  • Persistence: This environment variable is a ConfigVar variable.

SEARXNG_LANGUAGE​

  • Type: str
  • Default: all
  • Description: This variable is used in the request to searxng as the "search language" (argument "language").
  • Persistence: This environment variable is a ConfigVar variable.

GOOGLE_PSE_API_KEY​

  • Type: str
  • Description: Sets the API key for the Google Programmable Search Engine (PSE) service.
  • Persistence: This environment variable is a ConfigVar variable.

GOOGLE_PSE_ENGINE_ID​

  • Type: str
  • Description: The engine ID for the Google Programmable Search Engine (PSE) service.
  • Persistence: This environment variable is a ConfigVar variable.

BRAVE_SEARCH_API_KEY​

  • Type: str
  • Description: Sets the API key for the Brave Search API. Used by both the brave and brave_llm_context engines.
  • Persistence: This environment variable is a ConfigVar variable.
info

Brave's free tier enforces a rate limit of 1 request per second. Hrida.ai automatically retries requests that receive HTTP 429 rate limit errors after a 1-second delay. For free tier users, set WEB_SEARCH_CONCURRENT_REQUESTS to 1 to ensure sequential request processing. See the Brave web search documentation for more details.

BRAVE_SEARCH_CONTEXT_TOKENS​

  • Type: int
  • Default: 8192
  • Description: Maximum total tokens to retrieve per query when WEB_SEARCH_ENGINE=brave_llm_context. Sent to Brave's LLM Context API as maximum_number_of_tokens. Valid range is 1024–32768. Higher values pull richer extracted passages at the cost of API quota; lower values keep responses lean. Configurable via Admin Panel → Settings → Web Search → Context Tokens (only shown when the brave_llm_context engine is selected).
  • Persistence: This environment variable is a ConfigVar variable. Stored at config key rag.web.search.brave_search_context_tokens.

KAGI_SEARCH_API_KEY​

  • Type: str
  • Description: Sets the API key for Kagi Search API.
  • Persistence: This environment variable is a ConfigVar variable.

MOJEEK_SEARCH_API_KEY​

  • Type: str
  • Description: Sets the API key for Mojeek Search API.
  • Persistence: This environment variable is a ConfigVar variable.

SERPSTACK_API_KEY​

  • Type: str
  • Description: Sets the API key for Serpstack search API.
  • Persistence: This environment variable is a ConfigVar variable.

SERPSTACK_HTTPS​

  • Type: bool
  • Default: True
  • Description: Configures the use of HTTPS for Serpstack requests. Free tier requests are restricted to HTTP only.
  • Persistence: This environment variable is a ConfigVar variable.

SERPER_API_KEY​

  • Type: str
  • Description: Sets the API key for Serper search API.
  • Persistence: This environment variable is a ConfigVar variable.

SERPLY_API_KEY​

  • Type: str
  • Description: Sets the API key for Serply search API.
  • Persistence: This environment variable is a ConfigVar variable.

SEARCHAPI_API_KEY​

  • Type: str
  • Description: Sets the API key for SearchAPI.
  • Persistence: This environment variable is a ConfigVar variable.

SEARCHAPI_ENGINE​

  • Type: str
  • Description: Sets the SearchAPI engine.
  • Persistence: This environment variable is a ConfigVar variable.

TAVILY_API_KEY​

  • Type: str
  • Description: Sets the API key for Tavily search API.
  • Persistence: This environment variable is a ConfigVar variable.

JINA_API_KEY​

  • Type: str
  • Description: Sets the API key for Jina.
  • Persistence: This environment variable is a ConfigVar variable.

JINA_API_BASE_URL​

  • Type: str
  • Default: https://s.jina.ai/
  • Description: Sets the Base URL for Jina Search API. Useful for specifying custom or regional endpoints (e.g., https://eu-s-beta.jina.ai/).
  • Persistence: This environment variable is a ConfigVar variable. It can be configured in the Admin Panel > Settings > Web Search > Jina API Base URL.

BING_SEARCH_V7_ENDPOINT​

  • Type: str
  • Description: Sets the endpoint for Bing Search API.
  • Persistence: This environment variable is a ConfigVar variable.

BING_SEARCH_V7_SUBSCRIPTION_KEY​

  • Type: str
  • Default: https://api.bing.microsoft.com/v7.0/search
  • Description: Sets the subscription key for Bing Search API.
  • Persistence: This environment variable is a ConfigVar variable.

BOCHA_SEARCH_API_KEY​

  • Type: str
  • Default: None
  • Description: Sets the API key for Bocha Search API.
  • Persistence: This environment variable is a ConfigVar variable.

EXA_API_KEY​

  • Type: str
  • Default: None
  • Description: Sets the API key for Exa search API.
  • Persistence: This environment variable is a ConfigVar variable.

SERPAPI_API_KEY​

  • Type: str
  • Default: None
  • Description: Sets the API key for SerpAPI.
  • Persistence: This environment variable is a ConfigVar variable.

SERPAPI_ENGINE​

  • Type: str
  • Default: None
  • Description: Specifies the search engine to use for SerpAPI.
  • Persistence: This environment variable is a ConfigVar variable.

AZURE_AI_SEARCH_API_KEY​

  • Type: str
  • Default: None
  • Description: API key (query key or admin key) for authenticating with Azure AI Search service. Required for using Azure AI Search as a web search provider.
  • Persistence: This environment variable is a ConfigVar variable.

AZURE_AI_SEARCH_ENDPOINT​

  • Type: str
  • Default: None
  • Description: Azure Search service endpoint URL. Specifies which Azure Search service instance to connect to.
  • Example: https://myservice.search.windows.net, https://company-search.search.windows.net
  • Persistence: This environment variable is a ConfigVar variable.

AZURE_AI_SEARCH_INDEX_NAME​

  • Type: str
  • Default: None
  • Description: Name of the search index to query within your Azure Search service. Different indexes can contain different types of searchable content.
  • Example: my-search-index, documents-index, knowledge-base
  • Persistence: This environment variable is a ConfigVar variable.

SOUGOU_API_SID​

  • Type: str
  • Default: None
  • Description: Sets the Sogou API SID.
  • Persistence: This environment variable is a ConfigVar variable.

SOUGOU_API_SK​

  • Type: str
  • Default: None
  • Description: Sets the Sogou API SK.
  • Persistence: This environment variable is a ConfigVar variable.

OLLAMA_CLOUD_WEB_SEARCH_API_KEY​

  • Type: str
  • Default: None
  • Description: Sets the Ollama Cloud Web Search API Key.
  • Persistence: This environment variable is a ConfigVar variable.

TAVILY_EXTRACT_DEPTH​

  • Type: str
  • Default: basic
  • Description: Specifies the extract depth for Tavily search results.
  • Persistence: This environment variable is a ConfigVar variable.

YACY_QUERY_URL​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the query URL for YaCy search engine integration. Should point to a YaCy instance's search API endpoint.
  • Persistence: This environment variable is a ConfigVar variable.

YACY_USERNAME​

  • Type: str
  • Default: Empty string (' ')
  • Description: Specifies the username for authenticated access to YaCy search engine.
  • Persistence: This environment variable is a ConfigVar variable.

YACY_PASSWORD​

  • Type: str
  • Default: Empty string (' ')
  • Description: Specifies the password for authenticated access to YaCy search engine.
  • Persistence: This environment variable is a ConfigVar variable.

EXTERNAL_WEB_SEARCH_URL​

  • Type: str
  • Default: Empty string (' ')
  • Description: Specifies the URL of an external web search service API endpoint for custom search integrations.
  • Persistence: This environment variable is a ConfigVar variable.

EXTERNAL_WEB_SEARCH_API_KEY​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the API key for authenticating with the external web search service.
  • Persistence: This environment variable is a ConfigVar variable.

EXTERNAL_WEB_LOADER_URL​

  • Type: str
  • Default: Empty string (' ')
  • Description: Specifies the URL of an external web content loader service for fetching and processing web pages.
  • Persistence: This environment variable is a ConfigVar variable.

EXTERNAL_WEB_LOADER_API_KEY​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the API key for authenticating with the external web loader service.
  • Persistence: This environment variable is a ConfigVar variable.

YANDEX_WEB_SEARCH_URL​

  • Type: str
  • Default: https://searchapi.api.cloud.yandex.net/v2/web/search
  • Description: Specifies the URL of the Yandex Web Search service API endpoint.
  • Persistence: This environment variable is a ConfigVar variable.

YANDEX_WEB_SEARCH_API_KEY​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the API key for authenticating with the Yandex Web Search service.
  • Persistence: This environment variable is a ConfigVar variable.

YANDEX_WEB_SEARCH_CONFIG​

  • Type: str
  • Default: Empty string (' ')
  • Description: Optional JSON configuration string for Yandex Web Search. Can be used to set parameters like searchType or region as per the Yandex API documentation.
  • Persistence: This environment variable is a ConfigVar variable.

PERPLEXITY_API_KEY​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the API key for Perplexity API.
  • Persistence: This environment variable is a ConfigVar variable.

PERPLEXITY_SEARCH_API_URL​

  • Type: str
  • Default: https://api.perplexity.ai/search
  • Description: Configures the API endpoint for Perplexity Search. Allows using custom or self-hosted Perplexity-compatible API endpoints (such as LiteLLM's /search endpoint) instead of the hardcoded default for the official Perplexity API. This enables flexibility in routing search requests to alternative providers or internal proxies. Note: If using LiteLLM, append the specific provider name to the URL path.
  • Example: http://my-litellm-server.com/search/perplexity-search
  • Persistence: This environment variable is a ConfigVar variable.

PERPLEXITY_MODEL​

  • Type: str
  • Default: sonar
  • Description: Specifies the Perplexity AI model to use for search queries when using Perplexity as the web search engine.
  • Persistence: This environment variable is a ConfigVar variable.
info

Perplexity is different from perplexity_search. If you use perplexity_search, this variable is not relevant to you.

PERPLEXITY_SEARCH_CONTEXT_USAGE​

  • Type: str
  • Default: medium
  • Description: Controls the amount of search context used by Perplexity AI. Options typically include low, medium, high.
  • Persistence: This environment variable is a ConfigVar variable.
info

Perplexity is different from perplexity_search. If you use perplexity, this variable is not relevant to you.

YOUCOM_API_KEY​

  • Type: str
  • Default: Empty string (' ')
  • Description: Sets the API key for You.com YDC Index API web search. Required when WEB_SEARCH_ENGINE is set to youcom. Obtain an API key from You.com API.
  • Persistence: This environment variable is a ConfigVar variable.

Web Loader Configuration​

WEB_LOADER_ENGINE​

  • Type: str
  • Default: (empty)
  • Description: Specifies the loader to use for retrieving and processing web content.
  • Options:
    • safe_web (default) - Uses internal fetching with enhanced error handling.
    • playwright - Uses Playwright for rendering pages with JavaScript support.
    • firecrawl - Uses Firecrawl service.
    • tavily - Uses Tavily service.
    • external - Uses an external web loader API.
  • Persistence: This environment variable is a ConfigVar variable.
info

When using playwright, you have two options:

  1. If PLAYWRIGHT_WS_URI is not set, Playwright with Chromium dependencies will be automatically installed in the Hrida.ai container on launch.
  2. If PLAYWRIGHT_WS_URI is set, Hrida.ai will connect to a remote browser instance instead of installing dependencies locally.

PLAYWRIGHT_WS_URL​

  • Type: str
  • Default: None
  • Description: Specifies the WebSocket URI of a remote Playwright browser instance. When set, Hrida.ai will use this remote browser instead of installing browser dependencies locally. This is particularly useful in containerized environments where you want to keep the Hrida.ai container lightweight and separate browser concerns. Example: ws://playwright:3000
  • Persistence: This environment variable is a ConfigVar variable.
tip

Using a remote Playwright browser via PLAYWRIGHT_WS_URL can be beneficial for:

  • Reducing the size of the Hrida.ai container
  • Using a different browser other than the default Chromium
  • Connecting to a non-headless (GUI) browser

FIRECRAWL_API_BASE_URL​

  • Type: str
  • Default: https://api.firecrawl.dev
  • Description: Sets the base URL for Firecrawl API.
  • Persistence: This environment variable is a ConfigVar variable.

FIRECRAWL_API_KEY​

  • Type: str
  • Default: None
  • Description: Sets the API key for Firecrawl API.
  • Persistence: This environment variable is a ConfigVar variable.

FIRECRAWL_TIMEOUT​

  • Type: int
  • Default: None
  • Description: Specifies the timeout in milliseconds for Firecrawl requests. If not set, the default Firecrawl timeout is used.
  • Persistence: This environment variable is a ConfigVar variable. It can be configured in the Admin Panel > Settings > Web Search > Firecrawl Timeout.

PLAYWRIGHT_TIMEOUT​

  • Type: int
  • Default: Empty string (' '), since None is set as default.
  • Description: Specifies the timeout for Playwright requests.
  • Persistence: This environment variable is a ConfigVar variable.

WEB_LOADER_TIMEOUT​

  • Type: float
  • Default: Empty string (' '), since None is set as default.
  • Description: Specifies the request timeout in seconds for the SafeWebBaseLoader when scraping web pages. Without this setting, web scraping operations can hang indefinitely on slow or unresponsive pages. Recommended values are 10–30 seconds depending on your network conditions.
  • Persistence: This environment variable is a ConfigVar variable.
warning

This timeout only applies when WEB_LOADER_ENGINE is set to safe_web or left empty (the default). It has no effect on Playwright or Firecrawl loader engines, which have their own timeout configurations (PLAYWRIGHT_TIMEOUT and Firecrawl's internal settings respectively).

YouTube Loader​

YOUTUBE_LOADER_PROXY_URL​

  • Type: str
  • Description: Sets the proxy URL for YouTube loader.
  • Persistence: This environment variable is a ConfigVar variable.

YOUTUBE_LOADER_LANGUAGE​

  • Type: str
  • Default: en
  • Description: Comma-separated list of language codes to try when fetching YouTube video transcriptions, in priority order.
  • Example: If set to es,de, Spanish transcriptions will be attempted first, then German if Spanish was not available, and lastly English.
note

Note: If none of the specified languages are available and en was not in your list, the system will automatically try English as a final fallback.

  • Persistence: This environment variable is a ConfigVar variable.
Hrida.ai is proprietary software of Zlabs Innovation. See the license for terms. © 2026 Zlabs Innovation.