Skip to main content

Hybrid Search

Overview​

Hybrid Search is a powerful technique that combines the strengths of both semantic (vector-based) and lexical (keyword-based) search. This approach enhances the accuracy and relevance of results by leveraging the capabilities of both methods.

Hybrid Search can be configured and enabled in DataDios Hybrid Search Configuration.

Prerequisites​

Hybrid Search index updates are delivered through a task queue, fed from a PGMQ data source — the only connector in the Queue category today. Every tenant already has one: a PGMQ data source named default_pgmq_ds is created automatically and set as the tenant's Default Queue, so Hybrid Search works without any queue setup. The tenant's primary administrator can still choose where Hybrid Search gets its queue:

  1. Leave the Queue data source field as — None (use Default Queue) — — the default. Hybrid Search then uses the tenant-wide Default Queue, shared with other features such as Entity Catalog. To point that at a different queue, go to Manage → Configuration, open the Queue tab and change Default queue.
  2. Or create a separate PGMQ data source and set it in the Queue data source field (see Universal Semantic Search Configuration), to give Hybrid Search a queue of its own.

Only the primary administrator can save either setting. Other administrators are refused, because the queue is resolved and accessed under the primary administrator's account.

warning

Both settings can end up empty — for example if the Default Queue was cleared, or its data source was deleted from the Data sources page. If neither is set, Hybrid Search index updates stop. Existing results are still returned, but they go stale — newly added or changed records are not indexed until one of the two is set, and no error is shown to end users.

Tenants that already configured a Default Queue for another feature (for example, Entity Catalog) keep it — auto-provisioning does not overwrite an existing choice, and nothing needs to change.

On-premises upgrades

DataDios no longer runs a RabbitMQ broker. On-premises deployments upgrading in place should bring the stack up with docker compose up -d --remove-orphans so the now-unused rabbitmq container is removed.

Key Characteristics​

  • Semantic Understanding: Returns results based on the meaning of a query, not just keywords.
  • Hybrid Modes: Offers Full Text, Semantic, or Hybrid search options.
  • Module & User-Level Control: Admins can specify which modules and users can access semantic search.
  • Relevance Ranking: Results are scored and ranked by contextual relevance.

Benefits​

Best of Both Worlds: Precision + Semantics​

  • Lexical search (e.g., BM25) excels at exact keyword matches.
  • Semantic search (e.g., vector embeddings) captures meaning and context.
  • Hybrid search combines both, providing relevant results even when:
    • Users use different terminology (e.g., "laptop" vs. "notebook")
    • Queries are vague, long, or in natural language
    • Exact keywords are missing but the intent is clear

Improved User Experience​

  • Users find relevant content faster
  • Reduces frustration from irrelevant or missing results
  • Enhances discovery of related information
  • Improves overall satisfaction and productivity

Improved Recall Without Losing Precision​

  • Lexical-only search may miss results due to vocabulary mismatch
  • Semantic-only search may return overly broad or imprecise results
  • Hybrid search increases recall (more results found) while maintaining precision (relevance to the query)

Flexibility and Customization​

  • Admins can configure which modules and users access semantic search
  • Tailor the search experience to specific user needs and data types
  • Supports different search types (Full Text, Semantic, Hybrid) based on context
  • Enables gradual adoption of semantic search alongside keyword search

Best Practices​

  • Choose search type based on user needs and data characteristics
  • Apply semantic search only to modules where it adds meaningful value
  • Ensure underlying data is clean, consistent, and semantically rich:
    • Use clear object naming
    • Maintain well-labeled fields
    • Provide meaningful descriptions
  • Train Users on Semantic Behavior:
    • Educate users on how to phrase queries for optimal results
    • Clarify that ranking is based on meaning, not just word matches
  • Communicate Submission Finality:
    • Inform admins that once submitted, settings cannot be changed
    • Encourage careful review before submission

For additional support or advanced features, contact the DataDios support team.