RAG Settings
RAG stands for Retrieval Augmented Generation. In plain terms: DataDios reads through a data source, turns what it finds into a searchable index, and then lets the AI look things up in that index before it answers you. The AI answers from your own data instead of guessing.
You turn this on per data source, and you turn it on yourself — nothing is indexed until you ask for it.
Which sources support it
RAG is only offered for source types that support indexing — both interfaces read the same list from the backend. Today that is:
| Kind of source | Types |
|---|---|
| Databases | Databricks Database, MySQL, Oracle, PostgreSQL, SQL Server, Snowflake |
| Warehouse / query engines | Trino Engine |
| Files and folders | Amazon S3, Azure Blob, Folder |
| Knowledge bases | Confluence, GitHub Wiki, Google Drive, SharePoint |
| Code | GitHub |
| Issue trackers | Jira, ServiceNow |
| CRM | Salesforce Cloud |
| Governance | Collibra, Collibra DQ |
| Logs | Logs |
If a source does not offer RAG, that source type does not support it yet.
What gets indexed from a database
This is the part worth reading before you enable it, because databases behave differently from files.
For a database or warehouse source, DataDios indexes the shape of your tables, not the rows inside them. A table contributes its name and its column names. The values in the table are never sent to the embedding model and never stored in the index.
That is deliberate. It means you can index a production warehouse without customer data leaving it, and it keeps the index small — a table costs you one short entry however many rows it holds.
It also means the AI can answer "which table holds order totals?" but not "what was order 12345?". For questions about actual values, use AI Studio, which queries the source directly.
This is enforced in the connector, for all seven database and warehouse types that support RAG: Databricks, MySQL, Oracle, PostgreSQL, SQL Server, Snowflake and Trino Engine. Even if you tick RAG Data for Selected Objects, table rows are still not indexed and the setting has no effect. Indexing contents applies to file, knowledge base, issue and code sources.
For Databricks specifically, the indexed entry holds the catalog, schema, table and column names. Primary and foreign keys are not included, because Unity Catalog treats them as informational and DataDios does not collect them.
Before you start
You need two things:
- An embedding model data source. An embedding model is what turns text into the numbers that make meaning-based search possible. Add one under Data Sources first — any source whose category is Gen AI Models, such as an OpenAI, Claude, Gemini, Cohere or Ollama model.
- Somewhere to keep the index. DataDios ships with its own built-in store and uses it by default, so there is nothing to set up. It is the only option offered today.
Turn it on
- Go to Data Sources and find the source you want to index.
- Open Edit connection on that source.
- Select the RAG settings tab. The tab only appears when you are editing an existing source, not while creating one.
- Choose your embedding data source and embedding model. Both are required.
- Leave the rest at its defaults unless you have a reason to change it — see the table below.
- Click Enable.
DataDios then starts an indexing job in the background. The panel shows a progress card with the last job and, if you set up automatic syncing, the next run time. You can close the dialog; the job keeps running.
To change a setting later, come back to the same tab and click Update. To stop indexing and remove the source from RAG, click Disable.
The settings
| Setting | What it does | Default |
|---|---|---|
| Embedding data source | Which model turns your content into searchable numbers | none — you must choose |
| Embedding model | Which model from that source | none — you must choose |
| Chunk size | How much text goes into one searchable piece, in tokens. DataDios estimates 1.3 tokens per word, so 1000 tokens is roughly 770 words | 1000 |
| Chunk overlap | How much each piece repeats from the one before, so a sentence split across two pieces is still findable | 200 |
| Chunking strategy | How the text is cut up. auto picks a sensible method from the file type; you can also choose tokens, sentences, paragraphs, pages, lines, semantic, code or recursive | auto |
| RAG Metadata for All Objects | Index the shape of every object — table and column names | on |
| RAG Data for Selected Objects | Also index the contents. Has no effect on any database source (see above) | off |
| Ingest Scope | With contents turned on, whether to cover the whole source or just objects you pick | Partial Selection |
| Auto Sync Interval | Re-index on a schedule: Daily, Weekly or Monthly. Leave it blank to keep auto sync off | off |
Chunk size and overlap only matter for sources whose contents are indexed — files, wiki pages, issues and code. For a database source, each table's entry is short and these settings make no practical difference.
Where the index is used
Once a source is indexed, an agent workflow can use it as its RAG Datasource to supply extra context during analysis. The Agents module is not part of the new interface yet — see Agent workflows.
If something goes wrong
- RAG is not offered on the source. Either the source type does not support it (see the table above), or — in the new interface — you are creating a source rather than editing one. Save it first, then edit it.
- No embedding models to choose from. You have not added a Gen AI model data source yet, or the one you added has no embedding model configured.
- The job finishes but the search finds nothing. For a database source, check that RAG Metadata for All Objects is on. If you were expecting to search row values, that is not what database indexing does — see above.