Skip to main content

Datasources & Operations

This document explains how to create, update, test, clone, and delete datasources in DataDios.

Quick Start​

Key Actions You Can Perform​

Datasource Creation and Connection Parameters Validation​

  1. Navigate to Data Explorer from the left navigation menu. The Data Management page lists all your existing connections, along with header metrics for Active Sources, Avg Quality, Drifting, and Monthly Cost.

  2. Click + Add connection in the top-right corner.

    Data Management page with the Add connection button

  3. The Create data source dialog opens on the Connection tab. Additional tabs — Drift, Lineage, Workload, and Profiling — let you configure optional capabilities for the same connection.

  4. Under IDENTITY, enter a Name for the datasource and choose its Type (for example, PostgreSQL Database).

  5. Under CONNECTION PARAMETERS, fill in the credentials for your database:

    • Host and Port — the address of your database server
    • Username — the account used to connect
    • Select Secret Source — choose Direct Input to type the secret here, or select a configured secret store
    • Enter Secret Key — the password or secret for the account
    • Database — the database you want to connect to
    • Schema — the schema to read from
    • Object Types — optionally restrict to specific object types, such as tables,views
    • Schedule Sync and Sync Batch Size — optionally control how often metadata is synchronised and how much is fetched per batch

    Create data source dialog with connection parameters

  6. Click Test Connection to validate the credentials before saving.

  7. Once validation succeeds, click Create. The new datasource appears in the Sources list on the Data Management page.

Update Datasource Details​

  1. On the Data Management page, locate your datasource in the Sources list.

  2. In the Actions column at the end of the row, click the pencil icon (Edit connection).

    Edit connection action in the sources list

  3. The Edit data source dialog opens on the Connection tab, pre-filled with the current configuration. Alongside Drift, Lineage, Workload and Profiling, an extra RAG settings tab is available when editing an existing source.

  4. Update the connection details (credentials, parameters, etc.) as needed. The Name and Type identify the source and cannot be changed here.

  5. Click Test Connection to validate your changes, then click Save.

    Edit data source dialog

The dialog footer also provides Copy, which duplicates the source — see Clone a Datasource.

Import Datasource​

Importing is simply datasource creation from a file, for when you already know the credentials are correct.

  1. Click the ... menu at the top-right of the Data Management page and choose Import connections....

    Import connections option in the actions menu

  2. Select a JSON file that follows the structure shown below. DataDios reads the file and creates the datasources directly — you can import multiple datasources at the same time.

  3. After import, validate a datasource by opening Edit connection and clicking Test Connection.

The same menu also offers Export selected, which writes the connections you have ticked out to a file — useful for copying connections between environments.

JSON Structure

[
    {
       "Name": "Datasource_Name",
       "Type": "SQLServer Database",
       "Connection Parameters": [
             {
                "host": "host-address",
                "port": "1433",
                "username": "db-username",
                "password": "db-password",
                "database": "db-to-connect",
                "schema": "schema-to-connect",
                "object_types": "tables,views"
             }
       ]
    }
 ]

Clone a Datasource​

  1. On the Data Management page, locate your datasource in the Sources list.

  2. Click the pencil icon (Edit connection) in the Actions column to open the Edit data source dialog.

  3. Click Copy in the bottom-left of the dialog footer.

    Copy option in the Edit data source dialog

  4. Cloning follows the same process as datasource creation and creates a copy you can adjust.

  5. Give the copy a new Name, update any credentials and connection parameters that must differ, then click Test Connection and Save.

Ensure you update credentials and connection parameters correctly to avoid conflicts with the original source.

Delete a Datasource​

  1. On the Data Management page, tick the checkbox next to the datasource you want to remove. The ... menu at the top-right updates to show how many rows are selected.

  2. Open the ... menu and choose Delete selected.

    Delete selected in the actions menu

  3. The Delete data source? dialog opens, naming the source and its type so you can confirm you picked the right one.

  4. Expand View affected items to see exactly what the deletion will touch:

    • Stop scheduled workflows and set their state to discarded
    • Delete workflows that are not completed
    • Metadata timeline and versions

  5. Optionally tick Safe Delete, which also deletes the related workflows.

  6. Click Delete to confirm.

    Delete data source confirmation dialog

caution

The data source and its dependent data will be deleted. Review the affected items before confirming.

Explore Datasources​

For detailed steps on creating and testing specific datasources, check out the following: