> ## Documentation Index
> Fetch the complete documentation index at: https://docs.delphina.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Databricks Connection Guide

> Connect Delphina to Databricks with service account credentials and optional per-user OAuth

Every Databricks connection requires a **service account** — either a service principal or a personal access token. Delphina uses this for background operations like catalog refresh and knowledge extraction, and it serves as the default identity for user queries.

Optionally, you can enable **per-user OAuth** so that interactive queries run under each user's own Databricks identity, enforcing Unity Catalog permissions, row-level security, and column masking.

## Authentication overview

| Scenario                                             | Without per-user OAuth | With per-user OAuth    |
| ---------------------------------------------------- | ---------------------- | ---------------------- |
| Interactive queries (chat, data apps)                | Service account        | Per-user token         |
| Scheduled workflows                                  | Service account        | Workflow owner's token |
| Background jobs (catalog refresh, knowledge updates) | Service account        | Service account        |

## Connecting Databricks

### Step 1: Gather connection details

You'll need the following from your Databricks workspace:

| Field         | Where to find it                         | Example                           |
| ------------- | ---------------------------------------- | --------------------------------- |
| **Hostname**  | Workspace URL (without `https://`)       | `dbc-abc123.cloud.databricks.com` |
| **HTTP Path** | SQL Warehouse > Connection details       | `/sql/1.0/warehouses/abc123def`   |
| **Catalog**   | Optional — defaults to workspace default | `main`                            |
| **Schema**    | Optional — defaults to `default`         | `analytics`                       |

### Step 2: Choose a service account authentication method

Delphina supports two service account methods for Databricks:

<Tabs>
  <Tab title="Service Principal — Recommended">
    Use a Databricks service principal with OAuth machine-to-machine (M2M) credentials. Tokens are short-lived (1 hour) and refresh automatically — no manual rotation required.

    #### Create a service principal

    1. In your Databricks **account console**, go to **User management > Service principals**.
    2. Click **Add service principal** and give it a name (e.g., `Delphina`).
    3. Click on the service principal, then go to the **Credentials & secrets** tab.
    4. Click **Generate secret**. Copy the **Client ID** and **Secret** — the secret is only shown once.

    #### Grant permissions

    The service principal needs access to:

    * **SQL warehouse** — go to **SQL Warehouses**, click the kebab menu (⋮) on your warehouse row, select **Permissions**, and add the service principal with **Can use** access.
    * **Catalogs and schemas** — in Unity Catalog, grant `USE CATALOG`, `USE SCHEMA`, and `SELECT` on the tables Delphina should query:
      ```sql theme={null}
      GRANT USE CATALOG ON CATALOG my_catalog TO `my-service-principal`;
      GRANT USE SCHEMA ON SCHEMA my_catalog.my_schema TO `my-service-principal`;
      GRANT SELECT ON SCHEMA my_catalog.my_schema TO `my-service-principal`;
      ```
    * **Query history** — Delphina reads from `system.query.history` to identify important tables and active users during onboarding. Grant access:
      ```sql theme={null}
      GRANT SELECT ON TABLE system.query.history TO `my-service-principal`;
      ```
  </Tab>

  <Tab title="Personal Access Token">
    Use a Databricks personal access token (PAT) for simpler setups.

    #### Create a token

    1. In your Databricks workspace, go to **User Settings > Developer > Access Tokens**.
    2. Click **Generate New Token**.
    3. Set a description (e.g., `Delphina`) and expiration.
    4. Copy the token value.

    The token inherits the permissions of the user who created it.

    #### Required permissions

    The user account that owns the PAT needs:

    * **Can use** access on the SQL warehouse.
    * `SELECT` access on the catalogs, schemas, and tables Delphina should query.
    * `SELECT` on `system.query.history` — Delphina reads this table to identify important tables and active users during onboarding.

    <Warning>
      PATs are tied to a user account. If the user leaves or their permissions change, the connection breaks. For production use, prefer the Service Principal approach.
    </Warning>
  </Tab>
</Tabs>

### Step 3: Create the connection in Delphina

1. Go to **Org Admin > Warehouse Connections**.
2. Click **Add Connection** on the target workspace.
3. Set the **Warehouse Type** to **Databricks**.
4. Fill in **Hostname**, **HTTP Path**, and optionally **Catalog** and **Schema**.
5. Under **Authentication**, choose **Service Principal** or **Personal Access Token** and enter the credentials from Step 2.
6. Click **Create Connection**.

### Step 4: Test the connection

After saving, click **Test Connection** in the dialog footer to verify Delphina can reach your warehouse, list tables, and access query history.

***

## Per-user OAuth (optional)

Per-user OAuth is an optional layer on top of the service account. When enabled, queries run under each user's own Databricks identity instead of the shared service account. Scheduled workflows and data app refreshes run under the token of the user who created them. The service account is still used for catalog refresh and knowledge extraction.

### Why use per-user OAuth

* **Unity Catalog enforcement** — row-level security, column masking, and table ACLs apply per user.
* **Audit trail** — Databricks query logs show the actual user, not a shared service account.
* **Least privilege** — users only access data their Databricks role permits.

### Setting up per-user OAuth

#### 1. Create an OAuth application in Databricks

<Note>
  This is a separate OAuth application from the service principal (if you're using that for the service account). The per-user app uses the **authorization code** grant type.
</Note>

1. In your Databricks account console, go to **Settings > App Connections**.
2. Click **Add connection**.
3. Set the application name (e.g., `Delphina User Auth`).
4. Set the **redirect URI**:

```
https://app.delphina.ai/api/oauth/databricks/callback
```

5. Enable the **Authorization code** grant type.
6. Set the required scope: `sql`.
7. Generate a **Client Secret**.
8. Note the **Client ID** and **Client Secret**.

#### 2. Enable per-user OAuth in Delphina

1. Go to **Org Admin > Warehouse Connections**.
2. Edit your Databricks connection.
3. Switch to the **Authentication** tab.
4. Toggle **Enable per-user OAuth**.
5. Enter the **Client ID** and **Client Secret** from the per-user OAuth application.
6. Save the connection.

### User experience

Once per-user OAuth is enabled:

1. A banner appears at the top of the workspace prompting users to connect their Databricks account.
2. Clicking **Connect** opens a popup to the Databricks authorization page.
3. After authorizing, the popup closes and a confirmation toast appears.
4. The user's token is stored securely and refreshed automatically.

Users can manage their connections from **User Settings > Data Sources**, where they can see connection status, refresh tokens, or disconnect.

### Token validation

Delphina validates per-user OAuth tokens in the background. If a token becomes invalid (e.g., revoked in Databricks), the user's connection status updates to **Needs authentication** and they are prompted to re-authorize.

### Revoking access

To revoke a user's Delphina access to Databricks:

* **From Databricks** — remove the user's authorization from the OAuth application. Their refresh token will fail on the next validation check.
* **From Delphina** — the user can click **Disconnect** in **User Settings > Data Sources**.

***

## Troubleshooting

| Problem                                              | Fix                                                                                                                                           |
| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| **Test Connection fails**                            | Verify the hostname and HTTP path are correct. Ensure the service account has **Can use** access on the SQL warehouse.                        |
| **Query history check fails**                        | The service account needs `SELECT` on `system.query.history`. Run: `GRANT SELECT ON TABLE system.query.history TO \`my-service-principal\`;\` |
| **"Connect" banner doesn't appear**                  | Verify per-user OAuth is enabled on the connection and has a Client ID configured.                                                            |
| **OAuth popup shows redirect error**                 | Confirm the redirect URI in Databricks matches exactly: `https://app.delphina.ai/api/oauth/databricks/callback`                               |
| **User gets "insufficient privileges"**              | Queries run under the user's Databricks identity — check their permissions in Unity Catalog.                                                  |
| **Scheduled workflow fails**                         | The workflow owner must have completed the per-user OAuth flow.                                                                               |
| **Token shows "Needs authentication" after working** | The refresh token may have been revoked in Databricks, or the OAuth app's client secret was rotated. Have the user re-authorize.              |
| **Catalog refresh fails**                            | Catalog refresh uses the service account, not per-user tokens. Check the service account credentials on the connection.                       |