Connecting Databricks to Bicycle

Connect to Databricks SQL warehouses for lakehouse analytics. Bicycle reads Unity Catalog tables as SIGNAL sources for metrics, events, and dashboards.

Data in Databricks is organized as catalog → schema → table. Bicycle discovers tables from system.information_schema.tables and names streams as catalog/schema/table.

Requirements

To follow the steps in this guide, you'll need:

Permission to create connectors in Bicycle
A Databricks workspace with a running SQL warehouse
Permission in Databricks to create or manage a service principal or personal access token (PAT), and to grant Unity Catalog privileges
The warehouse Server hostname and HTTP path from the warehouse connection details

Work with your Databricks administrator to provision access. In most organizations, a platform or data team runs the SQL grants below; an analyst or engineer then completes the connection in Bicycle.

Item	Description
Server hostname	Workspace hostname for the SQL warehouse (for example, `dbc-12345.cloud.databricks.com`)
HTTP path	SQL warehouse HTTP path (for example, `/sql/1.0/warehouses/abc123def456`)
Port (optional)	Databricks port; defaults to `443` if omitted
Catalog (optional)	Unity Catalog name to scope discovery; omit to discover all accessible catalogs
Schema (optional)	Schema within the catalog; omit to discover all accessible schemas
Table Pattern (optional)	Regex used to filter table names during discovery
Authentication	API key (PAT) or OAuth (M2M) service-principal credentials

Official Databricks documentation

Prepare Databricks access

Complete these steps once, regardless of which authentication method you use in Bicycle.

Retrieve SQL warehouse connection details

In your Databricks workspace, go to SQL Warehouses and open the warehouse Bicycle should use.
Open the Connection details tab.
Copy the Server hostname and HTTP path.

Follow the Databricks guide for getting connection details for a SQL warehouse.

Grant SQL warehouse access

The Databricks user or service principal must have CAN USE on the SQL warehouse.

In the workspace, open SQL Warehouses and select your warehouse.
Go to the Permissions tab.
Grant CAN USE to the identity that will connect:
- PAT: the Databricks username that owns the token
- OAuth M2M: the service principal

Grant Unity Catalog permissions

The connecting identity must have the following privileges on each catalog and schema Bicycle should read. Run these in a SQL warehouse or notebook as a metastore admin or catalog owner:

Grant read access in Unity Catalog
GRANT USE CATALOG ON CATALOG <catalog> TO `<principal>`;
GRANT USE SCHEMA ON SCHEMA <catalog>.<schema> TO `<principal>`;
GRANT SELECT ON SCHEMA <catalog>.<schema> TO `<principal>`;

Replace:

<catalog> — Unity Catalog name (for example, main or analytics)
<schema> — schema within that catalog (for example, default)
<principal> — identity that connects:
- PAT: the Databricks username that owns the token (for example, connector_reader or user@company.com)
- OAuth M2M: the service principal Client ID / Application ID (UUID)

Repeat the schema-level grants for each schema Bicycle should discover and read. GRANT SELECT ON SCHEMA covers all current and future tables in that schema.

note

Create a dedicated service principal or token owner for Bicycle. Do not use personal credentials for production connections.

Connect with API key (PAT)

Use a personal access token when your organization allows token-based service access.

In Databricks

In Databricks, open your profile menu and go to Settings > Developer > Access tokens.
Click Generate new token, set an expiration if required, and copy the token. It is shown only once.
Grant the token owner CAN USE on the SQL warehouse and the Unity Catalog permissions above (use the token owner's username as <principal>).

For service-principal tokens, follow the Databricks guide for personal access tokens for service principals.

warning

Store the token in your organization's secrets manager, rotate it according to your security policy, and redact it from screenshots.

In Bicycle

Step 1: Open Connectors and select Databricks

In Bicycle, go to Connectors. Under Recommended, find Databricks and click + Connect.

Navigate to the Databricks connector

Step 2: Choose API key

Enter a connection name that identifies the environment, such as Databricks Production. Select API key as the Connector method, then click Next.

Select API key as the connector method

Step 3: Add Databricks connection details

On the Add details step, fill in the Databricks connection fields and paste the personal access token.

Databricks connection fields

Field	Required	Description
Server hostname	Yes	Workspace hostname from the SQL warehouse connection details
HTTP path	Yes	SQL warehouse HTTP path (for example, `/sql/1.0/warehouses/<id>`)
Port	No	Connection port. Defaults to `443` if omitted
Catalog	No	Unity Catalog to scope discovery. If omitted, Bicycle discovers all catalogs the identity can access
Schema	No	Schema to scope discovery. If omitted, Bicycle discovers all accessible schemas
Table Pattern	No	Regex used to filter discovered table names, for example `events_.*`

note

Setting Catalog is recommended when you know which catalog Bicycle should read. Leaving it blank scans all accessible catalogs and can be slower on large metastores.

API key credentials

Field	Required	Description
Personal Access Token	Yes	Databricks personal access token

Example values:

Server hostname: dbc-a1b2c3d4-e5f6.cloud.databricks.com
HTTP path: /sql/1.0/warehouses/abc123def456
Catalog: main
Schema: analytics
Table Pattern: orders_.*

Databricks connection details form

Filled Databricks API key connection form

Step 4: Test the connection

Click Test connection. Bicycle validates the credentials and confirms it can connect to the configured SQL warehouse. When the test succeeds, you will see Ready to add this connection.

Databricks connection test succeeded

Step 5: Add the connection

Click Add connection. When the connection is created successfully, you will see a confirmation with the connection name, ID, and method (API key). Click Close to return to the Connectors page.

Databricks connection successfully added

Connect with OAuth (M2M)

Use OAuth machine-to-machine credentials for a non-personal service principal. This is the recommended path for production connections.

note

Bicycle uses OAuth M2M (client ID and client secret). This is not an interactive browser login. The Databricks JDBC driver fetches and refreshes the access token automatically.