Skip to main content

Connecting Google Cloud Storage to Bicycle

Connect to Google Cloud Storage buckets for file storage.

Requirements​

Before you configure the connector in Bicycle, prepare the following in Google Cloud:

  • A Google Cloud project with access to the target GCS bucket
  • A GCS bucket containing the files you want to read
  • Bucket Name, Path, and File Pattern for the files you want to read

If you connect using the Private toggle (default), you also need:

  • A service account with access to the target bucket
  • A downloaded service account JSON key

Grant the service account at least Storage Object Viewer (roles/storage.objectViewer) on the target bucket for read access. If you also need write access, grant Storage Object Creator (roles/storage.objectCreator).

Official Google Cloud documentation

Prepare a service account in Google Cloud (Private mode)​

Complete these steps only if you plan to use Private mode and paste a service account JSON key in Bicycle.

Prepare a service account

  1. Go to the Google Cloud Console.
  2. Navigate to IAM & Admin > Service Accounts.
  3. Create a new service account, or select an existing one.

Grant bucket access

Open the target bucket, go to the Permissions tab, and grant the service account Storage Object Viewer on the bucket. Add Storage Object Creator only if Bicycle needs write access.

Grant Storage Object Viewer on the GCS bucket

Create a JSON key

  1. Open the service account.
  2. Go to Keys > Add Key > Create new key.
  3. Select JSON, then click Create.
  4. Download the JSON key file and keep it secure.

Select JSON as the private key type

Private key saved confirmation

Setup in Bicycle​

Open Connectors and select Google Cloud Storage

In the sidebar, go to Connectors. Under Recommended, find Google Cloud Storage and click + Connect.

Navigate to the Google Cloud Storage connector

Choose Service Account

Enter a connection name (for example, MyGCSConnection), select Service Account as the connector method, then click Next.

Select Service Account as the connector method

Add GCS connection details

On the Add details step, use the Private toggle to choose how Bicycle authenticates to GCS.

Private vs non-private​

SettingService Account JSONWhen to use
Private (on, default)RequiredConnect to a private bucket using an explicit service account JSON key
Non-private (off)Hidden — not requiredConnect using the environment's default Google Cloud credentials (for example, workload identity or application default credentials)
note

When Private is off, Bicycle does not send a service account JSON key. The connector service uses default Google Cloud credentials available in the runtime environment to access the bucket.

Bucket NameRequired

Name of the GCS bucket

PathRequired

Folder or prefix within the bucket to read from

File PatternRequired

Pattern used to match files (for example, .*\.csv$)

Service Account JSON

Paste the full contents of the downloaded service account JSON key

Private mode (default) — the Service Account JSON field is shown:

Private mode enabled with Service Account JSON field

Non-private mode — turn Private off. Only bucket, path, and file pattern are required:

Non-private mode with Private toggle off

The form also shows the required GCS permissions when using a service account:

  • Storage Object Viewer — read access
  • Storage Object Creator — write access, if needed

Example values:

  • Bucket Name: Mybucket1
  • Path: path/data
  • File Pattern: .*\.csv$
  • Service Account JSON (Private mode): paste the full JSON key contents

Filled GCS connection form

Click Add connection when finished.

Confirm the connection

When the connection is created successfully, you will see a confirmation with the connection name, ID, and method (Service Account). Click Close to return to the Connectors page.

Connection successfully added

Supported file types​

CSV, JSON, Parquet, and other delimited formats.

References​