Skip to main content

Set Up GCP Project and VPC for Private Data

Google Cloud Platform (GCP) private data processing involves running an Alteryx Analytics Cloud (AAC) data processing cluster inside of your GCP project and VPC. This combination of your infrastructure, together with Alteryx-managed GCP resources and software, is commonly referred to as a private data processing.

This page focuses on how to set up your GCP project and VPC for a private data processing on AACAAC.

Note

The GCP project and VPC setup require access and permissions to the GCP console. If you don’t have this access, please contact your IT team to complete this step.

Attention

Never delete resources provisioned for Private Data Processing.

Setup Steps

Important

To continue with these steps, you must have the GCP Owner RBAC role assigned to you.

Step 1: Select the GCP Project

Select the project where you’d like to run your private data processing.

To improve performance and reduce egress costs, your Google storage and private data handling GKE cluster should be in the same region that you selected for private data storage. This applies to any data sources that you want to connect to the AACAAC .

The VPC created in the GCP project should be dedicated to AACAAC. You can set up connectivity to private data sources using VPC peering, transit gateways, PrivateLink, or others.

Important

You should only set up 1 private data handling instance per GCP project.

Step 2: Enable Google APIs

To create cloud resources for Private Data Handling, you must enable APIs in the project.

  1. From the GCP console, select APIs & Services.

  2. Select ENABLED APIS AND SERVICES.

  3. Enable these APIs:

    1. Cloud Logging API

    2. Cloud Monitoring API

    3. Compute Engine API

    4. Secret Manager API

    5. Service Networking API

    6. Cloud Asset API

    7. Kubernetes Engine API

    8. Google Cloud Memorystore for Redis API

Step 3: Configure IAM

With your GCP project in place, now set up the service principal and access keys.

Step 3a: Create a Service Account

  1. Create a service account with the name aac-automation-sa.

  2. Generate keys with the key type as JSON.

  3. Store the JSON Blob file.

Note

You'll need the service key JSON Blob file to provision the cloud resources in a later step.

Step 3b: IAM Binding to the Service Account

Assign these roles to the aac-automation-sa service account:

  • Secret Manager Admin: roles/secretmanager.admin

  • Service Account Admin: roles/iam.serviceAccountAdmin

  • Service Account User: roles/iam.serviceAccountUser

  • Project IAM Admin: roles/resourcemanager.projectIamAdmin

  • Service Account key Admin: roles/iam.serviceAccountKeyAdmin

  • Compute Network Viewer: roles/compute.networkViewer

  • Cloud KMS Viewer: roles/cloudkms.viewer

Important

GCP doesn't allow wildcard (*) in the policy document. GCP also has limitations on the number of individual permissions assigned to a custom role. Therefore, you must assign the service account a set of GCP-managed predefined roles.

Step 4: Configure Virtual Private Network

Step 4a: Create a VPC Network

  1. Create a virtual network.

  2. Select Subnet creation mode = Custom.

  3. Disable or delete the default firewall rules.

  4. Select Dynamic routing mode = Global.

  5. The VPC requires 1 subnet. Configure the subnets as shown in this table:

Subnet Name

Subnet Size

Secondary Subnet Name

Secondary Subnet Size

aac-private

10.10.10.0/24

N/A

N/A

Important

The subnet IP addresses and sizes in the table are shown as an example.

Modify values as needed to meet your network architecture. Subnet region must be the region where ‘Private data Handling’ is to be provisioned.

The subnet name MUST match with the name as shown in the table.

Step 4b: Subnet Route Table

Important

You must configure the VPC with a network connection to the internet in your project.

Note

The <gateway id> could be either a NAT gateway or internet gateway, depending on your network architecture.

This is an example subnet route table:

Address Prefix

Next Hop

/24 CIDR Block (aac-private)

aac-vpc

0.0.0.0/0

<gateway_ID>

Step 5: Create Key Ring and Key

Your data source credentials are encrypted using your key and securely stored in a private vault (Private Credential Storage) within your private data plane project. These credentials are only retrieved from the vault when you need them.

Step 5a: Enable Secret Manager Service Agent

  1. Open the Google Cloud Shell by selecting the terminal icon in the Google Cloud Console toolbar.

  2. Execute the command:

    $ gcloud beta services identity create --service "secretmanager.googleapis.com" \
    --project <project id>

  3. The output you get:

    Service identity created: service-<project number>@gcp-sa-secretmanager.iam.gserviceaccount.com

Step 5b: Grant IAM Role to Service Agent

  1. Go to IAM and select Grant Access.

  2. Provide new principal service-<project number>@gcp-sa-secretmanager.iam.gserviceaccount.com.

  3. Provide role Cloud KMS CryptoKey Encrypter/Decrypter.

  4. Select Save.

Step 5c: Create Key Ring and Key

  1. Go to Security > Key Management.

  2. Select Create Key Ring.

  3. Provide key ring name aac-key-ring.

  4. Select the Multi-region region and choose the region from dropdown list.

  5. Select Create.

  6. Provide the key name aac-key.

  7. As Protection Level select Software.

  8. Select Create.

Note

Each Key Ring and Key name must be unique within your project. You need to provide the Key Ring and Key names on the Private Data Processing page.

Step 6: Trigger Private Data Handling Provisioning

Data processing provisioning triggers from the Admin Console inside AACAAC. You need Workspace Admin privileges within a workspace in order to see it.

  1. From the AACAAC landing page, select the circle icon on the top right with your initials in it. Select Admin Console from the menu.

  2. Select Private Data Handling from the left navigation menu.

Attention

Changing or removing any AAC-provisioned public cloud resources after Private Data Handling has been set up can cause inconsistencies. These inconsistencies may lead to errors during job execution or when deprovisioning the Private Data Handling setup.

Make sure that Private Data Storage shows Successfully Configured before you proceed. If the status is Not Configured, go to  GCS as Private Data Storage first, then return to this step.

In the Private Data Processing section, there are 5 fields to fill out. These values come from completing the steps in Setup Steps.

Selecting Create triggers a set of validation checks to verify the GCP project is setup as needed. If permissions are not configured correctly, or the Vnet resources are not created or tagged correctly, you’ll receive an error message with a description that should point you in the right direction.

Dans cette section​: