Skip to main content

Create a Google-integrated custom cloud

Create a custom cloud for BYOC in your Aiven organization to better address your specific business needs or project requirements.

To configure a custom cloud in your Aiven organization and prepare your Google Cloud account so that Aiven can access it:

  1. In the Aiven Console or with the Aiven CLI client, you specify new cloud details to generate a Terraform infrastructure-as-code template.

  2. You download the generated template and deploy it in your Google Cloud account to acquire a privilege-bearing service account, which Aiven needs for accessing your Google Cloud account only with permissions that are required.

    note

    Privilege-bearing service account is an identifier of the service account created when running the infrastructure template in your Google Cloud account. Aiven impersonates this service account and runs operations, such as creating VMs for service nodes, in your BYOC account.

  3. You deploy your custom cloud resources supplying the generated privilege-bearing service account to the Aiven platform, which gives Aiven the permissions to securely access your Google Cloud account, create resources, and manage them onward.

  4. You select Aiven projects that can use your new custom clouds for creating services.

  5. You add contact details for individuals from your organization that Aiven can reach out to in case of technical issues with the new cloud.

Before you start​

Prerequisites​

  • You have enabled the BYOC feature.
  • You have an active account with your cloud provider.
  • Depending on the tool to use for creating a custom cloud:
    • Console: Access to the Aiven Console or
    • CLI:
      • Aiven CLI client installed
      • Aiven organization ID from the output of the avn organization list command or from the Aiven Console > User information > Organizations.
  • You have the organization admin role in your Aiven organization.
  • You have Terraform installed.
  • You have required IAM permissions.
  • Your Google Cloud project is not under an organization policy that:
    • Enforces the constraints/compute.requireShieldedVm constraint. For more information, see Shielded VMs.
    • Restricts the constraints/compute.vmExternalIpAccess constraint. Aiven requires the ability to create instances with external IP addresses. An organization policy that only allows external IPs for specific listed instances is not compatible with BYOC.

IAM permissions​

You need cloud account credentials set up on your machine so that your user or role has required Terraform permissions to integrate with your cloud provider.

Show permissions needed by your service account that will run the Terraform script in your Google project

  • roles/iam.serviceAccountAdmin (sets up impersonation to the privilege-bearing service account)
  • roles/resourcemanager.projectIamAdmin (provides permissions to the privilege-bearing service account to use your project)
  • roles/compute.instanceAdmin.v1 (manages networks and instances)
  • roles/compute.securityAdmin (creates firewall rules)
  • Enable Identity and Access Management (IAM) API to create the privilege-bearing service account
  • Enable Cloud Resource Manager (CRM) API to set IAM policies to the privilege-bearing service account
  • Enable Compute Engine API.

For more information on Google Cloud roles, see IAM basic and predefined roles reference in the Google Cloud documentation.

Create a custom cloud​

Create a custom cloud either in the Aiven Console or with the Aiven CLI.

Launch the BYOC setup​

  1. Log in to the Aiven Console, and go to an organization.
  2. Click Admin in the top navigation, and click Bring your own cloud in the sidebar.
  3. In the Bring your own cloud view, select Create custom cloud.

Generate an infrastructure template​

In this step, an IaC template is generated in the Terraform format. In the next step, you'll deploy this template in your Google Cloud account to acquire a privilege-bearing service account (SA), which Aiven needs for accessing your Google Cloud account.

In the Create custom cloud wizard:

  1. Specify cloud details:

  2. Click Next.

  3. Specify deployment and storage details:

    • Deployment model

      Choose between:

      • Private model, which routes traffic through a proxy for additional security utilizing a bastion host logically separated from the Aiven services.
      • Public model, which allows the Aiven control plane to connect to the service nodes via the public internet.
      note

      The service endpoint has two hostnames. The public hostname is derived from the private hostname by adding a public- prefix to it. The Service URI shown by the Aiven Console displays only the private hostname.

    • CIDR for BYOC resources

      The CIDR block defines the IP address range of the VPC that Aiven creates in your own cloud account. Any Aiven service created in the custom cloud will be placed in the VPC and will get an IP address within this address range.

      In the CIDR field, specify an IP address range for the BYOC VPC using a CIDR block notation, for example: 10.0.0.0/16, 172.31.0.0/16, or 192.168.0.0/20.

      Make sure that an IP address range you use meets the following requirements:

      • IP address range is within the private IP address ranges allowed in RFC 1918.

      • CIDR block size is between /16 (65536 IP addresses) and /24 (256 IP addresses).

      • CIDR block is large enough to host the desired number of services after splitting it into per-availability-zone subnets.

        For example, the smallest /24 CIDR block might be enough for a few services but can pose challenges during node replacements or maintenance upgrades if running low on available free IP addresses.

      • CIDR block of your BYOC VCP doesn't overlap with the CIDR blocks of VPCs you plan to peer your BYOC VPC with. You cannot change the BYOC VPC CIDR block after your custom cloud is created.

  4. Click Generate template.

Your infrastructure Terraform template gets generated based on your inputs. You can view, copy, or download it. Now, you can use the template to acquire a privilege-bearing service account.

Deploy the template​

Use the generated Terraform template to create a privilege-bearing service account by deploying the template in your Google Cloud account.

  1. Copy or download the template and the variables file from the Create custom cloud wizard.

  2. Optionally, modify the template as needed.

    note

    To connect to a custom-cloud service from different security groups (other than the one dedicated for the custom cloud) or from IP address ranges, add specific ingress rules before you apply the Terraform infrastructure template in your Google Cloud account in the process of creating a custom cloud resources.

    Before adding ingress rules, see the examples provided in the Terraform template you generated and downloaded from Aiven Console.

  3. Set up Terraform to authenticate with Google Cloud.

    Configure your Google Cloud credentials using one of the following methods:

    • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable:

      export GOOGLE_APPLICATION_CREDENTIALS="/path/to/your/service-account-key.json"

    • Use gcloud CLI to set application default credentials:

      gcloud auth application-default login

    • Set the service account key in the Terraform provider configuration.

    For more information, see the Google Provider authentication documentation.

  4. Deploy the infrastructure template using Terraform:

    terraform init
    terraform plan -var-file=FILE_NAME.tfvars
    terraform apply -var-file=FILE_NAME.tfvars

    Replace FILE_NAME.tfvars with the name of the variables file you downloaded.

    important

    The -var-file option is required to pass the configuration variables to Terraform.

  5. Find the privilege-bearing service account in the Terraform output after running terraform apply.

  6. Supply the privilege-bearing service account into the Create custom cloud wizard.

  7. Click Next to proceed or park your cloud setup and save your current configuration as a draft by selecting Save draft. You can resume creating your cloud later.

Set up your custom cloud's availability​

Select projects where the new custom cloud will be available for hosting your services. These projects will support creating new services in the custom cloud or migrating your existing services to the custom cloud if your service and networking configuration allows it. For more information on migrating your existing services to the custom cloud, contact your account team.

Your cloud can be available in:

  • All the projects in your organization
  • Selected organizational units
  • Specific projects only

To set up your cloud's availability in the Create custom cloud wizard > the Assign BYOC to projects section, select one of the two following options:

  • By default for all projects to make your custom cloud available in all existing and future projects in the organization
  • By selection to pick specific projects or organizational units where you want your custom cloud to be available.
note

By selecting an organizational unit, you make your custom cloud available from all the projects in this unit.

Add customer contacts​

Select at least one person whom Aiven can contact in case of any technical issues with your custom cloud.

note

Admin is a mandatory role, which is required as a primary support contact.

In the Create custom cloud wizard > the Customer contacts section:

  1. Select a contact person's role using the Job title menu, and provide their email address in the Email field.
  2. Use + Add another contact to add as many customer contacts as needed for your custom cloud.
  3. Click Save and validate.

The custom cloud process has been initiated.

Complete the cloud setup​

Select Done to close the Create custom cloud wizard.

The deployment of your new custom cloud might take a few minutes. As soon as it's over, and your custom cloud is ready to use, you'll be able to see it in the list of your custom clouds in the Bring your own cloud view.

note

Your new custom cloud is ready to use only after its status changes to Active.

Related pages