As amazing as Pivotal Cloud Foundry (PCF) is, creating your own PCF setup is a rather challenging task. Speaking from experience, it took me many many days of countless attempts, and talking with support to get a setup up and running.
This tutorial is a guide to getting your own PCF setup up and running on AWS, while demonstrating the dos and don’ts of the setup process. We will be deploying PCF on AWS using Terraform, as opposed to doing this manually. This is mainly because the deployment process is much simpler and quicker. Additionally, there’s very little room for human error this way.
However, if you prefer to do a manual deployment, Pivotal documentation provides a great description on how this can be done.
Deploying PCF on AWS using Terraform
We will be using a Terraform template for our deployment. The Terraform template for PCF on AWS describes a set of AWS resources and properties. Before we begin, please ensure that you have the following prerequisites:
The Terraform CLI
An AWS account with an IAM user with the following permissions:
AmazonEC2FullAccess
AmazonRDSFullAccess
AmazonRoute53FullAccess
AmazonS3FullAccess
AmazonVPCFullAccess
IAMFullAccess
AWSKeyManagementServicePowerUser
Before executing Terraform commands, we must first download the AWS Terraform template and create a Terraform Template variables file.
First of all, navigate to the Pivotal Application Service (PAS) release on the Pivotal Network and download the AWS Terraform Templates zip file.
Extract the contents of the zip file and navigate to the terraforming-pas directory that contains the Terraform files for our runtime. Then, create the Terraform variables file terraform.tfvars.
$ cd pivotal-cf-terraforming-aws/terraforming-pas
$ touch terraform.tfvarsOpen the terraform.tfvars file and add the following:
env_name = "YOUR-ENVIRONMENT-NAME"
access_key = "YOUR-ACCESS-KEY"
secret_key = "YOUR-SECRET-KEY"
region = "YOUR-AWS-REGION"
availability_zones = ["YOUR-AZ-1", "YOUR-AZ-2", "YOUR-AZ-3"]
ops_manager_ami = "YOUR-OPS-MAN-IMAGE-AMI"
dns_suffix = "YOUR-DNS-SUFFIX"
ssl_cert = <<SSL_CERT
-----BEGIN CERTIFICATE-----
YOUR-CERTIFICATE
-----END CERTIFICATE-----
SSL_CERT
ssl_private_key = <<SSL_KEY
-----BEGIN EXAMPLE RSA PRIVATE KEY-----
YOUR-PRIVATE-KEY
-----END EXAMPLE RSA PRIVATE KEY-----
SSL_KEYFor the purposes of this tutorial, assume that you already own the domain example.com, our PCF setup will be hosted on pas.example.com. However, you may name the subdomain however as you please.
Edit the values in the file as follows:
In YOUR-ENVIRONMENT-NAME, Terraform prepends the names of the resources it creates with this environment name. For example, if you want the setup to be hosted on pas.example.com, the environment name should be pas.
YOUR-ACCESS-KEY refers to the AWS Access Key ID of the AWS account in which we want Terraform to create its resources.
YOUR-SECRET-KEY refers to the AWS Secret Key of the AWS account in which we want Terraform to create its resources.
YOUR-AWS-REGION is the name of the region we want Terraform to create resources. (Example: us-central1)
YOUR-AZ-1, YOUR-AZ-2, YOUR-AZ-3 are the availability zones from the selected region. (Example: us-central-1a, us-central-1b, us-central-1c).
YOUR-OPS-MAN-IMAGE-AMI refers to the source code for OPS Manager AMI. This can be found in the pdf included with the OPS Manager release on Pivotal Network.
YOUR-DNS-SUFFIX refers to the domain name to use as part of the system domain for our PCF deployment. In our case, it will be example.com. Terraform creates DNS records in AWS using YOUR-ENVIRONMENT-NAME and YOUR-DNS-SUFFIX. In our example, we enter example.com for your DNS suffix and have pas as your environment name, Terraform will create DNS records at pas.example.com.
YOUR-CERTIFICATE is the certificate used for HTTP load balancing. Pivotal recommends using certificates from a Certificate Authority (CA) for production environments, However, self-signed certificates can be used for test environments.
Our certificate must specify the system domain as the common name (CN).
SYSTEM-DOMAIN = YOUR-ENVIRONMENT-NAME.YOUR-DNS-SUFFIX (example: SYSTEM-DOMAIN = pas.example.com).
It must also include the following subdomains:
*.sys.SYSTEM-DOMAIN
*.login.sys.SYSTEM-DOMAIN
*.uaa.sys.SYSTEM-DOMAIN
*.apps.SYSTEM-DOMAIN
If you want to know how to create self signed SSL certificates with multiple subdomains, the following blog on “How to create a self-signed SSL certificate for multiple domains” provides a detailed explanation on how this can be done.
YOUR-PRIVATE-KEY refers to the private key for the generated certificate.
Now that we have populated our Terraform variables file, it’s time to create our AWS resources. We must follow these steps using the Terraform CLI to create resources on AWS.
From the directory that contains the terraform.tfvars file, run terraform init to initialize the directory.
Run terraform plan -out=plan to create an execution plan for Terraform. Run terraform apply plan to apply the execution plan. This may take several minutes as Terraform will create all resources in AWS.
$ terraform init
$ terraform plan -out=plan
$ terraform apply planFinally, we must create a DNS record with the DNS provider. This is done by creating a new NS record for our PCF system domain. In this record, enter the name servers included in env_dns_zone_name_servers from our Terraform output.
Our PCF setup has now been deployed! The next step is to configure our BOSH Director.
Configuring the BOSH Director
The first thing we must to is to access the Ops Manager. In a web browser, navigate to the fully qualified domain name (FQDN) of the BOSH Director. Enter the ops_manager_dns from running terraform output into the URL.
When Ops Manager starts for the first time, one of the following forms of authentication must be selected.
Internal Authentication: If you use Internal Authentication, PCF maintains your user database.
SAML Identity Provider: If you use a SAML Identity Provider (IdP), an external identity server maintains your user database.
LDAP Server: If you use a LDAP Server, an external identity server maintains your user database.
Internal Authentication would be the simplest form of authentication, where we would simply provide a username and password. Select your preferred method of authentication and access Ops Manager. In Ops Manager, click the BOSH Director for AWS tile.
AWS Config Page
Select AWS Config in the console config pane on the left.
You may either select Use AWS Keys or Use AWS Instance Profile. If you select Use AWS Keys, complete the following fields.
Access Key ID: Enter the value of ops_manager_iam_user_access_key from the Terraform output.
AWS Secret Key: Enter the value of ops_manager_iam_user_secret_key from the Terraform output.
If you select AWS Instance Profile:
AWS IAM Instance Profile: name of your AWS IAM profile
Security Group ID: value of vms_security_group_id from the Terraform output.
SSH Private Key: value of ops_manager_ssh_private_key. This value may be displayed as sensitive when you run terraform apply. In that case, run terraform output to view the value of ops_manager_ssh_private_key.
Region: AWS region where OPS Manager was deployed.
Encrypt Linux EBS Volumes: enable full encryption on persistent disks of all BOSH-deployed virtual machines (VMs), except for the Ops Manager VM and Director VM.
Click Save.
Director Config Page
In the Director Config page, enter at least two of the following NTP Servers in the NTP Servers (comma delimited) field, separated by a comma:
0.amazon.pool.ntp.org,1.amazon.pool.ntp.org,2.amazon.pool.ntp.org,3.amazon.pool.ntp.orgLeave the rest of the fields at their default values, and click Save.
Create Availability Zones Page
Click Add three times, and enter the values corresponding to the key in management_subnet_availability_zones from the Terraform output, and click Save.
Create Networks Page
Our PCF setup requires three networks, and in each network, a set of three subnets for each availability zone. To create a new network, click Add network, and follow these instructions.
Infrastructure Network
For Name, enter MY-ENVIRONMENT-infrastructure-network where MY-ENVIRONMENT is an identifier for our environment (example: pas-infrastructure-network).
First Subnet
VPC Subnet ID: The first value of infrastructure_subnet_ids from the Terraform output.
CIDR: 10.0.16.0/28
Reserved IP Ranges: 10.0.16.0-10.0.16.4
DNS: 10.0.0.2
Gateway: 10.0.16.1
Availability Zones: The first value of infrastructure_subnet_availability_zones from the Terraform output.
Second Subnet
VPC Subnet ID: The second value of infrastructure_subnet_ids from the Terraform output. CIDR: 10.0.16.16/288
Reserved IP Ranges: 10.0.16.16-10.0.16.20
DNS: 10.0.0.2
Gateway: 10.0.16.17
Availability Zones: The second value of infrastructure_subnet_availability_zones from the Terraform output.
Third Subnet
VPC Subnet ID: The third value of infrastructure_subnet_ids from the Terraform output.
CIDR: 10.0.16.32/28
Reserved IP Ranges: 10.0.16.32-10.0.16.36
DNS: 10.0.0.2
Gateway: 10.0.16.33
Availability Zones: The third value of infrastructure_subnet_availability_zones from the Terraform output.
PAS Network
For Name, enter MY-ENVIRONMENT-pas-network where MY-ENVIRONMENT is an identifier for our environment (example: pas-pas-network).
First Subnet
VPC Subnet ID: The first value of pas_subnet_ids from the Terraform output.
CIDR: 10.0.4.0/24
Reserved IP Ranges: 10.0.4.0-10.0.4.4
DNS: 10.0.0.2
Gateway: 10.0.4.1
Availability Zones: The first value of pas_subnet_availability_zones from the Terraform output.
Second Subnet
VPC Subnet ID: The second value of pas_subnet_ids from the Terraform output.
CIDR: 10.0.5.0/24
Reserved IP Ranges: 10.0.5.0-10.0.5.4
DNS: 10.0.0.2
Gateway: 10.0.5.1
Availability Zones: The second value of pas_subnet_availability_zones from the Terraform output.
Third Subnet
VPC Subnet ID: The third value of pas_subnet_ids from the Terraform output.
CIDR: 10.0.6.0/24
Reserved IP Ranges: 10.0.6.0-10.0.6.4
DNS: 10.0.0.2
Gateway: 10.0.6.1
Availability Zones: The third value of pas_subnet_availability_zones from the Terraform output.
Services Network
For Name, enter MY-ENVIRONMENT-services-network where MY-ENVIRONMENT is an identifier for our environment (example: pas-services-network).
First Subnet
VPC Subnet ID: The first value of services_subnet_ids from the Terraform output.
CIDR: 10.0.8.0/24
Reserved IP Ranges: 10.0.8.0-10.0.8.3
DNS: 10.0.0.2
Gateway: 10.0.8.1
Availability Zones: The first value of services_subnet_availability_zones from the Terraform output.
Second Subnet
VPC Subnet ID: The second value of services_subnet_ids from the Terraform output.
CIDR: 10.0.9.0/24
Reserved IP Ranges: 10.0.9.0-10.0.9.3
DNS: 10.0.0.2
Gateway: 10.0.9.1
Availability Zones: The second value of services_subnet_availability_zones from the Terraform output.
Third Subnet
VPC Subnet ID: The third value of services_subnet_ids from the Terraform output.
CIDR: 10.0.10.0/24
Reserved IP Ranges: 10.0.10.0-10.0.10.3
DNS: 10.0.0.2
Gateway: 10.0.10.1
Availability Zones: The third value of services_subnet_availability_zones from the Terraform output.
Click Save.
Assign AZs and Networks Page
For Singleton Availability Zone, select any availability zone of your choosing. The BOSH Director gets installed in this Availability Zone.
Use the dropdown in Network to select MY-ENVIRONMENT-infrastructure-network for our BOSH Director. Then, click Save.
Security
In Trusted Certificates, we can enter our custom CA certificates into our organization’s trust chain. This feature enables all BOSH-deployed components in our deployment to trust custom root certificates.
The other options can be kept at their default values, and click Save.
The options in the rest of the pages can be kept at their default values.
Complete the BOSH Director Installation
Click Installation Dashboard and return to the installation Dashboard. Thereafter, click Review Pending Changes, and then Apply Changes.
The installation of the BOSH Director may take a few moments. When the installation process finally completes, a Changes Applied message appears.
Our BOSH Director has been successfully installed! The next, and final step is to deploy Pivotal Application Services (PAS) in our setup.
Deploying Pivotal Application Services (PAS)
This final part describes how to install and configure PAS on AWS.
In your browser, login and navigate to PAS in Pivotal Network. Click and download the Pivotal Application Service .pivotal file. You may also download the Small Footprint PAS to download the small footprint runtime instead. The small footprint runtime requires less resources, and therefore, the operational cost of the setup on AWS would be lower. For more information on Small Footprint PAS, see Getting Started with Small Footprint Runtime.
This is a large file. Therefore, the download may take a long time. After the download is complete, navigate to the PCF Ops Manager Installation Dashboard and click Import a Product and add the downloaded PAS tile to Ops Manager. This also may take a long amount of time, as this is a large file.
Once imported, click the PAS tile in the installation Dashboard.
Assign Availability Zones and Networks
Select the Assign AZ and Networks page and select a preferred Availability Zone under Place Singleton Jobs. Ops Manager runs any job with a single instance in this Availability Zone.
Select all three availability zones under Balance other jobs. Ops Manager balances instances of jobs with more than one instance across the Availability Zones that you specify.
From the Network drop down, select MY-ENVIRONMENT-pas-network created when configuring the BOSH Director. Then, click Save.
Domains
Select Domains, and for System Domain, enter the value of sys_domain from the Terraform output. For Apps Domain, provide the value of apps_domain from the Terraform output. Finally, click Save.
Networking
In Networking, under Certificates and Private Key for HAProxy and Router, we must provide at least one Certificate and Private Key. Click the Add button to add a name, and a certificate pair.
You may either provide the same pair generated while creating the Terraform variables, or click on the Generate RSA Certificate link to generate your own self signed certificate in Ops Manager.
Leave the rest of the inputs at their default value, and click Save.
Application Security Groups
Type X in the box to acknowledge that once the Pivotal Application Service (PAS) deployment completes, you will review and set the appropriate application security groups. See Restricting App Access to Internal PCF Components for instructions.
UAA
Select UAA. Under Choose the location of your UAA database, select PAS Database.
Under SAML Service Provider Credentials, enter a certificate and a private key to be used by UAA as a SAML Service Provider for signing outgoing SAML authentication requests. The following domain must be associated with the certificate:
*.login.YOUR-SYSTEM-DOMAINClick Save.
CredHub
Select CredHub, and under Encryption Keys, specify one or more keys to use for encrypting and decrypting the values stored in the CredHub database.
Name: This is the name of the encryption key.
Provider: Select Internal.
Key: This key is used for encrypting all data. The key must be at least 20 characters long.
Primary: This checkbox is used for marking the key you specified above as the primary encryption key. Select this checkbox.
Click Save.
Internal MySQL
Select Internal MySQL, and provide test@example.com as the email address. The email address provided does not matter in our case.
Configure Router (or HAProxy) to Elastic Load Balancer
Click Resource Config
Provide the name of our SSH load balancer depending on the release we are using.
Pivotal Application Service (PAS): In the ELB Name field of the Diego Brain row, enter the value of ssh_elb_name from the Terraform output.
Small Footprint Runtime: In the ELB Name field of the Control row, enter the value of ssh_elb_name from the Terraform output.
In the ELB Name field of the Router row, enter the value of web_elb_name from the Terraform output.
In the ELB Name field of the TCP Router row, enter the value of tcp_elb_name from the Terraform output.
Click Save.
Download Stemcell
This step is only required if your Ops Manager does not already have the stemcell version required by PAS.
Open the Stemcell product page in the Pivotal Network and download the appropriate stemcell version. Then, navigate to Stemcell Library and click Import Stemcell to import the downloaded stemcell .tgz file.
When prompted, enable the PAS checkbox to stage your stemcell, and click Apply Stemcell to Products.
Complete the PAS Installation
Click the Installation Dashboard, and select Review Pending Changes. The installation process generally requires a minimum of 90 minutes to complete. The image shows the Changes Applied window that displays when the installation process successfully completes.
Congratulations! You have successfully created your own PCF setup on AWS! You are now ready to develop your own tiles, and use PCF as you wish!
If you wish to learn more about setting up PCF on AWS using Terraform, you may read the Pivotal Documentation.
Comments