Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Classified Provisioning Docs #101

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/classified-provisioning-docs/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Classified Provisioning",
"position": 6,
"link": {
"type": "generated-index",
"description": "Native Rancher Provisioning on AWS C2S/SC2S Environments"
}
}
75 changes: 75 additions & 0 deletions docs/classified-provisioning-docs/installation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Installation

## Bundle Artifacts in Connected Environment

In a connected environment, utilize the `hauler` CLI to verify and collect the Classified Provisioning images from the Carbide registry.

1. Download the Carbide public key.

```bash
# download the public key for carbide
curl -sfL https://raw.githubusercontent.com/rancherfederal/carbide-releases/main/carbide-key.pub -o /tmp/carbide-key.pub
```

1. Create the Hauler manifest file.

```yaml
cat <<EOF > /tmp/carbide-provisioning-manifest.yaml
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Images
metadata:
name: carbide-provisioning-images
annotations:
hauler.dev/key: /tmp/carbide-key.pub
# hauler.dev/platform: linux/amd64 # only fetch specific platform/architecture
spec:
images:
- name: rgcrprod.azurecr.us/rancher/machine:v0.15.0-rancher118-carbide-1
- name: rgcrprod.azurecr.us/rancher/rancher:v2.9.3-carbide-1
- name: rgcrprod.azurecr.us/rancher/rancher-agent:v2.9.3-carbide-1
---
apiVersion: content.hauler.cattle.io/v1alpha1
kind: Charts
metadata:
name: carbide-provisioning-charts
spec:
charts:
- name: rancher
repoURL: https://rancherfederal.github.io/carbide-charts
version: 2.9.3
EOF
```

2. Login into the Carbide registry with your Carbide Credentials, and then validate and fetch the images to the local `hauler` store.

```bash
hauler login rgcrprod.azurecr.us -u <redacted> -p <redacted>
hauler store sync --files /tmp/carbide-provisioning-manifest.yaml
hauler store save --filename carbide-provisioning-images.tar.zst
```

3. Move the resulting `carbide-provisioning-images.tar.zst` file into your airgapped/classified environment.

## Copy Files to Your Classified Registry

1. Copy the `carbide-provisioning-images.tar.zst` and the `hauler` binary to a server in your airgapped/classified environment. Please ensure `hauler` is added to your PATH.

2. Load the bundle to the local store and copy the images to your registry.

```bash
hauler login <registry-url> -u <redacted> -p <redacted>
hauler store load carbide-provisioning-images.tar.zst
hauler store copy registry://<registry-url>
```

## Update Your Rancher Installation

1. Extract the Carbide Rancher chart from the local `hauler` store and upgrade your existing Rancher installation with the new rancher image tag...

```bash
hauler store extract hauler/rancher:2.9.3
helm upgrade -i rancher rancher-2.9.3.tgz -n cattle-system --reuse-values --set rancherImageTag=v2.9.3-carbide-1
```

For more information about airgapped installation of Rancher, see the docs [here](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/airgapped-helm-cli-install).

52 changes: 52 additions & 0 deletions docs/classified-provisioning-docs/introduction.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Introduction

**Rancher** has had the ability and functionality to limitedly provision and manage clusters in the Classified Cloud Regions, but it has not been validated, documented, supported, or treated as a first class citizen.

**Rancher Government** has invested the time and engineering resources to provide full enterprise support for the native deployment, provisioning, and management of clusters on the Classified Cloud Regions.

### What Does that Mean for Customers Today?

**Rancher Government Carbide** enables supported customers to use the Rancher Manager User Interface (UI) or through a Helm Chart and/or Fleet (GitOps) to deploy, provision, and manage RKE2 Clusters and K3S Clusters on AWS EC2 Instances, in the Classified Cloud Regions such as AWS C2S/SC2S.

Additionally, **Rancher Government** actively tests and validates this configuration through regions in [AWS GovCloud](https://aws.amazon.com/govcloud-us/) and through regions in [Sequoia Combine](https://www.sequoiainc.com/combine) (Classified Cloud Region Emulator).

**Classified Provisioning** provides the ability to natively provision clusters through Rancher onto the same AWS account that Rancher is running in through the use of IAM roles attached to the instances running Rancher. This greatly simplifies Day 2 Operations and enables a seamless experience in any environment.

## Challenges

At a high level, the Classified Cloud Regions, such as AWS, introduce significant complexities and challenges for engineering solutions to deploy, provision, manage, and maintain infrastructure.

- **Temporary Credentials:** Short Lived Credentials, typically valid for less than an hour (STS Tokens)
Additional requirements such as providing CA Bundles, HTTP Proxies, etc…
- **Controlled Security Access:** Limited IAM Roles/Policies/Permissions and IAM Permission Boundaries
- **Limited Service Availability:** Subset of AWS Services are available and have limited features
- **Unique Regions and Service Endpoints:** Regions and Service Endpoints are unique per environment
- **Separation of Duties:** Unknown requirements and each customer imposes various changes to the above list of challenges and require human in the loop procedures and processes

## Future State

**Rancher Government** is working to execute a strategy to bring parity to the provisioning experience in the Classified Cloud Regions in comparison to the mainstream commercial clouds. It is imperative that Rancher provides a homogeneous user experience regardless of region or classification level. In order to achieve this, we plan to augment the current state...

- **Principle of Least Privileged Access:** Pod Level Permissions vs Node Level Permissions
- **Reducing Implementation/Migration Burden:** Utilizing Rancher Manager UI Extensions (i.e STIGATRON)
- **Additional Functionality:** Carbide Provisioning for all supported drivers (i.e. AWS EKS)

## Comparing Imported & Native-Provisioned Clusters

| Feature/Functionality | Imported Cluster | Native Provisioned Cluster |
|-|-|-|
| Cluster Access with `kubectl` or `kubeconfig` | &#x2713; | &#x2713; |
| Cluster Access Management (RBAC) | &#x2713; | &#x2713; |
| Managing Projects, Namespaces, & Workloads | &#x2713; | &#x2713; |
| Managing Persistent Volumes & Storage Classes | &#x2713; | &#x2713; |
| Use of Rancher App Catalog(s) | &#x2713; | &#x2713; |
| Configuring Infrastructure Tools (Monitoring, Logging, Istio, etc.) | &#x2713; | &#x2713; |
| Managing & Running Security Scans | &#x2713; | &#x2713; |
| Modifying Cluster Configuration | | &#x2713; |
| Automated Cluster Kubernetes Version Upgrades | | &#x2713; |
| Cluster Node Management (Adding/Removing/Scaling) | | &#x2713; |
| Cluster Node Access via Shell (SSH) | | &#x2713; |
| Automated Cluster Certificate Rotation | | &#x2713; |
| Automated Cluster Encryption Key Rotation | | &#x2713; |
| Ability to Snapshot, Backup, & Restore | | &#x2713; |
| Configuring PSS/PSA/PSP | | &#x2713; |
161 changes: 161 additions & 0 deletions docs/classified-provisioning-docs/prereqs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Prerequisites

In order to utilize Classified Provisioning in SC2S/C2S, you will need the following prerequisites.

## Tools

The following tools are required to package Classified Provisioning artifacts and install/upgrade Rancher.

- [Hauler](https://rancherfederal.github.io/hauler-docs/docs/introduction/install)
- [Helm](https://helm.sh/docs/intro/install)
- [Rancher](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade) (v2.7.13, v2.7.14, v2.8.4 or greater)
dweomer marked this conversation as resolved.
Show resolved Hide resolved

## IAM Role & Instance Profile

The Amazon EC2 provisioner requires a certain amount of access to read, write, and delete some AWS resources. Enabling this sans principal credentials (i.e. access key and secret key) requires an [EC2 Instance Profile](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#ec2-instance-profile) associated with an IAM Role, for each node in the Rancher Management cluster, with the following policy:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "RancherAWSEC2Permissions",
"Effect": "Allow",
"Action": [
"ec2:AuthorizeSecurityGroupIngress",
"ec2:Describe*",
"ec2:ImportKeyPair",
"ec2:CreateKeyPair",
"ec2:CreateSecurityGroup",
"ec2:CreateTags",
"ec2:DeleteKeyPair",
"ec2:ModifyInstanceMetadataOptions"
],
"Resource": "*"
},
{
"Sid": "RancherAWSKMSPermissions",
"Effect": "Allow",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKeyWithoutPlaintext",
"kms:Encrypt",
"kms:DescribeKey",
"kms:CreateGrant",
"ec2:DetachVolume",
"ec2:AttachVolume",
"ec2:DeleteSnapshot",
"ec2:DeleteTags",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteVolume",
"ec2:CreateSnapshot"
],
"Resource": [
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:volume/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:instance/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER':AWS_ACCOUNT_ID_PLACEHOLDER:snapshot/*",
"arn:aws:kms:AWS_REGION_PLACEHOLDER':AWS_ACCOUNT_ID_PLACEHOLDER:key/*"
zackbradys marked this conversation as resolved.
Show resolved Hide resolved
]
},
{
"Sid": "RancherAWSPassRolePermissions",
"Effect": "Allow",
"Action": [
"iam:PassRole",
"ec2:RunInstances",
"ec2:DetachVolume",
"ec2:AttachVolume",
"ec2:DeleteSnapshot",
"ec2:DeleteTags",
"ec2:CreateTags",
"ec2:CreateVolume",
"ec2:DeleteVolume",
"ec2:CreateSnapshot"
],
"Resource": [
"arn:aws:ec2:AWS_REGION_PLACEHOLDER::image/ami-*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:instance/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:placement-group/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:volume/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:subnet/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:key-pair/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:network-interface/*",
"arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:security-group/*",
"arn:aws:iam::AWS_ACCOUNT_ID_PLACEHOLDER:role/*"
]
},
{
"Sid": "RancherAWSEC2ScopedPermissions",
"Effect": "Allow",
"Action": [
"ec2:RebootInstances",
"ec2:TerminateInstances",
"ec2:StartInstances",
"ec2:StopInstances"
],
"Resource": "arn:aws:ec2:AWS_REGION_PLACEHOLDER:AWS_ACCOUNT_ID_PLACEHOLDER:instance/*"
}
]
}
```

![Modify IAM Role](/img/classified-provisioning/modify-iam-role.png)

## Rancher Manager Configurations

Rancher Manager needs to be configured with the following values to support full airgapped deployments as well as your custom certificates and certificate authority (CA).

### Helm Values

Configure the follow Helm values for your Rancher helm deployment, adjusting the `rancher-url` and `registry-url` for your specific environment.

```yaml
additionalTrustedCAs: true
hostname: <rancher-url>
ingress:
tls:
source: secret
rancherImage: <registry-url>/rancher/rancher
rancherImageTag: v2.9.3-carbide-1
systemDefaultRegistry: <registry-url>
useBundledSystemChart: true
privateCA: true
```

### Custom CA Certificate Secret

For Rancher to make requests to the Classified AWS endpoints, you need to create a secret containing the AWS CA of your specific environment.

```yaml
apiVersion: v1
kind: Secret
metadata:
name: tls-ca-additional
namespace: cattle-system
data:
ca-additional.pem: <encodedPrivateBundle>
type: Opaque
```
**NOTE**: Ensure this secret matches the name and namespace above.

### Registry configuration

You need to configure your Kubernetes cluster to utilize your private registry. Configure the `registries.yaml` file in your RKE2/K3s configuration directories with the following:

```yaml
mirrors:
"*":
endpoint:
- "https://<registry-url>"

configs:
"<registry-url>":
auth:
username: <redacted>
password: <redacted>
```

For more information on Registry configuration, check out the RKE2 docs [here](https://docs.rke2.io/install/containerd_registry_configuration).

For more detailed information about Air-Gapped Rancher, check out the Rancher docs [here](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/other-installation-methods/air-gapped-helm-cli-install).
42 changes: 42 additions & 0 deletions docs/classified-provisioning-docs/usage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Usage

## Creating a Carbide Cloud Credential

1. In the Rancher Mananger menu, access Cloud Credentials (`Cluster Management` -> `Cloud Credentials`)

2. Select `Create` and choose `Amazon`.

3. Choose a name, click the toggle to enable the `Carbide Instance Credential`, select the default region for your environment, and click `Create`.

![Create Carbide Cloud Credential](/img/classified-provisioning/create-carbide-cloud-creds.png)

## Creating a Classified Cluster

1. In the Rancher Manager menu, access `Cluster Management` and click `Create`. Ensure the `RKE2/K3s` toggle is enabled and select `Amazon EC2`.

2. Select the Cloud Credentials created in the previous step, then add node pools as you normally would for provisioning a cluster from Rancher.


atoy3731 marked this conversation as resolved.
Show resolved Hide resolved
**NOTE**: In SC2S/C2S, it is **required** for you to enter an `AMI ID` and the respective `SSH User` for that AMI ID under the `Advanced` settings.

**NOTE**: In SC2S/C2S, not all Instance Types are available. Ensure you are leverage an Instance Type that is **available in your environment**. This list will be dynamic in the future.

**NOTE**: In SC2S/C2S, **encrypted EBS volumes** are required. Ensure you select the `Encrypt EBS Volumes` and choose a respective KMS key when provisioning.

**NOTE**: If no security group is provided, Rancher will attempt to create a Security Group. If the ability to create Security Groups is limited, have a Security Group created by your Account Manager/Broker that has the [appropriate port configuration](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/installation-requirements/port-requirements#rancher-aws-ec2-security-group) and select that existing Security Group in the `Advanced` settings.

![Create RKE2 Cluster in C2S](/img/classified-provisioning/create-carbide-cloud-creds.png)

For more information on provisioning an RKE2/K3s cluster on Amazon EC2s, see the [Rancher docs](https://ranchermanager.docs.rancher.com/how-to-guides/new-user-guides/launch-kubernetes-with-rancher/use-new-nodes-in-an-infra-provider/create-an-amazon-ec2-cluster).

### Registry Configuration

When using a private registry, downstream clusters must be configured with correct authentication and certificate information to access that registry via the [registries.yaml](https://docs.rke2.io/install/containerd_registry_configuration) file. Rancher provisioned clusters can be configured through the UI to generate.

1. Under the `Cluster Configuration` section when provisioning a cluster, select the `Registries` section.

2. Enter the registry domain in the `Container Registry` box, and either select existing registry credentials or create new credentials for authentication to the registry.

3. If advanced configuration (mirroring, certificates, additional authentication), add those to the advanced settings.

For more information, see the [RKE2 Containerd Registry Configuration](https://docs.rke2.io/install/containerd_registry_configuration) documentation.
11 changes: 11 additions & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,17 @@ const sidebars = {
},
],
},
{
type: 'category',
label: 'Classified Provisioning',
collapsed: false,
items: [
'classified-provisioning-docs/introduction',
'classified-provisioning-docs/prereqs',
'classified-provisioning-docs/installation',
'classified-provisioning-docs/usage',
],
},
{
type: 'category',
label: 'STIGATRON',
Expand Down
5 changes: 5 additions & 0 deletions src/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -74,3 +74,8 @@
font-family: nasalization;
color: red;
}

table td{
text-align: center;
font-weight: bold;
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading