Skip to content

Commit e0d516d

Browse files
milldrBenbentwocloudpossebotCopilot
authored
DEV-3535: Argo CD GitHub App Branch Protection Requirements (#810)
Co-authored-by: Benbentwo <14911525+Benbentwo@users.noreply.github.com> Co-authored-by: Cloud Posse Bot (CI/CD) <bot@cloudposse.com> Co-authored-by: milldr <14060048+milldr@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent 94ffab4 commit e0d516d

File tree

2 files changed

+76
-171
lines changed

2 files changed

+76
-171
lines changed

‎docs/layers/software-delivery/eks-argocd/setup.mdx‎

Lines changed: 25 additions & 122 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ All deployment steps below assume that the environment has been successfully set
4343

4444
First vendor all related components for the Argo CD layer:
4545

46-
<AtmosWorkflow workflow="vendor" fileName="quickstart/app/argocd" />
46+
<AtmosWorkflow workflow="vendor" fileName="quickstart/platform/argocd" />
4747
</Step>
4848

4949
<Step>
@@ -67,143 +67,46 @@ All deployment steps below assume that the environment has been successfully set
6767

6868
<Steps>
6969
1. #### Terraform `argocd-repo` Access
70-
The first PAT is used by Terraform with the `argocd-repo` component. This is used to create and manage the Argo CD desired state repositories and (optionally) add deployment ssh keys for those repos.
7170

72-
Terraform will pull that PAT from SSM typically using the `argocd/github` path in the `core-auto` account.
71+
First we will need to apply the Argo CD desired state repositories configuration with Terraform. By default, we use local access to apply the component. This requires an engineer to locally authenticate with GitHub and apply this component locally. Since this component is rarely updated, this can be a reasonable trade-off.
7372

74-
Alternatively, you can enforce local access to apply the component. This would require an engineer to locally authenticate with GitHub and apply this component locally. Since this component is rarely updated, this can be a reasonable trade-off.
73+
2. #### Argo CD Instance
7574

76-
At this time, we do not have GitHub App support for this step and prefer using local developer access.
75+
Next, we need a GitHub App for Terraform and the `eks/argocd` component. This app is used to register the webhook in GitHub for the Argo CD Application created with this given component.
7776

78-
2. #### Terraform `eks/argocd` Access
77+
After creating the GitHub App, add the app's private key to AWS SSM Parameter Store in each account with Argo CD, typically the `plat-dev`, `plat-staging`, and `plat-prod` accounts, and add the App ID to the stack catalog. Detailed instructions linked below.
7978

80-
The next two PATs or GitHub App installations are also used by Terraform now with the `eks/argocd` component. Each of these is used to register the webhook in GitHub for the Argo CD Application created with this given component. One is used for non-prod stages and the other is used for prod stages.
79+
3. #### Argo CD Desired State Repository Access (2)
8180

82-
Terraform will pull that token from SSM typically in `plat-dev`, `plat-staging`, and `plat-prod` accounts.
81+
We will need two more GitHub Apps for accessing the ArgoCD desired state repositories _from GitHub Actions_. GitHub Actions running for an application repositories will build and update application manifests in the Argo CD desired state repositories and therefore will need write access to that respective non-prod or prod repository.
8382

84-
3. #### Argo CD GitHub Notification Access
83+
This GitHub App does not need to be added to the stack catalog or SSM since it will be used by GitHub Actions, not Terraform.
8584

86-
The next PAT or GitHub App token is used by the Argo CD notifications system to update the GitHub commit status on deployments. This is stored in SSM and pulled by the `eks/argocd` component. That component will pass the token to the Argo CD instance in the given EKS cluster. That Argo CD instance uses that token _only when synchronous mode is enabled_.
85+
4. #### Argo CD GitHub Notification Access
8786

88-
4. #### GitHub Release Workflow Access
87+
The last GitHub App is used by the Argo CD notifications system to update the GitHub commit status on deployments. This is stored in SSM and pulled by the `eks/argocd` component. That component will pass the ID and private key to the Argo CD instance in the given EKS cluster. That Argo CD instance uses that app _only when synchronous mode is enabled_.
8988

90-
The final two PATs or GitHub App installations are used in the release engineering workflows: one for non-prod and one for prod. This token should allow write permissions in the given Argo CD desired state repositories.
89+
After creating the GitHub App, add the app's private key to AWS SSM Parameter Store in each account with Argo CD, typically the `plat-dev`, `plat-staging`, and `plat-prod` accounts, and add the App ID to the stack catalog. Detailed instructions linked below.
9190

9291
</Steps>
9392

94-
<Tabs queryString="auth-method">
95-
<TabItem value="github-apps" label="GitHub Apps (Recommended)">
96-
<Steps>
97-
<Step>
98-
## <StepNumber/> Set up GitHub Apps
99-
100-
Follow the instructions in [Argo CD Integrations: How to set up Authorization for Argo CD with GitHub Apps](/layers/software-delivery/eks-argocd/tutorials/github-apps) to create and configure a GitHub App for Argo CD.
101-
</Step>
102-
103-
<Step>
104-
## <StepNumber/> Configure Argo CD Desire State Repositories to Use GitHub Apps
105-
106-
Update your Argo CD desired state repository configuration to use the GitHub App instead of PAT:
107-
108-
```yaml
109-
components:
110-
terraform:
111-
argocd-repo:
112-
vars:
113-
# 1. Use local access to apply this component rather than a PAT
114-
# https://registry.terraform.io/providers/integrations/github/latest/docs#github-cli
115-
use_local_github_credentials: true
116-
117-
# 2. If synchronous mode is enabled, set the notifications to send to "github" and not to the "webhook"
118-
github_notifications:
119-
- "notifications.argoproj.io/subscribe.on-deploy-started.github: \"\""
120-
- "notifications.argoproj.io/subscribe.on-deploy-succeeded.github: \"\""
121-
- "notifications.argoproj.io/subscribe.on-deploy-failed.github: \"\""
122-
123-
# 3. Optional, disable the SSH deploy keys to use a GitHub App
124-
# for the Argo CD instance to authenticate with the desired state repository
125-
deploy_keys_enabled: false
126-
```
127-
128-
</Step>
129-
130-
<Step>
131-
## <StepNumber/> Configure Argo CD to Use GitHub Apps
132-
133-
Update your Argo CD configuration to use the GitHub App instead of PATs:
134-
135-
```yaml
136-
components:
137-
terraform:
138-
eks/argocd:
139-
vars:
140-
# GitHub App (Argo CD Instance)
141-
# This GitHub App is used for the Argo CD instance to manage webhooks and read from the desired state repository.
142-
# ie https://github.com/acme/argocd-deploy-non-prod
143-
github_app_enabled: true
144-
github_app_id: "1234567"
145-
github_app_installation_id: "44444444"
146-
# The SSM parameter must exist in the account and region where Argo CD is deployed.
147-
ssm_github_app_private_key: "/argocd/argo_cd_instance/app_private_key"
148-
# Optional, disable the SSH deploy keys to use this GitHub App
149-
# for the Argo CD instance to authenticate with the desired state repository
150-
github_deploy_keys_enabled: false
151-
152-
# GitHub App (Argo CD Notifications)
153-
# This GitHub App is used for the Argo CD instance to send commit status updates back to each ap repository.
154-
# This is only required if synchronous mode is enabled.
155-
# ie https://github.com/acme/example-app-on-eks
156-
github_notifications_app_enabled: true
157-
github_notifications_app_id: "8901235"
158-
github_notifications_app_installation_id: "55555555"
159-
# The SSM parameter must exist in the account and region where Argo CD is deployed.
160-
ssm_github_notifications_app_private_key: "/argocd/argo_cd_notifications/app_private_key"
161-
```
162-
</Step>
163-
164-
</Steps>
165-
</TabItem>
166-
<TabItem value="pats" label="Personal Access Tokens (Legacy)">
167-
<Steps>
168-
<Step>
169-
## <StepNumber/> Establish a Bot User
170-
171-
We deploy a number of GitHub Personal Access Tokens (PATs) as part of the EKS with Argo CD application. By default each
172-
PAT is given the least-access required for the given job.
173-
174-
Each one of these PATs will be associated with a given user. We recommend creating or using an existing "bot" user. For
175-
example, at Cloud Posse we have the "cloudpossebot" GitHub user. This user has its own email address and GitHub account,
176-
is accessible from our internal 1Password vault for all privileged users, and has all access keys and tokens stored with
177-
it in 1Password.
178-
179-
This bot user will need permission to manage a few repositories in your Organization. If you wish to simplify
180-
deployment, you can grant this user permission to create repositories. See (Create Permission for the Bot User)[#Create
181-
Permission for the Bot User].
182-
183-
Use this bot user for all access tokens in the remainder of this guide.
184-
</Step>
185-
186-
<Step>
187-
## <StepNumber/> Create all Argo CD PATs
188-
189-
We will need several PATs for the following steps. Now that the Argo CD repos are created, you can create all required
190-
PATs.
191-
192-
Please see
193-
[Argo CD Integrations: How to set up Authorization for Argo CD with GitHub PATs](/layers/software-delivery/eks-argocd/tutorials/pats)
194-
and follow all steps.
195-
</Step>
196-
</Steps>
197-
</TabItem>
198-
</Tabs>
93+
Follow the instructions in [Argo CD Integrations: How to set up Authorization for Argo CD with GitHub Apps](/layers/software-delivery/eks-argocd/tutorials/github-apps) to create and configure all GitHub Apps for Argo CD. Once completed, you should have 4 GitHub Apps:
94+
95+
<TaskList>
96+
- `Argo CD Instance`
97+
- `Argo CD Deploy Non-Prod`
98+
- `Argo CD Deploy Prod`
99+
- `Argo CD Notifications`
100+
</TaskList>
101+
199102
</Step>
200103

201104
<Step>
202-
## <StepNumber/> Deploy Argo CD repo configuration
105+
## <StepNumber/> Deploy the Argo CD Desired State Repositories
203106

204-
Deploy the Argo CD configuration for the two GitHub repos with the following workflow:
107+
Deploy the Argo CD configuration for the two Argo CD desired state GitHub repositories with the following workflow:
205108

206-
<AtmosWorkflow workflow="deploy/argocd-repos" fileName="quickstart/app/argocd" />
109+
<AtmosWorkflow workflow="deploy/argocd-repos" fileName="quickstart/platform/argocd" />
207110

208111
Once this finishes, review the two repos in your GitHub Organization. These should both be fully configured at this point.
209112

@@ -220,14 +123,14 @@ All deployment steps below assume that the environment has been successfully set
220123
</Step>
221124

222125
<Step>
223-
## <StepNumber/> Deploy Argo CD to each EKS Cluster
126+
## <StepNumber/> Deploy the Argo CD Instances to each EKS Cluster
224127

225128
Once the GitHub repositories are in place and the SAML applications have been created and configuration uploaded to SSM,
226129
we're ready to deploy Argo CD to each cluster.
227130

228131
Deploy `eks/argocd` to each cluster with the following workflow:
229132

230-
<AtmosWorkflow workflow="deploy/argocd" fileName="quickstart/app/argocd" />
133+
<AtmosWorkflow workflow="deploy/argocd" fileName="quickstart/platform/argocd" />
231134
</Step>
232135

233136
<Step>

‎docs/layers/software-delivery/eks-argocd/tutorials/github-apps.mdx‎

Lines changed: 51 additions & 49 deletions
Original file line numberDiff line numberDiff line change
@@ -8,14 +8,15 @@ tags:
88
- eks
99
- github-apps
1010
---
11+
import Admonition from '@theme/Admonition'
12+
import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
1113
import Intro from '@site/src/components/Intro';
1214
import KeyPoints from '@site/src/components/KeyPoints';
13-
import Steps from '@site/src/components/Steps'
15+
import Note from '@site/src/components/Note'
1416
import Step from '@site/src/components/Step'
1517
import StepNumber from '@site/src/components/StepNumber'
16-
import Admonition from '@theme/Admonition'
17-
import Note from '@site/src/components/Note'
18-
import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
18+
import Steps from '@site/src/components/Steps'
19+
import TaskList from '@site/src/components/TaskList'
1920

2021
<Intro>
2122
GitHub Apps is now the preferred method for Argo CD integration with GitHub. GitHub Apps provide more granular permissions, better security, and improved audit capabilities compared to Personal Access Tokens (PATs). This guide will walk you through setting up Argo CD with GitHub Apps.
@@ -25,7 +26,7 @@ import AtmosWorkflow from '@site/src/components/AtmosWorkflow';
2526
- GitHub Apps provide more granular permissions than PATs
2627
- GitHub Apps can be installed on specific repositories
2728
- GitHub Apps have built-in rate limiting and audit capabilities
28-
- GitHub Apps can be used for both repository management and notifications
29+
- GitHub Apps need the ability to bypass branch protection rules
2930
</KeyPoints>
3031

3132
## Required GitHub Apps
@@ -70,19 +71,6 @@ Argo CD integration requires multiple GitHub Apps, each with specific permission
7071
- **Repository scope**:
7172
- All of the app repositories
7273

73-
## Compensating Controls
74-
75-
To address security concerns, the following compensating controls should be implemented:
76-
77-
1. **Organization-Level Branch Ruleset**:
78-
- Applied to all app repos but *not* to `acme/argocd-deploy-non-prod` and `acme/argocd-deploy-prod`
79-
- Prevents changes to the main branch with the Argo CD app *not* in the bypass list
80-
- The Argo CD repos are excluded because workflows need to write to the main branch in these repos
81-
82-
2. **Argo CD Desired State Repositories Branch Ruleset**:
83-
- Applied only to the Argo CD desired state repositories
84-
- Prevents writing to the main branch, but with the Argo CD app in the bypass list
85-
8674
## Deployment
8775

8876
<Steps>
@@ -117,10 +105,10 @@ To address security concerns, the following compensating controls should be impl
117105
- `repository.contents`: Read and Write
118106
- `repository.metadata`: Read-Only
119107

120-
#### Argo CD Deploy Prod
121-
- Name: `Argo CD Deploy Prod`
122-
- Homepage URL: Your organization's homepage
123-
- Webhook: Disabled
108+
#### Argo CD Deploy Prod
109+
- Name: `Argo CD Deploy Prod`
110+
- Homepage URL: Your organization's homepage
111+
- Webhook: Disabled
124112
- Permissions:
125113
- Push commits to Desired State Repositories:
126114
- `repository.contents`: Read and Write
@@ -144,13 +132,28 @@ To address security concerns, the following compensating controls should be impl
144132

145133
<Steps>
146134
1. For each GitHub App:
147-
<Steps>
148-
1. On the GitHub App page, scroll down to "Private keys" and click "Generate a private key"
149-
1. Download the private key file
150-
1. Store the App ID, Installation ID, and private key securely in 1Password
151-
</Steps>
152-
2. Upload these credentials to AWS SSM Parameter Store:
153-
Save these credentials to AWS SSM at the [paths specified by the `eks/argocd` component](/layers/software-delivery/eks-argocd/setup/#-configure-argo-cd-to-use-github-apps)
135+
<Steps>
136+
1. On the GitHub App page, scroll down to "Private keys" and click "Generate a private key"
137+
1. Download the private key file
138+
1. Store the App ID, Installation ID, and private key securely in 1Password
139+
</Steps>
140+
2. Upload these credentials to AWS SSM Parameter Store
141+
<TaskList>
142+
- Upload the `Argo CD Instance` private key to `/argocd/argo_cd_instance/app_private_key` in `core-auto`:
143+
```bash
144+
# Replace acme with your namespace or assume the role separately.
145+
# Your default region should be the same as your primary region.
146+
assume-role acme-core-gbl-auto-admin
147+
chamber write argocd argo_cd_instance/app_private_key \
148+
"$(cat /path/to/argocd-deploy-non-prod.private-key.pem)"
149+
```
150+
- Upload the `Argo CD Notifications` private key to `/argocd/argo_cd_notifications/app_private_key` to `core-auto`:
151+
```bash
152+
assume-role acme-core-gbl-auto-admin
153+
chamber write argocd argo_cd_notifications/app_private_key \
154+
"$(cat /path/to/argocd-notifications.private-key.pem)"
155+
```
156+
</TaskList>
154157
</Steps>
155158
</Step>
156159

@@ -193,29 +196,20 @@ To address security concerns, the following compensating controls should be impl
193196
<Step>
194197
### <StepNumber/> Configure Branch Protection Rules
195198

196-
Set up the compensating controls by configuring branch protection rules:
197-
198-
**Organization-Level Branch Ruleset**:
199+
If branch protection rules are enabled in your GitHub Organization, you'll need to configure exceptions for the ArgoCD GitHub Apps. This allows ArgoCD to update repositories while still maintaining security. The GitHub Apps must be able to bypass branch protection rules in order for ArgoCD's automated deployments to work correctly.
199200

200-
<Steps>
201-
- Go to your organization settings
202-
- Navigate to "Code, planning, and automation" > "Branch rules"
203-
- Create a new ruleset that applies to all app repos except `acme/argocd-deploy-non-prod` and `acme/argocd-deploy-prod`
204-
- Configure the ruleset to prevent changes to the main branch with the Argo CD app not in the bypass list
205-
</Steps>
201+
Branch rulesets may be configured both or either at an organization and repository level. Check the enabled rulesets in both ArgoCD desired state repositories under "Code, planning, and automation" > "Branch rules". Add the ArgoCD GitHub Apps to the bypass list of any ruleset that prevents changes to the main branch.
206202

207-
**Argo CD Desired State Repositories Branch Ruleset**:
208-
209-
<Steps>
210-
- Create a separate ruleset that applies only to `acme/argocd-deploy-non-prod` and `acme/argocd-deploy-prod`
211-
- Configure the ruleset to prevent writing to the main branch, but with the Argo CD app in the bypass list
212-
</Steps>
213203
</Step>
214204

215205
<Step>
216206
## <StepNumber/> Configure Argo CD Desire State Repositories to Use GitHub Apps
217207

218-
Update your Argo CD desired state repository configuration to use the GitHub App instead of PAT:
208+
<Note title="Reference Architecture Users">
209+
This step should be pre-configured for Reference Architecture users.
210+
</Note>
211+
212+
Update your Argo CD desired state repository configuration to use the GitHub App:
219213

220214
```yaml
221215
components:
@@ -242,9 +236,18 @@ To address security concerns, the following compensating controls should be impl
242236
<Step>
243237
### <StepNumber/> Configure Argo CD to Use GitHub Apps
244238
245-
Update your Argo CD configuration to use the GitHub Apps instead of PATs:
239+
<Note title="Reference Architecture Users">
240+
Reference Architecture users will need to update both GitHub App and Installation IDs.
241+
</Note>
242+
243+
Update your Argo CD configuration to use the GitHub Apps by modifying the component configuration shown below.
244+
245+
<Note title="Installation ID">
246+
You can find the Installation ID by going to your GitHub Organization settings, selecting "GitHub Apps", clicking on your app, then selecting "Install App". The Installation ID will be in the URL, e.g. https://github.com/organizations/acme/settings/installations/44444444.
247+
</Note>
246248
247249
```yaml
250+
# stacks/catalog/eks/argocd/defaults.yaml
248251
components:
249252
terraform:
250253
eks/argocd:
@@ -278,12 +281,11 @@ To address security concerns, the following compensating controls should be impl
278281
279282
Redeploy both the `argocd-repo` component for both nonprod and prod.
280283

281-
<AtmosWorkflow workflow="deploy/argocd-repos" fileName="quickstart/app/argocd" />
284+
<AtmosWorkflow workflow="deploy/argocd-repos" fileName="quickstart/platform/argocd" />
282285

283286
Then redeploy all instances of `eks/argocd`
284287

285-
<AtmosWorkflow workflow="deploy/argocd" fileName="quickstart/app/argocd" />
286-
288+
<AtmosWorkflow workflow="deploy/argocd" fileName="quickstart/platform/argocd" />
287289
</Step>
288290

289291
<Step>

0 commit comments

Comments
 (0)