Skip to content

Commit

Permalink
Merge branch 'main' into dev-alz-pattern
Browse files Browse the repository at this point in the history
  • Loading branch information
arjenhuitema committed Jul 19, 2024
2 parents 454628c + fd2a86d commit 074b0f0
Show file tree
Hide file tree
Showing 1,094 changed files with 182,498 additions and 83 deletions.
154 changes: 154 additions & 0 deletions .github/workflows/alz-pattern-update-policies.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
---
name: Update Policy Deployment Templates

##########################################
# Start the job on push for all branches #
##########################################

# yamllint disable-line rule:truthy
on:
pull_request_target:
types:
- opened
- reopened
- synchronize
- ready_for_review
paths:
- "services/**.json"
- "patterns/alz/**.json"
- "patterns/alz/templates/**.bicep"

env:
github_user_name: "github-actions"
github_email: "41898282+github-actions[bot]@users.noreply.github.com"
github_commit_message: "Auto-update Policies"
github_pr_number: ${{ github.event.number }}
github_pr_repo: ${{ github.event.pull_request.head.repo.full_name }}

permissions:
contents: write

###############
# Set the Job #
###############

jobs:
update-portal:
name: Update Policy Deployment Templates
runs-on: ubuntu-latest
if: |
(
github.event.pull_request.head.repo.full_name == 'Azure/azure-monitor-baseline-alerts'
)
||
(
github.event.pull_request.head.repo.full_name != 'Azure/azure-monitor-baseline-alerts'
&&
contains(github.event.pull_request.labels.*.name, 'PR: Safe to test :test_tube:')
)
||
(
github.event_name == 'workflow_dispatch'
)
||
(
github.event_name == 'merge_group'
)
steps:
- name: Check out repository
uses: actions/checkout@v3

- name: Show env
run: env | sort

- name: Check out PR
run: |
echo "==> Check out PR..."
gh pr checkout "$github_pr_number"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Configure local git
run: |
echo "git user name : $github_user_name"
git config --global user.name "$github_user_name"
echo "git user email : $github_email"
git config --global user.email "$github_email"
- name: Update policies
run: bicep build ./patterns/alz/templates/policies-Automation.bicep --outfile ./patterns/alz/policyDefinitions/policies-Automation.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Compute.bicep --outfile ./patterns/alz/policyDefinitions/policies-Compute.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Hybrid.bicep --outfile ./patterns/alz/policyDefinitions/policies-Hybrid.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-KeyManagement.bicep --outfile ./patterns/alz/policyDefinitions/policies-KeyManagement.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Monitoring.bicep --outfile ./patterns/alz/policyDefinitions/policies-Monitoring.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Network.bicep --outfile ./patterns/alz/policyDefinitions/policies-Network.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-NotificationAssets.bicep --outfile ./patterns/alz/policyDefinitions/policies-NotificationAssets.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-RecoveryServices.bicep --outfile ./patterns/alz/policyDefinitions/policies-RecoveryServices.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-ServiceHealth.bicep --outfile ./patterns/alz/policyDefinitions/policies-ServiceHealth.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Storage.bicep --outfile ./patterns/alz/policyDefinitions/policies-Storage.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policies-Web.bicep --outfile ./patterns/alz/policyDefinitions/policies-Web.json

- name: Update policy set definitions (initiatives)
run: bicep build ./patterns/alz/templates/policySets.bicep --outfile ./patterns/alz/policyDefinitions/policySets.json

- name: Update policy set definitions (initiatives)
run: bicep build ./src/templates/initiatives.bicep --outfile ./eslzArm/managementGroupTemplates/policyDefinitions/initiatives.json

- name: Check git status
run: |
echo "==> Check git status..."
git status --short --branch
- name: Stage changes
run: |
echo "==> Stage changes..."
mapfile -t STATUS_LOG < <(git status --short | grep eslzArm/)
if [ ${#STATUS_LOG[@]} -gt 0 ]; then
echo "Found changes to the following files:"
printf "%s\n" "${STATUS_LOG[@]}"
git add --all ./eslzArm
else
echo "No changes to add."
fi
- name: Push changes
run: |
echo "==> Check git diff..."
mapfile -t GIT_DIFF < <(git diff --cached)
printf "%s\n" "${GIT_DIFF[@]}"
if [ ${#GIT_DIFF[@]} -gt 0 ]; then
echo "==> Commit changes..."
git commit --message "$github_commit_message [$GITHUB_ACTOR/${GITHUB_SHA::8}]"
echo "==> Push changes..."
echo "Pushing changes to: $github_pr_repo"
git push "https://[email protected]/$github_pr_repo.git" "HEAD:$GITHUB_HEAD_REF"
else
echo "No changes found."
fi
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
88 changes: 88 additions & 0 deletions .github/workflows/generate-templates.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
# Workflow for generating (arm/bicep/etc) templates for each alert
name: Generate Templates

on:
# Runs on pushes targeting the default branch
push:
branches:
- main
paths:
- 'services/**/alerts.yaml'
- 'tooling/generate-templates/**'

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch: {}

permissions:
contents: write
pull-requests: write

jobs:
generate-templates:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
ref: main
submodules: recursive
fetch-depth: 0

- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: '3.12' # install the python version needed

- name: Install Python Packages and Requirements
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
working-directory: tooling/generate-templates

- name: Generate Templates
id: generate
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git config --local user.email "github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git checkout -b github-action-generate-templates
# Generate templates for alerts
echo "Generating templates for alerts..."
python tooling/generate-templates/generate-templates.py --path services --output services --template_path tooling/generate-templates/templates
# Check if there are any changes in the services directory
git add services
# Check if there are any changes to commit
if [[ `git status --porcelain` ]]; then
git commit -m "[GitHub Action - Generate Templates] Generate templates for alerts"
# Push changes to the current branch
git push --set-upstream origin github-action-generate-templates --force
prs=$(gh pr list \
--repo "$GITHUB_REPOSITORY" \
--head 'github-action-generate-templates' \
--base 'main' \
--json title \
--jq 'length')
if ((prs > 0)); then
echo "skippr=true" >> "$GITHUB_OUTPUT"
fi
else
echo "skippr=true" >> "$GITHUB_OUTPUT"
fi
# Diasble PR creation for now since it is not supported in the Azure repo
# - name: Create pull request
# if: '!steps.generate.outputs.skippr'
# env:
# GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# run: |
# # Create a pull request
# echo "Creating a pull request..."
# gh pr --repo ${{ github.repository }} create --title "[GitHub Action - Generate Templates] Generate templates for alerts" --body "This PR was automatically generated by the workflow." --base main --head github-action-generate-templates

5 changes: 5 additions & 0 deletions .github/workflows/hugo-build-pr-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@ on:
- main
paths:
- 'docs/**'
- 'services/**/alerts.yaml'
- 'services/**/*.md'
- 'services/**/templates/**'
- 'tooling/export-alerts/**'
- 'tooling/generate-templates/**'
- '.github/workflows/hugo-build-pr-check.yml'
workflow_dispatch: {}

Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/hugo-site-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ on:
- 'docs/**'
- 'services/**/alerts.yaml'
- 'services/**/*.md'
- 'services/**/templates/**'
- 'tooling/export-alerts/**'
- 'tooling/generate-templates/**'

Expand Down Expand Up @@ -69,12 +70,6 @@ jobs:
python export-alerts.py --path ../../services --template ./alerts-template.xlsx --output-xls ../../services/amba-alerts.xlsx --output-json ../../services/amba-alerts.json --output-yaml ../../services/amba-alerts.yaml
working-directory: tooling/export-alerts

- name: Generate Templates
run: |
pip install -r requirements.txt
python generate-templates.py --path ../../services --output ../../artifacts/templates
working-directory: tooling/generate-templates

- name: Setup Pages
id: pages
uses: actions/configure-pages@v5
Expand Down
9 changes: 3 additions & 6 deletions docs/content/patterns/alz/Bring-your-own-Managed-Identity.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,13 @@ This new feature is used in Log-search based alerts. At the moment of this relea
We're planning to use this feature more in the future and to include it as part of other alerts.
{{< /hint >}}

<!--
## Switching between BYO UAMI and new UAMI

The [conditional deployment behavior](../alz/Bring-your-own-Managed-Identity.md#conditional-deployment-behavior) discussed earlier, allows brownfield customers to switch from a new created UAMI to an existing one and viceversa.
Should customers decide to switch, it will be enough to:

- change the values in the parameter file to match one of the two scenarios previously discussed
- redeploy the AMBA-ALZ pattern
- run the remediation. Atthe moment it is sufficient to run the remediation for the [Deploy Azure Monitor Vaseline Alerts for Hybrid VMs](https://raw.githubusercontent.com/Azure/azure-monitor-baseline-alerts/main/patterns/alz/policySetDefinitions/Deploy-HybridVM-Alerts.json) policy initiative
- Change the values in the parameter file to match one of the two scenarios previously discussed
- Redeploy the AMBA-ALZ pattern
- Run the remediation for the [Deploy Azure Monitor Baseline Alerts for Hybrid VMs](https://raw.githubusercontent.com/Azure/azure-monitor-baseline-alerts/main/patterns/alz/policySetDefinitions/Deploy-HybridVM-Alerts.json) policy initiative as documented at [Remediate Policies](../deploy/Remediate-Policies)

The code will reconfigure the necessary alerts to use either the customer's provided UAMI or the new one created during the deployment.
-->
38 changes: 31 additions & 7 deletions docs/content/patterns/alz/Known-Issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,15 @@ weight: 100

### Error includes

*failed to resolve table or column expression named*
The error can be presented with one of the two following messages:

- *failed to resolve table or column expression named*

- *{"code": "BadRequest", "message": "{\r\n "error": {\r\n "code": "DraftClientException",\r\n "message": "The request had some invalid properties Activity D: 3332f9c0-b4d4-464b-8ec4-44a670ba745b."\r\n }\r\n}"}*

### Cause

The underlying data is not present in the Log Analytics table.
The underlying data isn't present in the Log Analytics table or there's no virtual machine associated to any VM Insights data collection rule.

### Resolution

Expand All @@ -26,7 +30,7 @@ Deployment of AMBA fails when there are orphaned role assignments.

*"error": { </br>
&emsp;"code": "RoleAssignmentUpdateNotPermitted", </br>
&emsp;"message": "Tenant ID, application ID, principal ID, and scope are not allowed to be updated." </br>
&emsp;"message": "Tenant ID, application ID, principal ID, and scope aren't allowed to be updated." </br>
&emsp;}*

### Cause
Expand All @@ -48,12 +52,12 @@ When a role or a role assignment is removed, some orphaned object can still appe

### Cause

A deployment has been performed using one region, for example "uksouth", and when you try to deploy again to the same scope but to a different region you will receive an error. This happens even when a cleanup has been performed (see [Cleaning up a Deployment](../Cleaning-up-a-Deployment) for more details). This is because deployment entries still exist from the previous operation, so a region conflict is detected blocking you to run another deployment using a different region.
A deployment has been performed using one region, for example "uksouth", and when you try to deploy again to the same scope but to a different region you'll receive an error. This happens even when a cleanup has been performed (see [Cleaning up a Deployment](../Cleaning-up-a-Deployment) for more details). This is because deployment entries still exist from the previous operation, so a region conflict is detected blocking you to run another deployment using a different region.

### Resolution
Situation 1: You are trying to deploy to a region different from the one used in previous deployment. Deploying to the same scope in a different region is not necessary. The definitions and assignments are scoped to a management group and are not region-specific. No action is required.
Situation 1: You're trying to deploy to a region different from the one used in previous deployment. Deploying to the same scope in a different region isn't necessary. The definitions and assignments are scoped to a management group and aren't region-specific. No action is required.

Situation 2: You cleaned up a previous implementation and want to deploy again to a different region. To resolve this issue, follow the steps below:
Situation 2: You cleaned up a previous implementation and want to deploy again to a different region. To resolve this issue, follow the following steps:

1. Navigate to ***Management Groups***
2. Select the management group (corresponding to the value entered for the *enterpriseScaleCompanyPrefix* during the deployment) were AMBA deployment was targeted to
Expand Down Expand Up @@ -83,7 +87,7 @@ If you deployed AMBA just one time, you have 14 deployment instances
The limit of 800 deployment for the given management group scope has been reached. More information can be found at [Management group limits](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#management-group-limits)

### Resolution
To resolve this issue, follow the steps below:
To resolve this issue, follow the following steps:

1. Navigate to ***Management Groups***
2. Select the management group (corresponding to the value entered for the *enterpriseScaleCompanyPrefix* during the deployment) were AMBA deployment was targeted to
Expand All @@ -101,3 +105,23 @@ To recognize the deployment names belonging to AMBA, select those deployments wh
If you deployed AMBA just one time, you have 14 deployment instances

{{< /hint >}}

## Failed to deploy because of 'location' property not specified

### Error includes

The error can be presented with one of the two following messages:

- *{"code": "InvalidDeployment", "message": "The 'location' property must be specified for 'amba-id-amba-prod-001'. Please see https://aka.ms/arm-deployment-subscription for usage details."}*

- *InvalidDeployment - Long running operation failed with status 'Failed'. Additional Info:'The 'location' property must be specified for 'amba-id-amba-prod-001'. Please see https://aka.ms/arm-deployment-subscription for usage details.'*

### Cause

The new [Bring Your Own User Assigned Managed Identity (BYO UAMI)](../Bring-your-own-Managed-Identity) allows you to either use an existing User Assigned Managed Identity (UAMI) or to create a new one in the management subscription automatically assigning the Monitoring reader role to it at the parent pseudo root Management Group. If you opted for creating a new UAMI, the management subscription id is needed.

### Resolution

Set the parameter for the management subscription id correctly in the parameter file:

![New UAMI deployed by the template](../media/alz-UAMI-Param-Example-2.png)
Loading

0 comments on commit 074b0f0

Please sign in to comment.