v4.1.4

What's new

• CodeQL and Security fixes
• Test updates for json log type
• Bug fixes (undefined rowKey, metadata length limit extended to 1024)

Commits

• 2dffce2: updated dependencies (Himanshu Pal) #121
• 59da290: updated readme,tests,fixed regex security issue (Himanshu Pal) #121
• 80a6993: added test dependency (Himanshu Pal) #121
• 1187447: updated readme (Himanshu Pal) #121
• 424323f: disable storage account related tests (Himanshu Pal) #121
• 54ebf2c: limit raised to 1024 for all metadata fields (Himanshu Pal) #121
• b394bb3: app service plan zone redundant check skipped (Himanshu Pal) #121
• 3fefd1c: skipping health,redundance,network access checks (Himanshu Pal) #121
• 91cb034: adding decoded file size, removed newline from file (Himanshu Pal) #122
• 640ab36: fixed undefined rowKey error (Himanshu Pal) #122
• 64bab20: updated zip file paths (Himanshu Pal) #122
• dd9fe99: updated tag (Himanshu Pal) #122

v4.1.3

Commits

• 0cd5928: comma condition added (Himanshu Pal) #120
• 46c3448: added fix for multiple events received with less size (Himanshu Pal) #120

v4.1.2

Commits

• fe625b4: added table dependency (Himanshu Pal) #118
• 16aa8a1: fixed timeout error (Himanshu Pal) #118
• d61d386: added test for dlq flow (Himanshu Pal) #118
• a70d74d: removed commented code (Himanshu Pal) #118
• ddd145a: bump zip version (Himanshu Pal)

v4.1.1

What's New

Updated NSG Flow logs collection to handle log truncation or overwritten scenario caused by following 3 ways: Host
Fault, VM Redeploy to a new Host , and re-use of MAC addresses.

Full Changelog

SumoLogic/sumologic-lambda-extensions@v4.1.0...v4.1.1

Commits

• 9977498: added nsglogs parsing function (Himanshu Pal) #116
• 239ab98: added fileoffset map update function (Himanshu Pal) #116
• 06c7cad: upgraded test dependencies (Himanshu Pal) #116

v4.1.0

What's New

Updates to Append Blob Reader (beta to GA) docs

1. Updated Append Blob Reader with Azure Function Runtime from 1.x to 4.x
2. Function app node version has been updated to node v18
3. We updated all the primary and secondary resources of the ARM template with new api versions that are compatible with the template pipeline
4. Added automatic creation of FileOffsetMap table

Updates to Block Blob Reader docs

1. Updated blobreaderzipdeploy.json for deploying using zip files.
2. Fixed bug by converting type of offset attribute to int64 in FileOffsetMap Table
3. Added automatic creation of FileOffsetMap table
4. Fixed bug with long blob file name
5. Dependency upgrades (azure-identity ^4.2.1)
6. Fixed bug in deployment with same storage account again.
7. Added filters for unknown file extensions and blob type.
8. Fixed memory issue by changing worker bitness to 64 bit.
9. Addition of default _sourceHost and _sourceName metadata

Full Changelog

SumoLogic/sumologic-lambda-extensions@v4.1.0-rc...v4.1.0

v4.1.0-rc

Commits

• Fixed the always on bug and added cors (Himanshu Pal) #86
• Added return for retryMax and passed context (Himanshu Pal) #88
• Added test for validating logs in Sumo Logic for BlobReader Solution(Himanshu Pal) #93
• Added test for validating metrics in Sumo Logic for EventHub Metrics Solution(Priyansh Patel) #94
• Build and package BlockBlobReader functions when releasing (Bradley Ford) #95

Azure Block Blob Collection security fixes and automation test updates

Changelog

Below changes are done for Azure Block Blob Collection

1. Features

   • Added automated tests for ARM templates in github action.
   • Added automated tests for finding older dependencies.
   • Updated function app logging with log levels.

2. Enhancements / Updates

   • Fixed broken unit tests for Azure Block Blob collection and upgraded test dependencies.
   • Added support for collector and source creation in unit tests.
   • Added test storage account creation in unit tests.
   • Upgraded api version in ARM template resources.

3. Fixes for soon to be deprecated resources

   • Migrated classic application Insights to workspace-based Application Insights Details

4. Security / CVE fixes
   Below fixes were done for security and best practices in the ARM template

   • BC_AZR_GENERAL_117: "Ensure that 'supportsHttpsTrafficOnly' is set to 'true'"
   • BC_AZR_NETWORKING_5: "Ensure web app redirects all HTTP traffic to HTTPS in Azure App Service"
   • BC_AZR_NETWORKING_8: "Ensure that 'HTTP Version' is the latest if used to run the web app"
   • BC_AZR_NETWORKING_6: "Ensure web app is using the latest version of TLS encryption"
   • apiVersions Should Be Recent In Reference Functions
   • Location Should Not Be Hardcoded

Upgradation Steps

1. Deploy the new template, in a new resource group with same http source url as was configured in earlier deployment.
2. Configure the metrics export to this new event hub namespace by updating the new diagnostic settings.
3. Verify whether the metrics are coming to Sumo Logic.
4. After verification, delete the resources in the older resource group created by earlier deployment.

Azure Metrics Function Runtime Upgraded to V4

Changelog

Below changes are done for Azure Metric Collection

1. Features

   • Added automated tests for ARM templates in github action.

2. Enhancements / Updates

   • 1. Fixed broken unit tests for Event Hub Metrics collection and upgraded test dependencies.
   • 2. Added support for collector and source creation in unit tests.
   • 3. Upgraded function runtime version to ~4 and nodejs version to node 18
   • 4. Upgraded api version in ARM template resources

3. Fixes for soon to be deprecated resources

   • Migrated classic application Insights to workspace-based Application Insights Details

4. Security / CVE fixes
   Below fixes were done for security and best practices for Event Hub Metrics ARM template

   • BC_AZR_GENERAL_117: "Ensure that 'supportsHttpsTrafficOnly' is set to 'true'"
   • BC_AZR_NETWORKING_16: "Ensure 'Trusted Microsoft Services' is enabled for Storage Account access"
   • BC_AZR_NETWORKING_5: "Ensure web app redirects all HTTP traffic to HTTPS in Azure App Service"
   • BC_AZR_NETWORKING_8: "Ensure that 'HTTP Version' is the latest if used to run the web app"
   • BC_AZR_NETWORKING_6: "Ensure web app is using the latest version of TLS encryption"
   • apiVersions Should Be Recent In Reference Functions
   • Location Should Not Be Hardcoded
   • Parameter Types and Names Should Be Consistent

Upgradation Steps

1. Deploy the new template, in a new resource group with same http source url as was configured in earlier deployment.
2. Configure the metrics export to this new event hub namespace by updating the new diagnostic settings.
3. Verify whether the metrics are coming to Sumo Logic.
4. After verification, delete the resources in the older resource group created by earlier deployment.

Azure BlockBlob Reader V2

Summary:

• Updated BlockBlob Reader with Azure Function Runtime from 1.x to 4.x

• Storage account can be in a different region than the collection resources deployment region, so a new user parameter, 'StorageAccountRegion' has been introduced

• Any JSON file in jsonlines format can be uploaded to send its data to SUMO

Architecture Overview:

• We updated all the primary and secondary resources of the ARM template with new api versions that are compatible with the template pipeline

• We added two new resources namely, EventGrid/systemTopics and EventGrid/systemTopics/eventSubscriptions in the ARM template

• Two other resources, namely, EventGrid/topics and Storage/storageAccounts/providers/eventSubscriptions have been removed from the original ARM template.

• New user parameter, 'StorageAccountRegion' has been introduced

• Storage account role assignment has been updated to storage account blob reader access role

• Function app node version has been updated to node v18

• Depreciated function code packages have been removed and updated with new code packages

• Node module libraries have been removed from function app folders

• Max retry of the consumer function has been set to 3 times from the current value of 10 times

v1.0.1

Older Release before azure runtime environment ~4 for blockblob reader