feat: Add options to specify AWS credential source for RDS functions

[hairyhum]

Change Overview

Currently AWS credentials are always taken from the profile, which limits
the ability to export into non-s3 storage and s3 storage in other regions.

Added options to specify a separate secret or use IAM role for service account.

Pull request type

Please check the type of change your PR introduces:

• 🚧 Work in Progress
• 🌈 Refactoring (no functional changes, no api changes)
• 🐹 Trivial/Minor
• 🐛 Bugfix
• 🌻 Feature
• 🗺️ Documentation
• 🤖 Test
• 🏗️ Build

Issues

• fixes #issue-number

Test Plan

• Run RDS example with new arguments added in the blueprint

  • Secret credentials with S3 in another region
  • Serviceaccount credentials

• 💪 Manual

• ⚡ Unit test

• 💚 E2E

[viveksinghggits]

Hi @hairyhum,
I was looking into the changes and was wondering if it would be a good idea to just extend profile in such a way that we can accept the new fields/IAM from profile instead of introducing the new secret way.
I think introducing the secret to accept the creds complicates the things and maybe complicates blueprints as well. And the other reason would be, profile usually is the resource where we say object store details should be defined.
Or, do you think it's not a good idea to just extend the profile for this new thing.

[hairyhum]

I was looking into the changes and was wondering if it would be a good idea to just extend profile in such a way that we can accept the new fields/IAM from profile instead of introducing the new secret way.

There is a principal issue with using profile as a source of the credentials. Because the credentials here are the data source credentials, while profile is by design the data destination credential.
So we either need to extend profile to have both source and destination, which doesn't make sense outside of RDS case, or we need to use another API to supply source credentials.

And the other reason would be, profile usually is the resource where we say object store details should be defined.

And it is still the case even with this change (unless I messed up and introduced a bug). There is no intention to switch away from profiles as destination (e.g. object store) configuration.