Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: oss Provider #193

Merged
merged 34 commits into from
Dec 16, 2024
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
3881f0b
feat: maxcompute provider
rahmatrhd Nov 27, 2024
50f1f15
fix: fix missing argument
rahmatrhd Nov 27, 2024
88ee6fb
feat: support revoke project member
rahmatrhd Nov 29, 2024
d95c75c
feat: support role assuming
rahmatrhd Nov 29, 2024
80ce978
feat: support grant dependencies
rahmatrhd Dec 3, 2024
e6dc506
debug: create client with fresh sts token
Dec 9, 2024
1556cb6
fix: add time package in dockerfile
Dec 10, 2024
293efd2
fix: remove debug logs
Dec 10, 2024
f165d8d
feat(maxcompute): add renew mechanism for clients
Dec 10, 2024
f3509b8
fix(maxcompute): update assume role duration seconds
Dec 10, 2024
a5c147e
feat(maxcompute): centralized sts token refresh logic
Dec 10, 2024
ec071ab
test: fix breaking tests
rahmatrhd Dec 11, 2024
b6dcac5
fix: store grant to db
rahmatrhd Dec 12, 2024
d2cd14f
fix: fix grant dependency behaviour
rahmatrhd Dec 12, 2024
3e6641c
fix: validate table permission
rahmatrhd Dec 12, 2024
ee59c47
chore: add comment on dockerfile
rahmatrhd Dec 12, 2024
c5bae7b
chore: go mod tidy
rahmatrhd Dec 12, 2024
aa8de0b
fix: fix client caching
rahmatrhd Dec 12, 2024
8396a0f
fix: handled edgecase where stsClients map key could be same for diff…
Dec 12, 2024
c32011c
feat: oss provider
Dec 3, 2024
0e125f9
feat: update grant and revoke logic
Dec 3, 2024
a9ab00c
feat(oss): add new policy statement for every role
Dec 6, 2024
e994c4f
fix(oss): handle edge case when policy is not found while granting
Dec 6, 2024
78ef674
fix(oss): delete bucket policy if there's no more statements left aft…
Dec 6, 2024
0b1f76b
fix(oss): update principal in statements for each role
Dec 6, 2024
9809f45
feat(oss): handle edge case when multiple policy statements are prese…
Dec 9, 2024
e5c22d4
fix(oss): add refresh sts token mechanism
Dec 10, 2024
b014f8e
chore(oss): refactor
Dec 10, 2024
0bd050f
feat(oss): centralize sts token refresh logic
Dec 10, 2024
b9b3857
fix(oss): extract principalID from appeals Account ID
Dec 11, 2024
d094e7f
fix: handled edgecase where stsClients map key could be same for diff…
Dec 12, 2024
038eeff
fix: update oss client caching logic
Dec 12, 2024
6dcb718
Merge branch 'maxcompute-provider' into oss-provider
Dec 12, 2024
405b18c
Merge remote-tracking branch 'origin/main' into oss-provider
rahmatrhd Dec 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ RUN apk add openssl

RUN apk add --no-cache ca-certificates && update-ca-certificates

# odps package has an unhandled (panicking) error that requires this pkg to be installed to avoid the panic:
# https://github.com/aliyun/aliyun-odps-go-sdk/blob/master/odps/restclient/rest_client.go#L171
# https://github.com/aliyun/aliyun-odps-go-sdk/blob/master/odps/common/http_const.go#L38
RUN apk --no-cache add tzdata

RUN curl --output /usr/local/share/ca-certificates/SectigoRSADomainValidationSecureServerCA.crt http://crt.sectigo.com/SectigoRSADomainValidationSecureServerCA.crt

RUN openssl x509 -inform DER -in /usr/local/share/ca-certificates/SectigoRSADomainValidationSecureServerCA.crt -out /usr/local/share/ca-certificates/SectigoRSADomainValidationSecureServerCA.pem -text
Expand Down
43 changes: 43 additions & 0 deletions core/appeal/mocks/grantService.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 55 additions & 0 deletions core/appeal/mocks/providerService.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

46 changes: 45 additions & 1 deletion core/appeal/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ type providerService interface {
ValidateAppeal(context.Context, *domain.Appeal, *domain.Provider, *domain.Policy) error
GetPermissions(context.Context, *domain.ProviderConfig, string, string) ([]interface{}, error)
IsExclusiveRoleAssignment(context.Context, string, string) bool
GetDependencyGrants(context.Context, domain.Grant) ([]*domain.Grant, error)
}

//go:generate mockery --name=resourceService --exported --with-expecter
Expand All @@ -96,6 +97,7 @@ type grantService interface {
List(context.Context, domain.ListGrantsFilter) ([]domain.Grant, error)
Prepare(context.Context, domain.Appeal) (*domain.Grant, error)
Revoke(ctx context.Context, id, actor, reason string, opts ...grant.Option) (*domain.Grant, error)
Create(ctx context.Context, grant *domain.Grant) error
}

//go:generate mockery --name=auditLogger --exported --with-expecter
Expand Down Expand Up @@ -1489,10 +1491,52 @@ func (s *Service) GrantAccessToProvider(ctx context.Context, a *domain.Appeal, o
}
}

if err := s.providerService.GrantAccess(ctx, *a.Grant); err != nil {
appealCopy := *a
appealCopy.Grant = nil
grantWithAppeal := *a.Grant
grantWithAppeal.Appeal = &appealCopy

// grant access dependencies (if any)
dependencyGrants, err := s.providerService.GetDependencyGrants(ctx, grantWithAppeal)
if err != nil {
return fmt.Errorf("getting grant dependencies: %w", err)
}
for _, dg := range dependencyGrants {
activeDepGrants, err := s.grantService.List(ctx, domain.ListGrantsFilter{
Statuses: []string{string(domain.GrantStatusActive)},
AccountIDs: []string{dg.AccountID},
AccountTypes: []string{dg.AccountType},
ResourceIDs: []string{dg.Resource.ID},
Permissions: dg.Permissions,
Size: 1,
})
if err != nil {
return fmt.Errorf("failed to get existing active grant dependency: %w", err)
}

if len(activeDepGrants) > 0 {
continue
}

dg.Status = domain.GrantStatusActive
dg.Appeal = &appealCopy
if err := s.providerService.GrantAccess(ctx, *dg); err != nil {
return fmt.Errorf("failed to grant an access dependency: %w", err)
}
dg.Appeal = nil

dg.Owner = a.CreatedBy
if err := s.grantService.Create(ctx, dg); err != nil {
return fmt.Errorf("failed to store grant of access dependency: %w", err)
}
}

// grant main access
if err := s.providerService.GrantAccess(ctx, grantWithAppeal); err != nil {
return fmt.Errorf("granting access: %w", err)
}

grantWithAppeal.Appeal = nil
return nil
}

Expand Down
35 changes: 27 additions & 8 deletions core/appeal/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import (
"time"

"github.com/go-playground/validator/v10"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/google/uuid"
"github.com/goto/guardian/core/appeal"
appealmocks "github.com/goto/guardian/core/appeal/mocks"
Expand Down Expand Up @@ -2476,6 +2478,7 @@ func (s *ServiceTestSuite) TestCreate() {
h.mockGrantService.EXPECT().List(mock.Anything, mock.Anything).Return([]domain.Grant{}, nil).Once()
h.mockGrantService.EXPECT().Prepare(mock.Anything, mock.Anything).Return(&domain.Grant{}, nil).Once()
h.mockPolicyService.EXPECT().GetOne(mock.Anything, mock.Anything, mock.Anything).Return(overriddingPolicy, nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(nil).Once()

err := h.service.Create(context.Background(), []*domain.Appeal{input}, appeal.CreateWithAdditionalAppeal())
Expand Down Expand Up @@ -2740,6 +2743,7 @@ func (s *ServiceTestSuite) TestCreate__WithExistingAppealAndWithAutoApprovalStep
).
Return(preparedGrant, nil).Once()
h.mockPolicyService.EXPECT().GetOne(mock.Anything, policies[0].ID, policies[0].Version).Return(policies[0], nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(nil).Once()

h.mockRepository.EXPECT().
Expand Down Expand Up @@ -2930,6 +2934,7 @@ func (s *ServiceTestSuite) TestCreate__WithAdditionalAppeals() {
h.mockPolicyService.EXPECT().GetOne(mock.AnythingOfType("*context.cancelCtx"), policies[0].ID, policies[0].Version).Return(policies[0], nil).Once()

// 2.b grant access for the additional appeal
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.AnythingOfType("*context.cancelCtx"), mock.AnythingOfType("domain.Grant")).Return(nil).Once().Run(func(args mock.Arguments) {
grant := args.Get(1).(domain.Grant)
s.Equal(expectedAdditionalGrant.ID, grant.ID)
Expand All @@ -2943,6 +2948,7 @@ func (s *ServiceTestSuite) TestCreate__WithAdditionalAppeals() {
h.mockNotifier.EXPECT().Notify(h.ctxMatcher, mock.Anything).Return(nil).Once()

// 1.b grant access for the main appeal
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(h.ctxMatcher, mock.AnythingOfType("domain.Grant")).Return(nil).Once().Run(func(args mock.Arguments) {
grant := args.Get(1).(domain.Grant)
s.Equal(expectedGrant.ID, grant.ID)
Expand Down Expand Up @@ -4830,6 +4836,7 @@ func (s *ServiceTestSuite) TestUpdateApproval() {
appeal.RevokeReasonForExtension, mock.Anything, mock.Anything).
Return(expectedNewGrant, nil).Once()
h.mockPolicyService.EXPECT().GetOne(mock.Anything, mock.Anything, mock.Anything).Return(dummyPolicy, nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(nil).Once()
h.mockRepository.EXPECT().Update(h.ctxMatcher, appealDetails).Return(nil).Once()
h.mockNotifier.EXPECT().Notify(h.ctxMatcher, mock.Anything).Return(nil).Once()
Expand Down Expand Up @@ -5276,7 +5283,8 @@ func (s *ServiceTestSuite) TestUpdateApproval() {
h.mockGrantService.EXPECT().
Prepare(mock.Anything, mock.Anything).Return(tc.expectedGrant, nil).Once()

h.mockProviderService.EXPECT().GrantAccess(mock.Anything, *tc.expectedGrant).Return(nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, grantArgMatcher(*tc.expectedGrant)).Return(nil).Once()
}

h.mockRepository.EXPECT().Update(h.ctxMatcher, tc.expectedResult).Return(nil).Once()
Expand Down Expand Up @@ -5433,7 +5441,8 @@ func (s *ServiceTestSuite) TestUpdateApproval() {
h.mockGrantService.EXPECT().
Prepare(mock.Anything, mock.Anything).Return(tc.expectedGrant, nil).Once()

h.mockProviderService.EXPECT().GrantAccess(mock.Anything, *tc.expectedGrant).Return(nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, grantArgMatcher(*tc.expectedGrant)).Return(nil).Once()
}

h.mockRepository.EXPECT().Update(h.ctxMatcher, tc.expectedResult).Return(nil).Once()
Expand Down Expand Up @@ -5531,6 +5540,7 @@ func (s *ServiceTestSuite) TestUpdateApproval() {
h.mockGrantService.EXPECT().
Prepare(mock.Anything, mock.Anything).Return(nil, nil).Once()

h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(nil).Once()

h.mockRepository.EXPECT().Update(h.ctxMatcher, mock.Anything).Return(nil).Once()
Expand Down Expand Up @@ -5611,9 +5621,8 @@ func (s *ServiceTestSuite) TestGrantAccessToProvider() {
Version: 1,
}, nil).Once()

h.mockProviderService.
On("GrantAccess", mock.Anything, mock.Anything).
Return(fmt.Errorf("error")).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(fmt.Errorf("error")).Once()

actualError := h.service.GrantAccessToProvider(context.Background(), &domain.Appeal{
PolicyID: "policy_1",
Expand All @@ -5633,9 +5642,8 @@ func (s *ServiceTestSuite) TestGrantAccessToProvider() {
Version: 1,
}, nil).Once()

h.mockProviderService.
On("GrantAccess", mock.Anything, mock.Anything).
Return(nil).Once()
h.mockProviderService.EXPECT().GetDependencyGrants(mock.Anything, mock.AnythingOfType("domain.Grant")).Return(nil, nil).Once()
h.mockProviderService.EXPECT().GrantAccess(mock.Anything, mock.Anything).Return(nil).Once()

actualError := h.service.GrantAccessToProvider(context.Background(), &domain.Appeal{
PolicyID: "policy_1",
Expand Down Expand Up @@ -6295,3 +6303,14 @@ func (s *ServiceTestSuite) TestGetAppealsTotalCount() {
s.NoError(actualError)
})
}

func grantArgMatcher(expected domain.Grant) any {
return mock.MatchedBy(func(actual domain.Grant) bool {
return cmp.Equal(
expected,
actual,
cmpopts.EquateApproxTime(time.Millisecond),
cmpopts.IgnoreFields(domain.Grant{}, "Appeal"),
)
})
}
43 changes: 43 additions & 0 deletions core/grant/mocks/repository.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions core/grant/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ type repository interface {
BulkUpsert(context.Context, []*domain.Grant) error
GetGrantsTotalCount(context.Context, domain.ListGrantsFilter) (int64, error)
ListUserRoles(context.Context, string) ([]string, error)
Create(context.Context, *domain.Grant) error
}

//go:generate mockery --name=providerService --exported --with-expecter
Expand Down Expand Up @@ -110,6 +111,10 @@ func (s *Service) GetByID(ctx context.Context, id string) (*domain.Grant, error)
return s.repo.GetByID(ctx, id)
}

func (s *Service) Create(ctx context.Context, grant *domain.Grant) error {
return s.repo.Create(ctx, grant)
}

func (s *Service) Update(ctx context.Context, payload *domain.GrantUpdate) (*domain.Grant, error) {
grant, err := s.GetByID(ctx, payload.ID)
if err != nil {
Expand Down
2 changes: 2 additions & 0 deletions core/provider/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,6 @@ var (
ErrAppealValidationInvalidDurationValue = errors.New("invalid duration value")
ErrAppealValidationMissingRequiredParameter = errors.New("missing required parameter")
ErrAppealValidationMissingRequiredQuestion = errors.New("missing required question")

ErrGrantAlreadyExists = errors.New("grant already exists")
)
Loading
Loading