Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EPIC] S3 multi-tenancy on tmail-backend #1315

Open
Arsnael opened this issue Nov 18, 2024 · 3 comments
Open

[EPIC] S3 multi-tenancy on tmail-backend #1315

Arsnael opened this issue Nov 18, 2024 · 3 comments

Comments

@Arsnael
Copy link
Member

Arsnael commented Nov 18, 2024

After the latest discussions on this topic on the james ML, it seems that multi-tenancy might be more suitable for implementing it on tmail-backend side.

Initial topic on james backlog: linagora/james-project#5263

We need to groom and find a solution how to achieve a similar result on Tmail side in a smart way

@vttranlina
Copy link
Member

vttranlina commented Nov 20, 2024

After analyzing the related code that uses the method org.apache.james.blob.api.BlobStore#getDefaultBucketName, here's my proposal:
We will modify all class where the BlobStore APIs are used to explicitly pass the calculated BucketName (based on the domain).

For the Tmail-backend module:

  • EncryptedEmailContentStore
  • PublicAssetRepository

We will update the interfaces to include an additional argument, such as a Domain or Tenant.

For example:
Current:
def store(messageId: MessageId, encryptedEmailContent: EncryptedEmailContent): Publisher[Unit]
Proposed:
def store(domain: Optional[Domain], messageId: MessageId, encryptedEmailContent: EncryptedEmailContent): Publisher[Unit]

And using MailboxSession to bypass Domain value

For the James site (classes from james-project module):

We will create new classes specifically for Tmail-backend and use Guice to rebind them:

  • DeleteMessageListener
  • CassandraAttachmentMapper (and related class, eg: StoreAttachmentManager...)
  • CassandraMessageDAOV3
  • DeletedMessageVaultDeletionCallback
  • ExportService

To reduce effort, we will focus on BlobStore for storing body data. For metadata, we can retain the existing logic and continue using a shared default bucket.

@quantranhong1999
Copy link
Member

My idea:

  • Make BucketName an interface in James (Or make the BucketName overriable), and provide the James default implementation for it as of today DefaultBucketName.

  • Implement a TenancyBucketName implementation on TMail, which carries on the domain information.
    How to propagate domain information from James callers to TMail blob store:

    Introduce in James a BucketNameSupplier:

    @FunctionalInterface
    public interface BucketNameSupplier {
        BucketName apply(BucketName sourceBucketName, Optional<Domain> domain);
    }

    James would implement a DefaultBucketNameSupplier that is tenant agnostic:

    public class DefaultBucketNameSupplier implements BucketNameSupplier {
        @Override
        public BucketName apply(BucketName sourceBucketName, Optional<Domain> domain) {
            return sourceBucketName;
        }
    }

    Meanwhile, TMail would implement its TMailBucketNameSupplier:

    public class TMailBucketNameSupplier implements BucketNameSupplier {
        @Override
        public BucketName apply(BucketName sourceBucketName, Optional<Domain> domain) {
            //Take domain/tenant information into account, and return a TenancyBucketName
        }
    }

    Then we would try on James side to make the callers provide the tenant/domain information at best.

  • On TMail, implement a TenancyS3BlobStoreDAO that applies the tenancy logic (bucket/prefix/sse-c) and, of course, takes into account the TenancyBucketName with tenant information.
    Then replace S3BlobStoreDAO binding with TenancyS3BlobStoreDAO, especially for the Postgres-app.

@hungphan227
Copy link
Contributor

UploadRepository implementations may be also needed to override

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants