Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question regarding Cache FS and how it evicts #1943

Closed
usrme opened this issue Nov 26, 2024 · 2 comments
Closed

Question regarding Cache FS and how it evicts #1943

usrme opened this issue Nov 26, 2024 · 2 comments

Comments

@usrme
Copy link

usrme commented Nov 26, 2024

I have a setup with a really large cache (see snippet from values file below) and I'm not sure based on what criteria exactly the eviction from local disk to S3 is being made. Based on this white paper it’s based on access times:

Since the maximum size can be set, artifacts in this cache will “rotate” based on their last downloaded date or creation/modified date.

But I have a concrete example of a blob that was created today (at the time of writing) that is already stored in S3:

$ stat /var/opt/jfrog/artifactory/data/cache/ff/ff7e66484d8723c7f1a7b1622e018604c2131581
  File: /var/opt/jfrog/artifactory/data/cache/ff/ff7e66484d8723c7f1a7b1622e018604c2131581
  Size: 34946           Blocks: 72         IO Block: 4096   regular file
Device: 10303h/66307d   Inode: 825388      Links: 1
Access: (0640/-rw-r-----)  Uid: ( 1030/artifactory)   Gid: ( 1030/artifactory)
Access: 2024-11-26 08:34:34.979291773 +0000
Modify: 2024-11-26 08:34:34.971291759 +0000
Change: 2024-11-26 08:34:34.971291759 +0000
 Birth: 2024-11-26 08:34:34.971291759 +0000

Image

I'd really love some clarification around this and to learn if I've possibly misconfigured something. I tried to grep the name of the blob across /var/opt/jfrog/artifactory/log and /opt/jfrog/artifactory/var/log, but nothing came up.

Could it be that I'm misunderstanding that significance of the files in that directory and that they end up there after whatever the configured cache settings are for any given remote repository? If so, where do they reside before they end up in that directory then and how does the S3 storage relate to the cache directory?


Snippet from values file regarding persistence:

---
persistence:
  mountPath: /var/opt/jfrog/artifactory
  enabled: true
  size: 500Gi
  accessMode: ReadWriteOnce
  cacheProviderDir: /var/opt/jfrog/artifactory/data/cache
  # 450GB
  maxCacheSize: 450000000000
  type: aws-s3-v3
  awsS3V3:
    bucketName: bucket-name
    region: eu-west-1
    path: artifactory/filestore
    endpoint: s3.amazonaws.com
    roleName: artifactory-role
    testConnection: true

Chart version: 107.90.8.

@gitta-jfrog
Copy link
Collaborator

Hi @usrme This question is related to Artifactory Filestore. Can you open a ticket to JFrog Support?

@usrme
Copy link
Author

usrme commented Dec 10, 2024

Answer from support:

Regarding the Artifactory cache-fs populating, please be informed that the changes on the artifacts like download or upload of artifacts will be updated on the LRU map(present in the java memory). Based on the changes the artifacts are updated in the LRU map and further binaries related to the artifacts gets deleted or added to the cache-fs directory based on the storage assigned. The artifacts that are not updated will be removed from the LRU map and the same from the cache-fs folder.

The Artifacts binaries will be pushed to the binarystore after the successful artifacts upload which will be used for the further download of artifacts. Binaries that are present in the cache-fs directory will be based on the LRU map and will be removed from the cache-fs directory after the binary entry removed from the LRU map. Please refer to the What is cache-fs? knowledge base article for more information.

@usrme usrme closed this as completed Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants