Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace shell script with Python #30

Open
in0rdr opened this issue Sep 12, 2024 · 3 comments · May be fixed by #31
Open

Replace shell script with Python #30

in0rdr opened this issue Sep 12, 2024 · 3 comments · May be fixed by #31
Assignees
@in0rdr
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@in0rdr
Copy link
Collaborator

in0rdr commented Sep 12, 2024

The current shell script implementation for the Kubernetes CronJob works for simple scripting purposes, but does not scale well for more complex logic (such as custom retention and maybe other features in the future).

Reasons for Python implementation:

  • We don't have to write to a file (we can read the file stream and store the object directly from memory to s3)
  • We can get rid of S3_URI input parameter
@in0rdr in0rdr linked a pull request Sep 12, 2024 that will close this issue
#30: Replace shell script with Python #31
Open
3 tasks
@in0rdr in0rdr mentioned this issue Sep 12, 2024
Merged
@in0rdr in0rdr added good first issue Good for newcomers help wanted Extra attention is needed labels Sep 13, 2024
@in0rdr
Copy link
Collaborator Author

in0rdr commented Sep 13, 2024

Good first issue if you would like to to do some Python programming. The draft is already testable. Would be good if we keep the VAULT_TOKEN authentication as a fallback (for testing purposes) when no Kubernetes auth credentials are provided as an input.

@in0rdr in0rdr self-assigned this Nov 25, 2024
@in0rdr
Copy link
Collaborator Author

in0rdr commented Nov 25, 2024

Wrote some pytests and will probably throw the code against my lab later this week.

The interpretation of S3_EXPIRE_DAYS changed slightly. I think the Python code can't handle an empty variable that well and you should just remove it from the CronJob when not used (no expiration).

@in0rdr
Copy link
Collaborator Author

in0rdr commented Nov 26, 2024

  • Added the ability to VAULT_SKIP_VERIFY for testing purposes
  • Notice that S3_EXPIRE_DAYS in the Cronjob spec needs to be a string, so it needs to be converted back to an integer in the Python program
  • Fixed some small issue with restartPolicy in the Cronjob
  • Tested with Minio and https://github.com/adfinis/vault-playground-on-kubernetes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@in0rdr in0rdr

Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

#30: Replace shell script with Python adfinis/vault-raft-backup-agent
1 participant
@in0rdr