Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add endpoints for upload, list and delete #1

Closed
wants to merge 6 commits into from
Closed

Conversation

psankhe28
Copy link
Collaborator

@psankhe28 psankhe28 commented Jun 29, 2024

Summary by Sourcery

This pull request introduces new endpoints for file upload, list, and delete functionalities using Flask and Minio. It also adds a Makefile for managing various tasks, sets up GitHub Actions workflows for CI/CD, and configures Sphinx for documentation. Additionally, it includes initial test setups and various configuration files for code quality and security.

  • New Features:
    • Added endpoints for file upload, list, and delete functionalities using Flask and Minio.
    • Introduced a new Makefile for managing environment, code quality, testing, and documentation tasks.
  • Enhancements:
    • Implemented a TUS server for handling file uploads and assembly.
    • Configured Sphinx for generating project documentation.
  • Build:
    • Added a Makefile to streamline build and environment management tasks.
  • CI:
    • Set up GitHub Actions workflows for code quality checks, testing, documentation, and vulnerability scanning.
    • Added a workflow for automatic project structure updates using Cruft.
  • Deployment:
    • Configured a GitHub Actions workflow for publishing the package to PyPI upon release.
  • Documentation:
    • Added Sphinx configuration and Makefile for generating project documentation.
    • Included a Code of Conduct document.
  • Tests:
    • Added unit and integration test directories and initial test setup.
    • Configured GitHub Actions workflows for running unit and integration tests and uploading coverage reports.
  • Chores:
    • Added a .pre-commit-config.yaml file for managing pre-commit hooks.
    • Included a .safety-policy.yml file for managing security policies.

Copy link
Contributor

sourcery-ai bot commented Jun 29, 2024

Reviewer's Guide by Sourcery

This pull request introduces several new features and configurations to the project. It adds endpoints for file upload, listing, and deletion using Flask and Minio. Additionally, it includes various configuration files for documentation, code quality, testing, and continuous integration.

File-Level Changes

Files Changes
Makefile
docs/Makefile
docs/make.bat
Added Makefile and related files for building and managing the project and documentation.
TusStorageHandler/app.py
TusStorageHandler/tus_server.py
TusStorageHandler/minio_client.py
Added Flask application with endpoints for file operations and integrated Minio for object storage.
.github/actions/setup/poetry/action.yaml
.github/workflows/code_quality.yaml
.github/workflows/code_test.yaml
.github/workflows/pr_validation.yaml
.github/workflows/update.yaml
.github/workflows/docs.yaml
.github/workflows/vulnerability.yaml
.github/workflows/release.yaml
Added various GitHub Actions and Workflows for CI/CD, code quality, testing, and security.
CODE_OF_CONDUCT.md
PULL_REQUEST_TEMPLATE.md
.github/ISSUE_TEMPLATE/general-purpose.md
Added community and contribution guidelines including Code of Conduct, PR template, and issue template.
docs/source/conf.py
.readthedocs.yaml
Added Sphinx configuration and Read the Docs configuration for generating and hosting documentation.
.safety-policy.yml
.pre-commit-config.yaml
.yamllint.yaml
Added configurations for safety checks, pre-commit hooks, and YAML linting.
tests/__init__.py
tests/test_integration/__init__.py
tests/test_unit/__init__.py
Added initial test package structure.

Tips
  • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
  • You can change your review settings at any time by accessing your dashboard:
    • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    • Change the review language;
  • You can always contact us if you have any questions or feedback.

Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @psankhe28 - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 3 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟡 Documentation: 3 issues found

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.

TusStorageHandler/app.py Outdated Show resolved Hide resolved
TusStorageHandler/minio_client.py Outdated Show resolved Hide resolved
.github/workflows/docs.yaml Show resolved Hide resolved
CODE_OF_CONDUCT.md Show resolved Hide resolved
CODE_OF_CONDUCT.md Show resolved Hide resolved
.github/ISSUE_TEMPLATE/general-purpose.md Show resolved Hide resolved
TusStorageHandler/tus_server.py Outdated Show resolved Hide resolved
TusStorageHandler/tus_server.py Outdated Show resolved Hide resolved
TusStorageHandler/app.py Outdated Show resolved Hide resolved
TusStorageHandler/tus_server.py Outdated Show resolved Hide resolved
@JaeAeich
Copy link
Collaborator

@psankhe28 dw about test and documentation CI, I'll fix that. for types, you need to install the types of the libraries you are using. Check the CI logs, or below.

Run poetry run mypy TusStorageHandler/
TusStorageHandler/tus_server.py:5: error: Cannot find implementation or library stub for module named "minio.error"  [import-not-found]
TusStorageHandler/minio_client.py:23: error: Cannot find implementation or library stub for module named "dotenv"  [import-not-found]
TusStorageHandler/minio_client.py:24: error: Cannot find implementation or library stub for module named "minio"  [import-not-found]
TusStorageHandler/app.py:7: error: Cannot find implementation or library stub for module named "flask"  [import-not-found]
TusStorageHandler/app.py:8: error: Library stubs not installed for "flask_cors"  [import-untyped]
TusStorageHandler/app.py:8: note: Hint: "python3 -m pip install types-Flask-Cors"
TusStorageHandler/app.py:8: note: (or run "mypy --install-types" to install all missing stub packages)
TusStorageHandler/app.py:9: error: Cannot find implementation or library stub for module named "minio.error"  [import-not-found]
TusStorageHandler/app.py:9: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
TusStorageHandler/app.py:[1](https://github.com/elixir-cloud-aai/tus-storagehandler/actions/runs/9723599468/job/26838898556?pr=1#step:4:1)2: error: Cannot find implementation or library stub for module named "werkzeug.utils"  [import-not-found]
Found 7 errors in 3 files (checked 3 source files)

@@ -0,0 +1,21 @@
{
"template": "D:\\open-source\\gsoc\\cookiecutter-python",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@uniqueg we need to merge and finalize the cookiecutter, when generated via gh, this shoul dbe gh link, that is not possible before merge.

.cruft.json Outdated Show resolved Hide resolved

import os

from dotenv import load_dotenv
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally think, using dotenv package, and letting python handle env variable is a good idea. I think environment variables should be handled by the "environment", expecting having a .env file with creds might be a bad idea.

What I do is I use either my bash/zsh{rc}, export the variable in the shell env or just use a tool that help me manage my env variables, like direnv. Not asking you to set them up, or change your workflow just think .env file might not be ideal imo.

thoughts @uniqueg ??

pyproject.toml Outdated Show resolved Hide resolved
tus_server = TusServer(minio_client, MINIO_BUCKET)


@app.route("/", methods=["GET"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're writing an API why not use FOCA??

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I plan to upgrade it using FOCA. First, I want to create a basic template to test it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this??

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of the requirements.txt file are not showing up in the list of modified files, but they are visible when viewing the file in the branch. I'm not sure why this is happening.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need requiements.txt, you should install the deps via poetry. Read the the guide.

@JaeAeich
Copy link
Collaborator

JaeAeich commented Jun 29, 2024

use snake_case in directory name.

Copy link
Member

@uniqueg uniqueg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot, @psankhe28. Unfortunately, this PR is way too big for us to review effectively. Please break it down into several smaller PRs:

  • 1 chore for .gitignore, CODE_OF_CONDUCT.md, PULL_REQUEST_TEMPLATE.md etc.
  • 1 'ci for CI workflows
  • 1 docs for docs stuff
  • 1 build for build system (pyproject.toml)
  • 3 feat for code (1 for each endpoint), each with unit tests and other corresponding changes, if applicable (e.g., doc/build updates); e.g., you don't wanna add dependencies (e.g., tus) to your previous build PR that you don't need at the time of adding the build system; only add them to the PR where they are needed

@uniqueg
Copy link
Member

uniqueg commented Jul 25, 2024

See #3 (comment) and the comment above, @JaeAeich & @psankhe28

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants