Merge branch 'main' of github.com:jason810496/pgmq-sqlalchemy

.github/CONTRIBUTING.md

@@ -0,0 +1,93 @@
+# Contributing
+
+Welcome to contribute to `pgmq-sqlalchemy` !  <br>
+This document will guide you through the process of contributing to the project.
+
+## How to Contribute
+
+1. Fork the repository
+    - Click the `Fork` button in the upper right corner of the repository page.
+2. Clone the repository
+    - Clone the repository to your local machine.
+    ```bash
+    git clone https://github.com/your-username/pgmq-sqlalchemy.git
+    ```
+3. Create a new branch
+    - Create a new branch for your changes.
+    ```bash
+    git checkout -b feature/your-feature-name
+    ```
+4. Make your changes
+    - Make your changes to the codebase.
+    - Add tests for your changes.
+    - Add documentation if changes are user-facing.
+5. Commit your changes
+    - Commit your changes with meaningful commit messages.
+        - [ref: conventional git commit messages](https://www.conventionalcommits.org/en/v1.0.0/)
+    ```bash
+    git commit -m "feat: your feature description"
+    ```
+6. Push your changes
+    - Push your changes to your forked repository.
+    ```bash
+    git push origin feature/your-feature-name
+    ```
+7. Create a Pull Request
+    - Create a Pull Request from your forked repository to the `develop` branch of the original repository.
+
+## Development
+
+### Setup
+
+Install dependencies and `ruff` pre-commit hooks.
+```bash
+make install
+```
+
+> Prerequisites: **Docker** and **Docker Compose** installed.
+
+Start development PostgreSQL
+```bash
+make start-db
+```
+
+Stop development PostgreSQL
+```bash
+make stop-db
+```
+
+### Makefile utility
+
+```bash
+make help
+```
+> will show all available commands and their descriptions.
+
+### Linting 
+
+We use [pre-commit](https://pre-commit.com/) hook with [ruff](https://github.com/astral-sh/ruff-pre-commit) to automatically lint the codebase before committing.
+
+
+### Testing
+
+Run tests in local
+```bash
+make test-local
+```
+
+Run tests in docker
+```bash
+make test-docker
+```
+
+### Documentation
+
+Serve documentation
+```bash
+make doc-serve
+```
+
+Clean documentation build
+```bash
+make doc-clean
+```

.github/pull_request_template.md

@@ -0,0 +1,19 @@
+# Title (replace this with the title of your pull request)
+
+## Description
+
+Describe the changes you made in this pull request. <br>
+( replace this with the description of your pull request )
+
+## Status
+
+- [ ] In progress
+- [ ] Ready for review
+- [ ] Done
+
+## Checklist
+
+- [ ] Read the [Contributing Guide](CONTRIBUTING.md)
+- [ ] Passes tests
+- [ ] Linted ( we use `pre-commit` with `ruff` )
+- [ ] Updated documentation

.github/workflows/codecov.yml

@@ -14,7 +14,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: ["3.12"]
+        python-version: ["3.9","3.10","3.11","3.12"]
 
     name: Test pgmq-sqlalchemy
     steps:
@@ -31,7 +31,7 @@ jobs:
           run: |
             poetry env use python${{ matrix.python-version }}
         - name: Install dependencies
-          run: poetry install
+          run: poetry install --without=dev
         - name: Start PostgreSQL
           run: |
             cp pgmq_postgres.template.env pgmq_postgres.env

.github/workflows/publish.yml

@@ -0,0 +1,46 @@
+name: Release
+
+on:
+  push:
+    tags:
+      - '*.*.*'
+
+permissions:
+  contents: read
+
+jobs:
+  pypi-publish:
+    name: Upload release to PyPI
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/project/pgmq-sqlalchemy/
+    permissions:
+      id-token: write
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.10"
+
+      - name: Install Poetry
+        run: |
+          curl -sSL https://install.python-poetry.org | python - -y
+
+      - name: Update PATH
+        run: echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      - name: Update Poetry configuration
+        run: poetry config virtualenvs.create false
+
+      - name: Install dependencies
+        run: poetry install --sync --no-interaction --without=dev
+
+      - name: Package project
+        run: poetry build
+
+      - name: Publish package distributions to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

Makefile

@@ -4,6 +4,9 @@ install: ## Install dependencies and `ruff` pre-commit hooks
 	pre-commit install
 	poetry install --with dev
 
+build: ## Build the package
+	poetry build
+
 test-local: ## Run tests locally
 	poetry run pytest tests --cov=pgmq_sqlalchemy.queue
 
@@ -37,9 +40,12 @@ exec-db: ## Enter the database container
 doc-build: ## Build the documentation
 	cd doc && poetry run sphinx-build -nW . _build
 
-doc-serve: ## Serve the documentation
+doc-serve: doc-clean ## Serve the documentation
 	cd doc && poetry run sphinx-autobuild -nW . _build
 
+doc-clean: ## Clean the documentation
+	cd doc && rm -r _build
+
 .PHONY: install test-local test-docker test-docker-rebuild clear-db start-db exec-db doc-build doc-serve
 
 # generate help from comments

README.md

@@ -8,16 +8,117 @@
 
 # pgmq-sqlalchemy
 
-Python client using **sqlalchemy ORM** for the PGMQ Postgres extension.
+More flexible [PGMQ Postgres extension](https://github.com/tembo-io/pgmq) Python client that using **sqlalchemy ORM**, supporting both **async** and **sync** `engines`, `sessionmakers` or built from `dsn`.
+
+## Table of Contents
+
+* [pgmq-sqlalchemy](#pgmq-sqlalchemy)
+   * [Features](#features)
+   * [Installation](#installation)
+   * [Getting Started](#getting-started)
+      * [Postgres Setup](#postgres-setup)
+      * [Usage](#usage)
+   * [Issue/ Contributing / Development](#issue-contributing--development)
+   * [TODO](#todo)
 
-支援 **SQLAlchemy ORM** 的 Python 客戶端 <br>
-用於 [PGMQ Postgres 插件](https://github.com/tembo-io/pgmq) 。
 
 ## Features
 
-- 支援 **async** 和 **sync** `engines`、`sessionmakers`，或由 `dsn` 構建。
-- 支援所有 sqlalchemy 支持的 postgres DBAPIs。
-    > 例如：`psycopg`, `psycopg2`, `asyncpg`
-    > 可見 [SQLAlchemy Postgresql Dialects](https://docs.sqlalhttps://docs.sqlalchemy.org/en/20/dialects/postgresql.html)
+- Supports **async** and **sync** `engines` and `sessionmakers`, or built from `dsn`.
+- **Automatically** creates `pgmq` (or `pg_partman`) extension on the database if not exists.
+- Supports **all postgres DBAPIs supported by sqlalchemy**.
+    > e.g. `psycopg`, `psycopg2`, `asyncpg` .. <br>
+    > See [SQLAlchemy Postgresql Dialects](https://docs.sqlalhttps://docs.sqlalchemy.org/en/20/dialects/postgresql.html)
+
+## Installation
+
+Install with pip:
+
+```bash
+pip install pgmq-sqlalchemy
+```
+
+Install with additional DBAPIs packages:
+
+```bash
+pip install pgmq-sqlalchemy[psycopg2]
+pip install pgmq-sqlalchemy[asyncpg]
+# pip install pgmq-sqlalchemy[postgres-python-driver]
+```
+
+## Getting Started
+
+### Postgres Setup
+
+Prerequisites: **Postgres** with **PGMQ** extension installed. <br>
+For quick setup: 
+```bash
+docker run -d --name postgres -e POSTGRES_PASSWORD=postgres -p 5432:5432 quay.io/tembo/pg16-pgmq:latest
+```
+> For more information, see [PGMQ](https://github.com/tembo-io/pgmq)
+
+### Usage
+
+> [!NOTE]  
+> Check [pgmq-sqlalchemy Document](https://pgmq-sqlalchemy-python.readthedocs.io/en/latest/) for more examples and detailed usage.
+
+
+For `dispatcher.py`:
+```python
+from typing import List
+from pgmq_sqlalchemy import PGMQueue
+
+postgres_dsn = 'postgresql://postgres:postgres@localhost:5432/postgres'
+
+pgmq = PGMQueue(dsn=postgres_dsn)
+pgmq.create_queue('my_queue')
+
+msg = {'key': 'value', 'key2': 'value2'}
+msg_id:int = pgmq.send('my_queue', msg)
+
+# could also send a list of messages
+msg_ids:List[int] = pgmq.send_batch('my_queue', [msg, msg])
+```
+
+For `consumer.py`:
+```python
+from pgmq_sqlalchemy import PGMQueue
+from pgmq_sqlalchemy.schema import Message
+
+postgres_dsn = 'postgresql://postgres:postgres@localhost:5432/postgres'
+
+pgmq = PGMQueue(dsn=postgres_dsn)
+
+# read a single message
+msg:Message = pgmq.read('my_queue')
+
+# read a batch of messages
+msgs:List[Message] = pgmq.read_batch('my_queue', 10)
+```
+
+For `monitor.py`:
+```python
+from pgmq_sqlalchemy import PGMQueue
+from pgmq_sqlalchemy.schema import QueueMetrics
+
+postgres_dsn = 'postgresql://postgres:postgres@localhost:5432/postgres'
+
+pgmq = PGMQueue(dsn=postgres_dsn)
+
+# get queue metrics
+metrics:QueueMetrics = pgmq.metrics('my_queue')
+print(metrics.queue_length)
+print(metrics.total_messages)
+```
+
+## Issue/ Contributing / Development 
+
+Welcome to open an issue or pull request ! <br>
+See [`Development` on Online Document](https://pgmq-sqlalchemy-python.readthedocs.io/en/latest/) or [CONTRIBUTING.md](.github/CONTRIBUTING.md) for more information.
 
+## TODO 
 
+- [ ] Add **time-based** partition option and validation to `create_partitioned_queue` method.
+- [ ] Read(single/batch) Archive Table ( `read_archive` method )
+- [ ] Detach Archive Table ( `detach_archive` method )
+- [ ] Add `set_vt` utils method.

doc/api-reference.rst

@@ -0,0 +1,31 @@
+.. _api-reference:
+
+API Reference
+=============
+
+.. tip::
+    | For a more detailed explanation or implementation of each `PGMQ function`,
+    | see `PGMQ: SQL functions.md <https://github.com/tembo-io/pgmq/blob/main/docs/api/sql/functions.md>`_.
+
+
+.. autoclass:: pgmq_sqlalchemy.PGMQueue
+    :members:
+    :inherited-members:
+    :member-order: bysource
+    :special-members: __init__
+
+
+.. autoclass:: pgmq_sqlalchemy.schema.Message
+    :members:
+    :undoc-members:
+    :inherited-members:
+    :exclude-members: __init__
+
+
+
+.. autoclass:: pgmq_sqlalchemy.schema.QueueMetrics
+    :members:
+    :undoc-members:
+    :inherited-members:
+    :exclude-members: __init__
+

doc/conf.py

@@ -7,6 +7,7 @@
 import sys
 
 # path setup
+sys.path.insert(0, os.path.abspath(".."))
 sys.path.insert(0, os.path.abspath("../pgmq_sqlalchemy"))
 
 extensions = [