From b840229a6555e9f485f97a126a3dcca87f004220 Mon Sep 17 00:00:00 2001 From: Serge Smertin <259697+nfx@users.noreply.github.com> Date: Mon, 25 Mar 2024 22:12:09 +0100 Subject: [PATCH] Release v0.2.4 (#69) * Fixed `Builder` object is not callable error ([#67](https://github.com/databrickslabs/lsql/issues/67)). In this release, we have made an enhancement to the `Backends` class in the `databricks/labs/lsql/backends.py` file. The `DatabricksSession.builder()` method call in the `__init__` method has been changed to `DatabricksSession.builder`. This update uses the `builder` attribute to create a new instance of `DatabricksSession` without calling it like a function. The `sdk_config` method is then used to configure the instance with the required settings. Finally, the `getOrCreate` method is utilized to obtain a `SparkSession` object, which is then passed as a parameter to the parent class constructor. This modification simplifies the code and eliminates the error caused by treating the `builder` attribute as a callable object. Software engineers may benefit from this change by having a more streamlined and error-free codebase when working with the open-source library. * Prevent silencing of `pylint` ([#65](https://github.com/databrickslabs/lsql/issues/65)). In this release, we have introduced a new job, "no-lint-disabled", to the GitHub Actions workflow for the repository. This job runs on the latest Ubuntu version and checks out the codebase with a full history. It verifies that no new instances of code suppressing `pylint` checks have been added, by filtering the differences between the current branch and the main branch for new lines of code, and then checking if any of those new lines contain a `pylint` disable comment. If any such lines are found, the job will fail and print a message indicating the offending lines of code, thereby ensuring that the codebase maintains a consistent level of quality by not allowing linting checks to be bypassed. * Updated `_SparkBackend.fetch()` to return iterator instead of list ([#62](https://github.com/databrickslabs/lsql/issues/62)). In this release, the `fetch()` method of the `_SparkBackend` class has been updated to return an iterator instead of a list, which can result in reduced memory usage and improved performance, as the results of the SQL query can now be processed one element at a time. A new exception has been introduced to wrap any exceptions that occur during query execution, providing better debugging and error handling capabilities. The `test_runtime_backend_fetch()` unit test has been updated to reflect this change, and users of the `fetch()` method should be aware that it now returns an iterator and must be consumed to obtain the desired data. Thorough testing is recommended to ensure that the updated method still meets the needs of the application. --- CHANGELOG.md | 7 +++++++ src/databricks/labs/lsql/__about__.py | 2 +- 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0f14f9ba..f3834da0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Version changelog +## 0.2.4 + +* Fixed `Builder` object is not callable error ([#67](https://github.com/databrickslabs/lsql/issues/67)). In this release, we have made an enhancement to the `Backends` class in the `databricks/labs/lsql/backends.py` file. The `DatabricksSession.builder()` method call in the `__init__` method has been changed to `DatabricksSession.builder`. This update uses the `builder` attribute to create a new instance of `DatabricksSession` without calling it like a function. The `sdk_config` method is then used to configure the instance with the required settings. Finally, the `getOrCreate` method is utilized to obtain a `SparkSession` object, which is then passed as a parameter to the parent class constructor. This modification simplifies the code and eliminates the error caused by treating the `builder` attribute as a callable object. Software engineers may benefit from this change by having a more streamlined and error-free codebase when working with the open-source library. +* Prevent silencing of `pylint` ([#65](https://github.com/databrickslabs/lsql/issues/65)). In this release, we have introduced a new job, "no-lint-disabled", to the GitHub Actions workflow for the repository. This job runs on the latest Ubuntu version and checks out the codebase with a full history. It verifies that no new instances of code suppressing `pylint` checks have been added, by filtering the differences between the current branch and the main branch for new lines of code, and then checking if any of those new lines contain a `pylint` disable comment. If any such lines are found, the job will fail and print a message indicating the offending lines of code, thereby ensuring that the codebase maintains a consistent level of quality by not allowing linting checks to be bypassed. +* Updated `_SparkBackend.fetch()` to return iterator instead of list ([#62](https://github.com/databrickslabs/lsql/issues/62)). In this release, the `fetch()` method of the `_SparkBackend` class has been updated to return an iterator instead of a list, which can result in reduced memory usage and improved performance, as the results of the SQL query can now be processed one element at a time. A new exception has been introduced to wrap any exceptions that occur during query execution, providing better debugging and error handling capabilities. The `test_runtime_backend_fetch()` unit test has been updated to reflect this change, and users of the `fetch()` method should be aware that it now returns an iterator and must be consumed to obtain the desired data. Thorough testing is recommended to ensure that the updated method still meets the needs of the application. + + ## 0.2.3 * Added support for common parameters in StatementExecutionBackend ([#59](https://github.com/databrickslabs/lsql/issues/59)). The `StatementExecutionBackend` class in the `databricks.labs.lsql` package's `backends.py` file now supports the passing of common parameters through keyword arguments (kwargs). This enhancement allows for greater customization and flexibility in the backend's operation, as the kwargs are passed to the `StatementExecutionExt` constructor. This change empowers users to control the behavior of the backend, making it more adaptable to various use cases. The key modification in this commit is the addition of the `**kwargs` parameter in the constructor signature and passing it to `StatementExecutionExt`, with no changes made to any methods within the class. diff --git a/src/databricks/labs/lsql/__about__.py b/src/databricks/labs/lsql/__about__.py index d31c31ea..788da1fb 100644 --- a/src/databricks/labs/lsql/__about__.py +++ b/src/databricks/labs/lsql/__about__.py @@ -1 +1 @@ -__version__ = "0.2.3" +__version__ = "0.2.4"