Adding support for ntext datatype

tap-mssql 2.6.5 2024-10-24

• Adding support for ntext to list of valid STRING_TYPES.

Support timestamp and rowversion datatypes

tap-mssql 2.6.4 2024-10-24

• Update to handle timestamp (not a datetime value, a deprecated synonym of internal rowversion) as string
• Add tests for incremental syncing using a timestamp column as replication-key

Updating CDC documentation

tap-mssql 2.6.3 2024-10-17

• Updating CDC documentation with a packaged method to maintain CDC tables.

Resolving issue with PK and UNIQUE constraints, overstating target primary key

tap-mssql 2.6.2 2024-10-09

• Resolving issue when a table has a primary key and unique key. Both unique and primary key
  columns were being identified as the primary key for the target table. Prioritising the
  primary key first, and unique key secondary if there is no primary key.

Resolving issue with obtain prior LSN number

tap-mssql 2.6.1 2024-10-09

• Resolving issue with call get the prior LSN number (passing in an unescaped table).

Adding new config settings conn_properties and enable_tds_logging

New setting for the wintersrd variant of tap-mssql.

Optional:

The "conn_properties" allows specific tweaking of database settings via SQL set statements to send to the database instance upon connection establishment. Can be a string or another kind of iterable of strings.

The default values set if this settings is not defined are:

"SET ARITHABORT ON; SET CONCAT_NULL_YIELDS_NULL ON; SET ANSI_NULLS ON; SET ANSI_NULL_DFLT_ON ON; SET ANSI_PADDING ON; SET ANSI_WARNINGS ON; SET ANSI_NULL_DFLT_ON ON; SET CURSOR_CLOSE_ON_COMMIT ON; SET QUOTED_IDENTIFIER ON; SET TEXTSIZE 2147483647;"

Example: override the built-in session properties supplied by pymssql by default, because one of the default settings (CURSOR_CLOSE_ON_COMMIT) is not available on PDW

{
  "conn_properties": "SET ARITHABORT ON; SET CONCAT_NULL_YIELDS_NULL ON; SET ANSI_NULLS ON; SET ANSI_NULL_DFLT_ON ON; SET ANSI_PADDING ON; SET ANSI_WARNINGS ON; SET ANSI_NULL_DFLT_ON ON; SET QUOTED_IDENTIFIER ON; SET TEXTSIZE 2147483647;"
}

Optional:

The "enable_tds_logging" When set it will dump out the underlying TDS driver logs. Useful for diagnosing issues if you are having connection issues to SQL Server databases. WARNING! this
does dump a lot of information and may log secure data, should be only used in Development
environments.

{
  "enable_tds_logging": true
}

Patching tap-mssql to remove CVES and bumping to latest versions

Patching tap-mssql to work with the latest versions of the pytest components. This is required to address CVES identified by dependabot.

tap-mssql 2.5.0 2024-08-26

• removing tox-poetry-install - Not well supported and not required for the pipelines.
• Updating integration tests variables to CAPS to support passing MSSQL Port
• tox-docker requires expose rather than ports syntax in tox.ini
• Bumping docker = "^6.1.3" -> "^7.1.0"
• Bumping pytest = "^7.0.0" -> "^8.3.2"
• Bumping pytest-cov = "^4.1.0" -> = "^5.0.0"
• Bumping mypy = "^1.10.1" -> = "^1.11.1"
• Bumping pytest-sugar = "^0.9.7" -> = "^1.0.0"
• Bumping pytest-datafiles = "^2.0" -> = "^3.0"
• Bumping tox-docker = "4.1.0" -> = "5.0.0"
• Bumping requests = "2.31.0" -> "2.32.3"
• Bumping tox = "^3.28.0" -> "^4.18.0"

Local testing has been as well as the integration testing to ensure the tap-mssql application still works correctly.

Patching tap-mssql (highlight singer-python with improved speed)

tap-mssql 2.4.0 2024-08-08

• Moving to patched version of Singer Framework plus using msgspec rather than orjson for JSON serialization speed.
• Adjusting tox file for test pipeline to point to correct location of the sqlcmd in the latest MSSQL docker image.
• Adjusting tox file. Added -No switch calling sqlcmd to allow user/password connection to MSSQL.
• Patching dependencies.
• Pinned Requests to "2.31.0". This is required as the tox pipeline fails with a Docker API error with requests "2.32.x".
• Explicitly setting python version 3.8 -> 3.12
• Moving to using recommended poetry-core for build. https://python-poetry.org/docs/pyproject/#poetry-and-pep-517
• NOTE: Recommend refactoring to avoid using tox-poetry-installer as restrictive dependencies are stopping tox and docker updates. This may resolve the requests/docker api issue.

Bug Fix: Change to preserve column ordering as per source

tap-mssql 2.3.1 2024-07-22

Bug Fix. Issue wintersrd#62 - change to column selection using lists instead of sets to preserve column ordering

Resolving pendulum datetime issue for incremental loads

Bug Fix. Change pendulum DateTime type to datetime.datetime as pymssql 2.3.0 is no longer compatible with query parameters as pendulum DateTime (pymssql/pymssql#889).