Skip to content

Releases: GoogleCloudDataproc/spark-bigquery-connector

0.41.0

06 Sep 03:12
Compare
Choose a tag to compare
  • PR #1265 : Add additional parentheses for EqualNullSafe filter generation. Thanks @tom-s-powell !
  • PR #1267 : Implement OpenLineage spark-extension-interfaces
  • PR #1281 : Configure alternative BigNumeric precision and scale defaults
  • Issue #1175: Add details to schema mismatch message
  • BigQuery API has been upgraded to version 2.42.2
  • BigQuery Storage API has been upgraded to version 3.9.0
  • GAX has been upgraded to version 2.52.0
  • Netty has been upgraded to version 4.1.113.Final

0.40.0

06 Aug 08:04
Compare
Choose a tag to compare
  • PR #1259 : Encode snapshotTimeMillis in view materialization query. Thanks @tom-s-powell !
  • PR #1261 : Adding IdentityToken header in readRows call
  • Issue #1043 : Fix Indirect write drops policy tags
  • Issue #1244 : Set schema Field Nullables as per ALLOW_FIELD_RELAXATION
  • Issue #1254 : fix getting partitioning fields for pseudo columns
  • Issue #1263 : Support ClusteredFields in Direct Write
  • BigQuery API has been upgraded to version 2.42.0
  • BigQuery Storage API has been upgraded to version 3.8.0
  • GAX has been upgraded to version 2.51.0
  • gRPC has been upgraded to version 1.65.1
  • Netty has been upgraded to version 4.1.112.Final

0.36.4

24 Jul 23:08
Compare
Choose a tag to compare
  • PR #1261: Adding IdentityToken header in readRows call

0.39.1

24 Jun 23:06
Compare
Choose a tag to compare
  • PR #1236: Fixing unshaded artifacts, added shading verification
  • PR #1239: Allow GCS bucket to be supplied including a scheme. Thanks @tom-s-powell !
  • Issue #1126: Fixing Kryo serialization issues
  • Issue #1223: Fix gRPC status creates 'object not serializable' errors

0.36.3

30 May 20:25
Compare
Choose a tag to compare
  • PR #1156: Propagate stats for BigLake Managed tables
  • PR #1205: Sending Identity token in the read API header
  • PR #1212: Support OpenLineage in spark-3.x-bigquery connectors
  • PR #1227: Optimizing dynamic partition overwrite for Time partitioned tables

0.39.0

21 May 20:30
Compare
Choose a tag to compare
  • PR #1221: Adding support for table snapshots. Thanks @tom-s-powell !
  • PR #1222: Add option to request lz4 compressed ReadRowsResponse
  • PR #1225: Fixing multi-release jar shading
  • PR #1227: Optimizing dynamic partition overwrite for Time partitioned tables
  • BigQuery API has been upgraded to version 2.40.1
  • BigQuery Storage API has been upgraded to version 3.5.1
  • GAX has been upgraded to version 2.48.1
  • gRPC has been upgraded to version 1.64.0

0.36.2

07 May 21:28
Compare
Choose a tag to compare
  • Issue #1215: Support predicate pushdown for DATETIME

0.38.0

02 May 03:43
Compare
Choose a tag to compare
  • PR #1205: Sending Identity token in the read API header
  • Issue #1195: Support map type with complex value
  • Issue #1215: Support predicate pushdown for DATETIME
  • BigQuery API has been upgraded to version 2.39.0
  • BigQuery Storage API has been upgraded to version 3.5.0
  • GAX has been upgraded to version 2.47.0
  • Arrow has been upgraded to version 16.0.0
  • gRPC has been upgraded to version 1.63.0
  • Netty has been upgraded to version 4.1.109.Final

0.37.0

25 Mar 22:44
Compare
Choose a tag to compare
  • ⚠️ Starting version 0.38.0 of the connector, the spark-2.4-bigquery version won't be released as Spark 2.4 is
    well-supported by the spark-bigquery-with-dependencies connectors.
  • PR #1156: Propagate stats for BigLake Managed tables
  • PR #1181: Add caching during protobuf generation
  • PR #1190: Enable connection sharing for atLeastOnce writes
  • Issue #1182: Fix query check logic
  • BigQuery API has been upgraded to version 2.38.1
  • BigQuery Storage API has been upgraded to version 3.3.1
  • GAX has been upgraded to version 2.45.0
  • Arrow has been upgraded to version 15.0.1
  • gRPC has been upgraded to version 1.62.2
  • Netty has been upgraded to version 4.1.107.Final
  • Protocol Buffers has been upgraded to version 3.25.3

0.36.1

01 Feb 05:38
Compare
Choose a tag to compare
  • PR #1176: fix timestamp filter translation issue