Skip to content

Commit

Permalink
Apply suggestions from code review
Browse files Browse the repository at this point in the history
  • Loading branch information
msmygit authored Sep 12, 2024
1 parent 7cfe5d5 commit 7f0a8d2
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 5 deletions.
4 changes: 2 additions & 2 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Release Notes
## [4.3.10] - 2024-09-11
## [4.3.10] - 2024-09-12
- Added property `spark.cdm.trackRun.runId` to support a custom unique identifier for the current run. This can be used by wrapper scripts to pass a known `runId` and then use it to query the `cdm_run_info` and `cdm_run_details` tables.

## [4.3.9] - 2024-09-11
- Added new `status` value of `DIFF_CORRECTED` on `cdm_run_details` table to specifically mark partitions that were corrected during the CDM validation run.
- Upgraded Validation job skip partitions with `DIFF_CORRECTED` status on rerun with a previous `runId`.
- Upgraded Validation job to skip partitions with `DIFF_CORRECTED` status on rerun with a previous `runId`.

## [4.3.8] - 2024-09-09
- Upgraded `spark.cdm.trackRun` feature to include `status` on `cdm_run_info` table. Also improved the code to handle rerun of previous run which may have exited before being correctly initialized.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,8 +68,8 @@ public TargetUpsertRunDetailsStatement(CqlSession session, String keyspaceTable)
// ignore if column already exists
logger.trace("Column 'status' already exists in table {}", cdmKsTabInfo);
}
this.session.execute("create table if not exists " + cdmKsTabDetails
+ " (table_name text, run_id bigint, start_time timestamp, token_min bigint, token_max bigint, status text, primary key ((table_name, run_id), token_min))");
this.session.execute("CREATE TABLE IF NOT EXISTS " + cdmKsTabDetails
+ " (table_name TEXT, run_id BIGINT, start_time TIMESTAMP, token_min BIGINT, token_max BIGINT, status TEXT, PRIMARY KEY ((table_name, run_id), token_min))");

boundInitInfoStatement = bindStatement("INSERT INTO " + cdmKsTabInfo
+ " (table_name, run_id, run_type, prev_run_id, start_time, status) VALUES (?, ?, ?, ?, dateof(now()), ?)");
Expand Down
2 changes: 1 addition & 1 deletion src/resources/cdm-detailed.properties
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ spark.cdm.autocorrect.missing.counter false
# reasons including if the job get killed, or some token-ranges fail due to load on
# the cluster (origin or target) or any other reasons.
# .runId : Default is an auto generated unique long value. When a non-zero value is provided,
# it will be used a custom and unique identifier for the current run. Note the value
# it will be used as a custom and unique identifier for the current run. Note the value
# of this id must be numeric and can be any java `long` unique value. This can be used
# by wrapper scripts to pass a known `runId` and then use it to query the
# `cdm_run_info` and `cdm_run_details` tables.
Expand Down

0 comments on commit 7f0a8d2

Please sign in to comment.