upstream pull #8
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.3 to 3.0.1. - [Release notes](https://github.com/pallets/jinja/releases) - [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst) - [Commits](pallets/jinja@2.11.3...3.0.1) --- updated-dependencies: - dependency-name: jinja2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Peter Kosztolanyi <[email protected]>
…tync pairing logic (transferwise#787)
* downgrade setuptools to support use_2to3 * fixed install method
* introducing makefile * updated docker files * updated documentations and scripts to use makefile * updated workflow * update workflow to install only connectors * fixed wrong place of character * removed unnecessary target for all_connectors * fixed to support ubuntu 18 * fixed list variables to be compatibale with ubuntu 18 * updated ci check * updated help message
* added json properties validator * fixed pep8 * fixed mocked object * refactoring * refactoring testcase * fixed lint * Update tests/units/cli/test_commands.py Co-authored-by: Samira El Aabidi <[email protected]> * using unittest mock * separate tests * refactored unit test * refactored tests * fixed annotations Co-authored-by: Samira El Aabidi <[email protected]>
Co-authored-by: Jeet Parekh <[email protected]> Co-authored-by: Samira El Aabidi <[email protected]>
* release v0.41.0 * Update CHANGELOG.md * parameterise python in makefile * Update CHANGELOG.md Co-authored-by: Amir Mofakhar <[email protected]> * Update Makefile Co-authored-by: Amir Mofakhar <[email protected]>
* improve OS signal handling * change tests * remove commented code for terminating main process
* Improve signal handling Only the `bash` process which PipelineWise creates needs to be terminated. Any other processes are not PipelineWise's responsibility to kill. * Move log rename outside `try...except` * Make sure pidfile is removed * Guarentee the correct process is killed * Remove temp files on exit * Use `pgid` to determine which children to kill * Add recursive killing back in
* [AP-1109] Fix docker build process in github actions * Update dockerhub.yml
* [AP-1123] run linting in docker * [AP-1123] add __name__ check
…lity in favor of internally implemented version
amadigan
commented
Mar 2, 2022
| @@ -181,42 +186,37 @@ def load_yaml(yaml_file, vault_secret=None): | |||
| secret_file.load() | |||
| vault.secrets = [('default', secret_file)] | |||
|
|
|||
| # YAML ENV VAR | |||
There was a problem hiding this comment.
There was a conflict in this file. It looks like the way the YAML configuration is loaded was changed. The changes we made were to enable reading environment variables when loading the config, but it seems pipelinewise has added equivalent functionality internally (https://transferwise.github.io/pipelinewise/installation_guide/creating_pipelines.html#environment-variables-in-yaml-config). I'm submitting a PR to change the syntax in datawarehouse.
There was a problem hiding this comment.
Just to be clear, this file is now identical to upstream, I removed any changes we had made.
amadigan
commented
Mar 2, 2022
Comment on lines
-201
to
+209
| file_data = stream.read() | ||
| data = yaml.load(file_data, Loader=yaml.Loader) | ||
|
|
||
| ''' | ||
| Commenting code below for posterity. We are not using ansible functionality but yaml load should | ||
| follow the same code path regardless of the file encryption state. | ||
| ''' | ||
| #loader = AnsibleLoader(stream, None, vault.secrets) | ||
| #try: | ||
| # data = loader.get_single_data() | ||
| #except Exception as exc: | ||
| # raise Exception(f'Error when loading YAML config at {yaml_file} {exc}') from exc | ||
| #finally: | ||
| # loader.dispose() | ||
| loader = AnsibleLoader(stream, None, vault.secrets) | ||
| try: | ||
| data = loader.get_single_data() | ||
| except Exception as exc: | ||
| raise Exception( | ||
| f'Error when loading YAML config at {yaml_file} {exc}' | ||
| ) from exc | ||
| finally: | ||
| loader.dispose() |
There was a problem hiding this comment.
This is where the conflict was.
jonathansu
approved these changes
Mar 2, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pipelinewise-target-bigquery(Updatepipelinewise-target-bigquerytransferwise/pipelinewise#860)fastsync(Fix broken Postgres to BigQueryfastsynctransferwise/pipelinewise#861)NamedTupletodataclass(ConvertNamedTupletodataclasstransferwise/pipelinewise#889)split_file_max_chunks(Relax upper limit onsplit_file_max_chunkstransferwise/pipelinewise#897)ansiblewithansible-core(Replaceansiblewithansible-coretransferwise/pipelinewise#900)