Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model build time unapply #13

Open
robscriva opened this issue Sep 13, 2021 · 1 comment
Open

Model build time unapply #13

robscriva opened this issue Sep 13, 2021 · 1 comment
Labels
enhancement New feature or request

Comments

@robscriva
Copy link
Contributor

Currently for the unapply operation in the apply_masking_policy_list_for_models macro it loops through the nodes in the graph. Perhaps this could be decoupled to allow a pre-hook unapply to the model as well.

So there would be two approaches to applying policies:

  • unapply / apply in pre-post hook steps
  • unapply / apply in batch via run-operations or on-run-start/end
@entechlog entechlog added the enhancement New feature or request label Sep 14, 2021
@entechlog
Copy link
Owner

@robscriva , Sorry it took so long to get back to you on this one

  • Just was testing this out and unapply works in post/pre-hooks
  • Here are the steps I followed, added post-hook
  post-hook: 
    - "{{ dbt_snow_mask.unapply_masking_policy('models') }}"
  • Executed the models by dbt run and it ran fine removing the policy
00:17:54  Running with dbt=1.0.1
00:17:54  Unable to do partial parsing because a project config has changed
00:18:00  Found 1 model, 0 tests, 0 snapshots, 0 analyses, 404 macros, 0 operations, 1 seed file, 1 source, 0 exposures, 0 metrics
00:18:00  
00:18:04  Concurrency: 10 threads (target='dev')
00:18:04  
00:18:04  1 of 1 START view model staging.stg_customer.................................... [RUN]
00:18:05  00:18:05 | creating masking policy           : DEMO_DB.STAGING.MP_ENCRYPT_PII
00:18:06  00:18:06 | unapplying masking policy to model  : DEMO_DB.STAGING.MP_ENCRYPT_PII on DEMO_DB.staging.stg_customer.email
00:18:07  1 of 1 OK created view model staging.stg_customer............................... [SUCCESS 1 in 2.52s]
00:18:07  
00:18:07  Finished running 1 view model in 7.09s.
00:18:07  
00:18:07  Completed successfully
00:18:07  
00:18:07  Done. PASS=1 WARN=0 ERROR=0 SKIP=0 TOTAL=1
  • Did I miss something in understanding your request ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants