Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Examples #5206

Merged
merged 6 commits into from
Apr 26, 2024
Merged

Fixing Examples #5206

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1a55205
Fixing Examples
matthewshaver Apr 1, 2024
772bf90
More examples
matthewshaver Apr 3, 2024
d622522
Merge branch 'current' into examples
runleonarun Apr 3, 2024
8c07ef6
Update hive-setup.md
runleonarun Apr 3, 2024
6d0f168
Merge branch 'current' into examples
runleonarun Apr 3, 2024
d92f25e
Merge branch 'current' into examples
matthewshaver Apr 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 18 additions & 18 deletions website/docs/docs/core/connect-data-platform/databricks-setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,12 +53,12 @@ your_profile_name:
outputs:
dev:
type: databricks
catalog: [optional catalog name if you are using Unity Catalog]
schema: [schema name] # Required
host: [yourorg.databrickshost.com] # Required
http_path: [/sql/your/http/path] # Required
token: [dapiXXXXXXXXXXXXXXXXXXXXXXX] # Required Personal Access Token (PAT) if using token-based authentication
threads: [1 or more] # Optional, default 1
catalog: CATALOG_NAME #optional catalog name if you are using Unity Catalog]
schema: SCHEMA_NAME # Required
host: YOURORG.databrickshost.com # Required
http_path: /SQL/YOUR/HTTP/PATH # Required
token: dapiXXXXXXXXXXXXXXXXXXXXXXX # Required Personal Access Token (PAT) if using token-based authentication
threads: 1_OR_MORE # Optional, default 1
```

</File>
Expand All @@ -76,14 +76,14 @@ your_profile_name:
outputs:
dev:
type: databricks
catalog: [optional catalog name if you are using Unity Catalog]
schema: [schema name] # Required
host: [yourorg.databrickshost.com] # Required
http_path: [/sql/your/http/path] # Required
catalog: CATALOG_NAME #optional catalog name if you are using Unity Catalog
schema: SCHEMA_NAME # Required
host: YOUR_ORG.databrickshost.com # Required
http_path: /SQL/YOUR/HTTP/PATH # Required
auth_type: oauth # Required if using OAuth-based authentication
client_id: [OAuth-Client-ID] # The ID of your OAuth application. Required if using OAuth-based authentication
client_secret: [XXXXXXXXXXXXXXXXXXXXXXXXXXX] # OAuth client secret. # Required if using OAuth-based authentication
threads: [1 or more] # Optional, default 1
client_id: OAUTH_CLIENT_ID # The ID of your OAuth application. Required if using OAuth-based authentication
client_secret: XXXXXXXXXXXXXXXXXXXXXXXXXXX # OAuth client secret. # Required if using OAuth-based authentication
threads: 1_OR_MORE # Optional, default 1
```
</File>
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved

Expand All @@ -97,20 +97,20 @@ The following profile fields are always required.

| Field | Description | Example |
| --------- | ------- | ----------- |
| `host` | The hostname of your cluster.<br/><br/>Don't include the `http://` or `https://` prefix. | `yourorg.databrickshost.com` |
| `http_path` | The http path to your SQL Warehouse or all-purpose cluster. | `/sql/your/http/path` |
| `schema` | The name of a schema within your cluster's catalog. <br/><br/>It's _not recommended_ to use schema names that have upper case or mixed case letters. | `my_schema` |
| `host` | The hostname of your cluster.<br/><br/>Don't include the `http://` or `https://` prefix. | `YOURORG.databrickshost.com` |
| `http_path` | The http path to your SQL Warehouse or all-purpose cluster. | `/SQL/YOUR/HTTP/PATH` |
| `schema` | The name of a schema within your cluster's catalog. <br/><br/>It's _not recommended_ to use schema names that have upper case or mixed case letters. | `MY_SCHEMA` |

## Authentication parameters

The `dbt-databricks` adapter supports both [token-based authentication](/docs/core/connect-data-platform/databricks-setup?tokenoauth=token#examples) and [OAuth client-based authentication](/docs/core/connect-data-platform/databricks-setup?tokenoauth=oauth#examples).

Refer to the following **required** parameters to configure your profile for each type of authentication:

| Field | Authentication type | Description | Example | Authentication type |
| Field | Authentication type | Description | Example |
| --------- | ------- | ----------- | ---- |
| `token` | Token-based | The Personal Access Token (PAT) to connect to Databricks. | `dapiXXXXXXXXX`<br /> `XXXXXXXXXXXXXX` |
| `client_id` | OAuth-based | The client ID for your Databricks OAuth application.<br /> | `<oauth-client-id>` |
| `client_id` | OAuth-based | The client ID for your Databricks OAuth application.<br /> | `OAUTH_CLIENT_ID` |
| `client_secret` | OAuth-based | The client secret for your Databricks OAuth application. <br /> | `XXXXXXXXXXXXX`<br /> `XXXXXXXXXXXXXX` |
| `auth_type` | OAuth-based | The type of authorization needed to connect to Databricks. <br /> | `oauth` |

Expand Down
36 changes: 18 additions & 18 deletions website/docs/docs/core/connect-data-platform/hive-setup.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ dbt-hive supports two transport mechanisms:
- binary
- HTTP(S)

The default mechanism is `binary`. To use HTTP transport, use the boolean option `use_http_transport: [true / false]`.
The default mechanism is `binary`. To use HTTP transport, use the boolean option. For example, `use_http_transport: true`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@matthewshaver I changed this to be an example for clarity.


## Authentication Methods

Expand All @@ -51,8 +51,8 @@ your_profile_name:
dev:
type: hive
host: localhost
port: [port] # default value: 10000
schema: [schema name]
port: PORT # default value: 10000
schema: SCHEMA_NAME

```

Expand All @@ -72,15 +72,15 @@ your_profile_name:
outputs:
dev:
matthewshaver marked this conversation as resolved.
Show resolved Hide resolved
type: hive
host: [host name]
http_path: [optional, http path to Hive] # default value: None
port: [port] # default value: 10000
host: HOST_NAME
http_path: YOUR/HTTP/PATH # optional, http path to Hive default value: None
port: PORT # default value: 10000
auth_type: ldap
use_http_transport: [true / false] # default value: true
use_ssl: [true / false] # TLS should always be used with LDAP to ensure secure transmission of credentials, default value: true
username: [username]
password: [password]
schema: [schema name]
use_http_transport: BOOLEAN # default value: true
use_ssl: BOOLEAN # TLS should always be used with LDAP to ensure secure transmission of credentials, default value: true
username: USERNAME
password: PASSWORD
schema: SCHEMA_NAME
```

</File>
Expand All @@ -99,13 +99,13 @@ your_profile_name:
outputs:
dev:
type: hive
host: [hostname]
port: [port] # default value: 10000
auth_type: [GSSAPI]
kerberos_service_name: [kerberos service name] # default value: None
use_http_transport: true # default value: true
use_ssl: true # TLS should always be used to ensure secure transmission of credentials, default value: true
schema: [schema name]
host: HOSTNAME
port: PORT # default value: 10000
auth_type: GSSAPI
kerberos_service_name: KERBEROS_SERVICE_NAME # default value: None
use_http_transport: BOOLEAN # default value: true
use_ssl: BOOLEAN # TLS should always be used to ensure secure transmission of credentials, default value: true
schema: SCHEMA_NAME

```

Expand Down
4 changes: 2 additions & 2 deletions website/docs/reference/resource-configs/databricks-configs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -459,9 +459,9 @@ To configure this inside of dbt Cloud, use the [extended attributes feature](/do

compute:
Compute1:
http_path:[`/some/other/path']
http_path: /SOME/OTHER/PATH
Compute2:
http_path:[`/some/other/path']
http_path: /SOME/OTHER/PATH

```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed the profile.yml Under Selecting compute per model also uses the <profile-name> but maybe we should change all profile.yml and project.yml examples in one PR to reduce confusion. what do you think?

Expand Down
20 changes: 10 additions & 10 deletions website/docs/reference/resource-properties/constraints.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,26 +37,26 @@ models:
# model-level constraints
constraints:
- type: primary_key
columns: [<first_column>, <second_column>, ...]
- type: foreign_key # multi_column
columns: [<first_column>, <second_column>, ...]
expression: "<other_model_schema>.<other_model_name> (<other_model_first_column>, <other_model_second_column>, ...)"
columns: FIRST_COLUMN, SECOND_COLUMN, ...
- type: FOREIGN_KEY # multi_column
columns: FIRST_COLUMN, SECOND_COLUMN, ...
expression: "OTHER_MODEL_SCHEMA.OTHER_MODEL_NAME (OTHER_MODEL_FIRST_COLUMN, OTHER_MODEL_SECOND_COLUMN, ...)"
- type: check
columns: [<first_column>, <second_column>, ...]
expression: "<first_column> != <second_column>"
name: human_friendly_name
columns: FIRST_COLUMN, SECOND_COLUMN, ...
expression: "FIRST_COLUMN != SECOND_COLUMN"
name: HUMAN_FRIENDLY_NAME
- type: ...

columns:
- name: <first_column>
data_type: <data_type>
- name: FIRST_COLUMN
data_type: DATA_TYPE

# column-level constraints
constraints:
- type: not_null
- type: unique
- type: foreign_key
expression: <other_model_schema>.<other_model_name> (<other_model_column>)
expression: OTHER_MODEL_SCHEMA.OTHER_MODEL_NAME (OTHER_MODEL_COLUMN)
- type: ...
```

Expand Down
Loading