Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added options immedate & replace to eng_SQL #2128

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 42 additions & 28 deletions R/engine.R
Original file line number Diff line number Diff line change
Expand Up @@ -560,13 +560,14 @@ is_sql_update_query = function(query) {
grepl('^\\s*(INSERT|UPDATE|DELETE|CREATE|DROP).*', query, ignore.case = TRUE)
}


# sql engine
eng_sql = function(options) {
# return chunk before interpolation eagerly to avoid connection option check
if (isFALSE(options$eval) && !isTRUE(options$sql.show_interpolated)) {
return(engine_output(options, options$code, ''))
}

# Return char vector of sql interpolation param names
varnames_from_sql = function(conn, sql) {
varPos = DBI::sqlParseVariables(conn, sql)
Expand All @@ -575,27 +576,27 @@ eng_sql = function(options) {
sub('^\\?', '', varNames)
}
}

# Vectorized version of exists
mexists = function(x, env = knit_global(), inherits = TRUE) {
vapply(x, exists, logical(1), where = env, inherits = inherits)
}

# Interpolate a sql query based on the variables in an environment
interpolate_from_env = function(conn, sql, env = knit_global(), inherits = TRUE) {
names = unique(varnames_from_sql(conn, sql))
names_missing = names[!mexists(names, env, inherits)]
if (length(names_missing) > 0) {
stop("Object(s) not found: ", paste('"', names_missing, '"', collapse = ", "))
}

args = if (length(names) > 0) setNames(
mget(names, envir = env, inherits = inherits), names
)

do.call(DBI::sqlInterpolate, c(list(conn, sql), args))
}

# extract options
conn = options$connection
if (is.character(conn)) conn = get(conn, envir = knit_global())
Expand All @@ -608,13 +609,26 @@ eng_sql = function(options) {
max.print = -1
sql = one_string(options$code)
params = options$params

immediate= F
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DBI seems to default to NULL for dbExecute and dbGetQuery, which means
https://dbi.r-dbi.org/reference/dbexecute#specification-for-the-immediate-argument-1

The default NULL means that the backend should choose whatever API makes the most sense for the database

So setting FALSE, will change behavior for all the DB that backends that will use TRUE by default.

Maybe we should stick to that, and pass immediate = TRUE or FALSE when the chunk option is explicitly set ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DBI seems to default to NULL for dbExecute and dbGetQuery, which means
https://dbi.r-dbi.org/reference/dbexecute#specification-for-the-immediate-argument-1

The default NULL means that the backend should choose whatever API makes the most sense for the database

So setting FALSE, will change behavior for all the DB that backends that will use TRUE by default.

Maybe we should stick to that, and pass immediate = TRUE or FALSE when the chunk option is explicitly set ?

if(isTRUE(options$immediate)) {
immediate= T
replace= F
if(isTRUE(options$replace)) {
replace= T
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
immediate= F
if(isTRUE(options$immediate)) {
immediate= T
replace= F
if(isTRUE(options$replace)) {
replace= T
immediate = isTRUE(options$immediate)
replace = isTRUE(options$replace)
if (immediate && replace) {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would use sql. prefix on those new options, to go with existing sql.print and sql.show_interpolated.

@yihui I don't know if we can end up with a rule on under which condition which should use engine.opts, bare chunk option name or prefixed option name. I think we should try being consistent. sql.show_interpolated is not really documented so we could also support show_interpolated to be consistent.

Probably than the chunk engine is enough to scope the variable (advanced usage of chunk options hooks would require checking the engine, and then checking the option in case of same option name used by two different engine with different meaning - may never happen though).

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's use the sql. prefix for the new options here.

I don't know if we can end up with a rule on under which condition which should use engine.opts, bare chunk option name or prefixed option name. I think we should try being consistent.

I agree. It has been a mess, which I don't like, but I feel it's tough to decide which is better:

```{sql, engine.opts = list(immediate = TRUE, replace = TRUE)}
```{sql, sql.immediate = TRUE, sql.replace = TRUE}

chunk engine is enough to scope the variable (advanced usage of chunk options hooks would require checking the engine, and then checking the option in case of same option name used by two different engine with different meaning - may never happen though).

That's also what I think.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would use sql. prefix on those new options, to go with existing sql.print and sql.show_interpolated.

@yihui I don't know if we can end up with a rule on under which condition which should use engine.opts, bare chunk option name or prefixed option name. I think we should try being consistent. sql.show_interpolated is not really documented so we could also support show_interpolated to be consistent.

Probably than the chunk engine is enough to scope the variable (advanced usage of chunk options hooks would require checking the engine, and then checking the option in case of same option name used by two different engine with different meaning - may never happen though).

reptable = gsub('(^.*into[[:space:]]+)(#.+)([[:space:]]+from.*$)', '\\2', sql, ignore.case = T)
if(sub('(.).*.$', '\\1', reptable) != '#') stop2(
"To replace a table, the table has to be a temporary table (tablename staring with # or ##)."
)
sql <- paste0("if object_id('tempdb.dbo.", reptable, "') is not null drop table ", reptable, " ;", sql)
}
cderv marked this conversation as resolved.
Show resolved Hide resolved
}

query = interpolate_from_env(conn, sql)
if (isFALSE(options$eval)) return(engine_output(options, query, ''))

data = tryCatch({
if (is_sql_update_query(query)) {
DBI::dbExecute(conn, query)
DBI::dbExecute(conn, query, immediate = immediate)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to take precaution here.
DBI::dbExecute() is a generic and immediate argument, is indeed among the specifications. However, immediate is not part of the generic argument to improve compatibility with the different backends.
https://dbi.r-dbi.org/reference/dbexecute#additional-arguments-1

The following arguments are not part of the dbExecute() generic (to improve compatibility across backends) but are part of the DBI specification:

  • params (default: NULL)
  • immediate (default: NULL)

They must be provided as named arguments. See the "Specification" sections for details on their usage

This is valid for other calls too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to take precaution here.
DBI::dbExecute() is a generic and immediate argument, is indeed among the specifications. However, immediate is not part of the generic argument to improve compatibility with the different backends.
https://dbi.r-dbi.org/reference/dbexecute#additional-arguments-1

The following arguments are not part of the dbExecute() generic (to improve compatibility across backends) but are part of the DBI specification:

  • params (default: NULL)
  • immediate (default: NULL)

They must be provided as named arguments. See the "Specification" sections for details on their usage

This is valid for other calls too.

NULL
} else if (is.null(varname) && max.print > 0) {
# execute query -- when we are printing with an enforced max.print we
Expand All @@ -623,50 +637,50 @@ eng_sql = function(options) {
data = DBI::dbFetch(res, n = max.print)
DBI::dbClearResult(res)
data

} else {
if (length(params) == 0) {
DBI::dbGetQuery(conn, query)
DBI::dbGetQuery(conn, query, immediate = immediate)
} else {
# If params option is provided, parameters are not interplolated
DBI::dbGetQuery(conn, sql, params = params)
DBI::dbGetQuery(conn, sql, immediate = immediate, params = params)
}
}
}, error = function(e) {
if (!options$error) stop(e)
e
})

if (inherits(data, "error"))
return(engine_output(options, query, one_string(data)))

# create output if needed (we have data and we aren't assigning it to a variable)
output = if (length(dim(data)) == 2 && ncol(data) > 0 && is.null(varname)) capture.output({

# apply max.print to data
display_data = if (max.print == -1) data else head(data, n = max.print)

# get custom sql print function
sql.print = opts_knit$get('sql.print')

# use kable for markdown
if (!is.null(sql.print)) {
options$results = 'asis'
cat(sql.print(data))
} else if (out_format('markdown')) {

# we are going to output raw markdown so set results = 'asis'
options$results = 'asis'

# force left alignment if the first column is an incremental id column
first_column = display_data[[1]]
if (is.numeric(first_column) && length(first_column) > 1 && all(diff(first_column) == 1))
display_data[[1]] = as.character(first_column)

# wrap html output in a div so special styling can be applied
add_div = is_html_output() && getOption('knitr.sql.html_div', TRUE)
if (add_div) cat('<div class="knitsql-table">\n')

# determine records caption
caption = options$tab.cap
if (is.null(caption)) {
Expand All @@ -680,27 +694,27 @@ eng_sql = function(options) {
}
# disable caption
if (identical(caption, NA)) caption = NULL

# print using kable
print(kable(display_data, caption = caption))

# terminate div
if (add_div) cat("\n</div>\n")

# otherwise use tibble if it's available
} else if (loadable('tibble')) {
print(tibble::as_tibble(display_data), n = max.print)

} else print(display_data) # fallback to standard print
})
if (options$results == 'hide') output = NULL

# assign varname if requested
if (!is.null(varname)) assign(varname, data, envir = knit_global())

# reset query to pre-interpolated if not expanding
if (!isTRUE(options$sql.show_interpolated)) query <- options$code

# return output
engine_output(options, query, output)
}
Expand Down