v0.11.0

Authoring

• It is now possible to provide customized feedback when an exercise submission produces an evaluation error. The checking function applied when the user code results in an error is defined via the exercise.error.checker option of tutorial_options() (#403). gradethis::grade_learnr_error() is a sensible default for this option.

  Additionally, user submissions for R code exercises are now checked for parsing errors prior to any other checks. If the submitted code is unparsable, a friendly error feedback message is returned and no further evaluation or checking is performed (@rossellhayes #547).

• Authors can now provide data files for use with interactive exercises. Any files stored in a data/ directory adjacent to the tutorial R Markdown document are now automatically made available within exercises. An alternative directory can be specified using the tutorial.data_dir global option (@rossellhayes #539).

• An informative error is now thrown when an exercise chunk’s body contains nothing, which lets the tutorial author know that something (e.g., empty line(s)) must be present in the chunk body for it to be rendered as an exercise (@KatherineCox #172, #410).

• Custom CSS files are now loaded last, after all of learnr’s other web dependencies (#574).

• Footnotes now appear at the end of the section in which they appear (thanks @plukethep, #647).

Setup Chunk Chaining

• Exercise chunks can now be “chained together” via chained setup chunks. The setup chunk of one exercise may depend on other chunks, including the setup chunks of other exercises, allowing the author to form a chain of setup code that allows interactive exercises to progressively work through a problem. These chains are defined using the exercise.setup chunk option; use run_tutorial("setup_chunks", "learnr") to run a demo tutorial (@nischalshrestha #390).

  • As part of this work, learnr now throws an error at pre-render when an author specifies an non-existent chunk label in the exercise.setup of an exercise.
  • learnr also now forces the chunk option include = FALSE for setup chunks when evaluated as part of an exercise to avoid unexpected printing of results.

Internationalization and Customization

• Text throughout the learnr interface can be customized or localized using the new language argument of tutorial() (#456, #479). The random positive and encourage phrases generated by learnr can also be translated (#538). Community contributions for additional languages are welcomed, but it is possible to customize the language used for the learnr interface and phrases without requiring official support via the learnr package. You can read more about these features in vignette("multilang", package = "learnr").

  We are very grateful to the following community members for providing additional languages:

  • Basque language support was contributed by @mikelmadina (#489)
  • Portuguese language support was contributed by @beatrizmilz (#488, #551)
  • Spanish language support was contributed by @yabellini (#483, #546)
  • Turkish language support was contributed by @hyigit2 and @coatless (#493, #554)
  • German language support was contributed by @NinaCorrelAid (#611, #612)
  • Korean language support was contributed by @choonghyunryu (#634)
  • Chinese language support was contributed by @shalom-lab (#681)
  • Polish language support was contributed by Jakub Jędrusiak (@kuba58426) (#686)

• Messages generated by R during exercises are now translated to match the tutorial language, if translations are available either in base R or in the R package generating the message (@rossellhayes #558).

• Breaking Change: random_praise() no longer includes the phrase "Correct! ". Instead, it simply returns a random praising statement (#453, #463).

Support for Additional Programming Languages

• Added a new polyglot tutorial to learnr. This tutorial demonstrates how learnr tutorials might mix R, Python, and SQL exercises. See run_tutorial("polyglot", "learnr") for a an example (#397).

• engine is now passed to the exercise.checker to communicate which programming language is being checked in the exercise (#397).

• The exercise.cap exercise/chunk option now accepts HTML input. If no exercise.cap is provided, the exercise.cap will default to the combination of the exercise engine and " code" (#397, #429).

• Improved support for SQL exercises makes it possible to check student submissions for SQL exercises. See run_tutorial("sql-exericse", "learnr") or the online SQL exercise demo for an example tutorial with graded SQL exercises (#668).

• Exercise editors now use syntax highlighting and basic autocompletion for exercises in languages other than R with syntax highlighting support for JavaScript, Julia, Python and SQL (#693).

• Broadly improved support for additional programming languages and added support for Python exercises (#724).

Interactive Exercises and Questions

Exercises

• Users are now warned if their submission contains blanks they are expected to fill in. The default blank pattern is three or more underscores, e.g. ____. The pattern for blanks can be set with the exercise.blanks chunk or tutorial option (@rossellhayes #547).

• Users who submit unparsable code containing non-ASCII characters are now presented with more informative feedback. Non-ASCII characters are a common source of code problems and often appear in code when students copy and paste text from a source that applies automatic Unicode formatting. If the submission contains Unicode-formatted quotation marks (e.g. curly quotes) or dashes, the student is given a suggested replacement with ASCII characters. In other cases, the student is simply prompted to delete the non-ASCII characters and retype them manually (@rossellhayes #642).

• Authors can choose to reveal (default) or hide the solution to an exercise. Set exercise.reveal_solution in the chunk options of a *-solution chunk to choose whether or not the solution is revealed to the user. The option can also be set globally with tutorial_options(). In a future version of learnr, the default will likely be changed to hide solutions (#402).

• Exercises may now include -tests chunks. These chunks don't appear in the tutorial text but the code in them is stored in the internal exercise data. In the future, these chunks will be used to provide automated exercise testing (#664, #700).

• Keyboard navigation and keyboard shortcuts for the interactive exercise code editor have been improved:

  • To avoid trapping keyboard focus and to allow users to navigate through a tutorial with the keyboard, pressing Esc in an interactive exercise code editor now temporarily disables the use of Tab for indenting, making it possible for users to move to the next or previous element in the tutorial (#652).

  • Interactive exercises now know the RStudio shortcuts for the pipe (%>%) (Command/Control + Shift + M) and assignment (<-) (Alt + -) operators in exercise code boxes (#472).

  • Clicking Run Code or using the keyboard shortcut (Cmd/Ctrl + Enter) now runs the selected code only, if any code is selected (thanks @petzi53 #512, #514).

  • Commented code within an exercise is no longer be auto completed (#604).

  • Hitting the TAB key in an exercise has always opened the auto-completion drop down. Now, hitting the TAB key will also complete the currently selected code completion (#428).

• The native R pipe, introduced in R 4.1, is now recognized as a valid R operator in the interactive exercise editor (thanks @ijlyttle, #595).

• Feedback messages can now be an htmltools::tag(), htmltools::tagList(), or a character message (#458).

• We no longer display an invisible exercise result warning automatically. Instead, authors must set the exercise chunk option exercise.warn_invisible = TRUE to display an invisible result warning message (@nischalshrestha #373).

• When exercise.completion = TRUE, completion is no longer performed inside of quotes. This (intentionally) prevents the student from being able to list files on the R server (#401).

• When an exercise returns HTML generated with {htmlwidgets} or {htmltools}, learnr will remove the HTML dependences created with htmltools::tags$head() from the result returned to the tutorial. This avoids conflicts with the scripts and dependencies used by the learnr tutorial (thanks @andysouth, #484).

• Fixed exercise progress spinner being prematurely cleared (#384).

• Empty exercise chunks are now allowed. Please use caution: in very rare cases, knitr and learnr may not notice duplicate chunk labels when an exercise uses a duplicated label. Allowing empty exercise chunks improves the ergonomics when using knitr's chunk option comments (#712).

Exercise Evaluation

• Breaking Change: If a -code-check chunk returns feedback for an exercise submission, the result of the exercise is no longer displayed for a correct answer (only the feedback is displayed). If both the result and feedback should be displayed, all checking should be performed in a -check chunk (i.e., don’t provide a -code-check chunk) (#403).

• Exercise checking is now conducted in the same temporary directory where exercises are evaluated (@rossellhayes #544).

• Exercises evaluation now communicates the stage of evaluation via a new stage argument passed to the checker function. Stages may be "code_check", "error_check", or "check". This makes it easier for the exercise checking function to determine at which point checking is being applied in the exercise evaluation life cycle (@rossellhayes #610).

• options() and environment variables are now reset after rendering exercises so that changes made by user input or checking code cannot affect other exercises (@rossellhayes #542).

• Parse errors from user code that fails to parse can now be inspected by the error checker, but errors in exercise setup chunks cannot. Instead, global setup and setup chunk errors are raised as internal errors with a user-facing warning. In general, internal errors are now handled more consistently (#596).

  • The parsing error object now has a "parse_error" class so that you can use inherits(last_value, "parse_error") in learnr error checking code or inherits(.result, "parse_error") in gradethis error checking to differentiate the parse error from other error types (#658).

• learnr now properly enforces the time limit set by the exercise.timelimit chunk option (#366, #368, #494).

• The envir_prep environment used in exercise checking now accurately captures the result of both global and exercise-specific setup code, representing the environment in which the user code will be evaluated (as was described in the documentation). learnr also ensures that envir_result (the environment containing the result of evaluating global, setup and user code) is a sibling of envir_prep (#480).

• When allow_skip is set to FALSE, users are now required to run an exercise once with non-empty code in order to move forward. If the exercise has grading code, users are required to submit one (non-empty) answer (thanks @gaelso #616, #633).

• If an exercise includes a -check chunk but no exercise.checker function has been defined, learnr will now throw an error at render reminding the author to use tutorial_options() to define an exercise checker (#640).

Questions

• Authors can now provide function-answers with answer_fn(). Authors can provide a function that takes a single argument that will be passed the student's question submission. This function decides if the question is correct and provides feedback by returning correct() or incorrect() with a feedback message (#657).

• A new question_numeric() question type allows authors to ask users to provide a number (#461).

• question_text() gains rows and cols parameters. If either is provided, a multi-line textAreaInput() is used for the text input (thanks @dtkaplan #455, #460).

• Correct/incorrect question markers are now configurable via CSS. You can change or style these markers using the .tutorial-question .question-final .correct::before and .tutorial-question .question-final .incorrect::before selectors. A new helper function, finalize_question(), can be used to apply the .question-final class to custom learnr questions (#531).

• Fixed a bug to avoid selecting the answer labeled "FALSE" by default in question_radio() (#515).

• Fixed unexpected behavior for question_is_correct.learnr_text() where trim = FALSE. Comparisons will now happen with the original input value, not the HTML() formatted answer value (#376).

• When a quiz’s question or answer text are not characters, e.g. HTML, {htmltools} tags, numeric, etc., they are now cast to characters for the displayed answer text and the quiz’s default loading text (#450).

Events and State

• Breaking Change: The exercise_submission event was renamed to exercise_result and now includes the following new fields (#337):

  • id - a randomly generated identifier that can be used to align with the associated exercise_result event
  • time_elapsed - the time required to run the exercise (in seconds)
  • timeout_exceeded - indicates whether the exercise was interrupted due to an exceeded timeout. May be NA for some platforms/evaluators if that information is not known or reported.

• Added a general-purpose event handler system, powered by the functions event_register_handler() and one_time(). These functions can be used to execute code every time a learnr-specific event occurs (#398).

• Several learnr events were updated and two new events were created:

  • A new event named "section_viewed" now reports when a new section becomes visible.
  • Added an "exercise_submitted" event which is fired before evaluating an exercise. This event can be associated with an "exercise_result" event using the randomly generated id included in the data of both events (#337). The "exercise_submitted" event also now contains a restore field indicating whether the exercise is being restored from a previous execution (TRUE), or that the exercise is being run interactively (FALSE) (#370).
  • A new label field of the "exercise_hint" event identifies the exercise for which the user requested a hint (#377).
  • Previously, when a question submission was reset, it would be recorded as a "question_submission" event with the value reset = TRUE. Now it a separate event, "reset_question_submission".

• Tutorial authors can now access the current state of the user’s progress in a tutorial with get_tutorial_state() or get information about the current tutorial with get_tutorial_info() (#562). Tutorial state is now returned by get_tutorial_state() in order of appearance in the tutorial. The full list of exercises and questions is included as items in the list returned by get_tutorial_info() (thanks @NuoWenLei #570, #571).

• get_tutorial_info() can now provide complete tutorial info for an .Rmd or pre-rendered .html tutorial file outside of a Shiny app (#688, #702).

• We no longer forward the checker code to browser (in html), but instead cache it (@nischalshrestha #390).

• Fail gracefully when unable to open an indexedDB store (e.g. in cross-origin iframes in Safari) (#417).

Running Tutorials

• Running learnr tutorials with run_tutorial() has been improved (#601):

  • run_tutorial() can now run local tutorials in addition to tutorials hosted in a package. To run a local tutorial provide the path to the tutorial or the directory containing the tutorial via the name argument without providing the package argument.
  • Breaking change: names must be provided for all arguments to run_tutorial() other than name and package.
  • run_tutorial() gains a clean argument to completely re-render the tutorial if needed.
  • learnr tutorials are now run as background RStudio jobs and open in the viewer pane when run_tutorial() is called in RStudio. This default is disabled in non-interactive settings or when as_rstudio_job = FALSE. You can control where the tutorial is opened with the shiny.launch.browser global option.
  • run_tutorial() now renders tutorials in a temp directory if the R user does not have write permissions (#347).

• Many of the HTML dependencies used by learnr have been updated to more recent versions (#655). learnr now uses:

  • Ace version 1.10.1
  • clipboard.js version 2.0.10
  • Bootbox version 5.5.2
  • i18next version 21.6.10

• learnr’s knitr hooks are now set by the learnr::tutorial R Markdown format. They are also registered for any tutorials run by run_tutorial() (thanks @czucca #598, #599).

• For the “forked” evaluator (the default used on Linux), learnr now limits the number of forked exercises that learnr will execute in parallel. Previously, this was uncapped, which could cause a learnr process to run out of memory when an influx of traffic arrived. The default limit is 3, but it can be configured using the tutorial.max.forked.procs option or the TUTORIAL_MAX_FORKED_PROCS environment variable (#353).

• Introduced an experimental function external_evaluator() which can be used to define an exercise evaluator that runs on a remote server and is invoked via HTTP. This allows all exercise execution to be performed outside of the Shiny process hosting the learnr document (#345, #354).

• Added option for quickly restoring a tutorial without re-evaluating the last stored exercise submission. This feature is enabled by setting the global option tutorial.quick_restore = TRUE or the environment variable TUTORIAL_QUICK_RESTORE=1 (thanks @mstackhouse, #509).

• exercise_result() no longer combines the code output and feedback; this now happens just before presenting the exercise result to the user (#522).

• Support the updated Bootstrap 4+ popover dispose method name, previously destroy (#560).

• Forked evaluator (used by default on Linux and shinyapps.io) now only collects the exercise evaluation result once, avoiding a "cannot wait for child" warning (thanks @tombeesley #449, #631).

• learnr::tutorial() now allows authors to adjust the value of lib_dir (#648).

• learnr now uses and suggests shinytest2 for automated testing of tutorials in the browser. If you were previously using shinytest to test your tutorials, you may find the Migrating from shinytest article to be helpful (#694).