- Update minitar version to 0.9, to match the version installed with puppet-agent
- Switched to thread-safe OpenSSL::Digest creation. #979
- Updated thread pool error handling and logging
- Adds support for installing modules concurrently
- (RK-343) Pins CRI dependency to 2.15.6 to resolve regression in options parsing.
- Flag for overriding default branch configuration in Puppetfile
- Plumbing for internationalization
- Numerous test fixes and legacy docker work
- Add support for running
puppet generate types
- (RK-335) Postrun
modifiedenvsdoesn't include environment prefixes
-
Substitute environments acted on in postrun command.
Now post run commands that contain the string "$modifiedenvs" (eg. ["/usr/local/bin/my-postrun-cmd", "--verbose", "$modifiedenvs"]) will have the string substituted with a space separated list of environments acted upon (either a single environment if specified on the command line or all environments).
Specifically this should allow users to easily wrap
puppet generate typesand matches the terminology used in g10k.Many thanks to @raphink for the contribution.
- Flag for overriding default branch configuration in Puppetfile
- Plumbing for internationalization
- Numerous test fixes and legacy docker work
- (RK-324) Fix Ruby pipe bug affecting Ubuntu
- Minor test fixes.
Because of dependency issues R10K 3.0.0 required Ruby >= 2.3 rather than the reported 2.0. This release makes the requirement of Ruby >= 2.3 official and documented.
- (#853) (RK-327 Uninitialized Constant Cri::Error When resolving the Cri dependency >= 2.13 R10K would fail with an uninitialized constant error. Thanks to @ostavnaas for the bug report, @ddfreyne for the fix, and @baurmatt for the review.
- Child processes may die unexpectedly when deploying many environments on Ubuntu Bionic. See RK-324.
- Drop support for Ruby < 2.0
- Remove support for PUPPETFILE and PUPPETFILE_DIR environment variables
when running the
puppetfileaction, please use flags instead. - Fail when duplicate module definitions in Puppetfile
- More reliable pruning of refs on fetch
- Improved error messaging when:
- Unable to connect to a proxy
- r10k.yaml file is empty
- Unable to parse Puppetfile
- Various perfomance improvements
- Flag for overriding default branch configuration in Puppetfile
- Plumbing for internationalization
- Numerous test fixes and legacy docker work
(RK-324) Fix Ruby pipe bug affecting Ubuntu
Numerous test fixes.
Update specs with new error string.
NOTE - CHANGELOG is only assured to be up to date for a particular branch when a release is made on that branch.
(RK-311) Yard dependency updated for security fix.
(RK-310) Fix ChecksumMismatch error on Windows for forge caching.
(RK-307) Branches can now be ignored by prefixes during deployment.
(RK-305) Add --no-force to deploy action to avoid overwriting local module changes.
(RK-264) Add --force action to puppetfile install to force overwriting local module changes.
(RK-291) (RK-304) Add caching of forge modules.
(RK-306) Remove the dependency on semantic_puppet.
(RK-161) Deprecate the usage of PUPPETFILE and PUPPETFILE_DIR environment variables.
2017/06/02
(#696) Move deprecated module check to install/reinstall/upgrade rather than synchronize. This fixes a major slowdown on redeployment.
(RK-290) Add deprecation warning for duplicate module names in Puppetfile. This will cause an error in r10k v3.0.0.
(RK-285) Update minitar dependency to 0.6.1
2017/04/05
(PF-1317) Only use deprecated attribute when it exists.
2017/03/31
(#645) Fix undefined method error when r10.yaml empty
(#659)(RK-269) Puppetfile actions acknowledge :branch and :default_branch (Thanks to Chris Cowley for the report.)
(PF-1317) Emit a warning when syncing a deprecated Forge module.
2017/02/07
(#699) Pin minitar dependency to 0.5.4 to avoid a bug with the 0.6.0 release. (Thanks to Logan Garrett for the report and fix.)
2016/12/05
(RK-78) Use :prune option for #fetch in Rugged::BareRepository
Versions of the "rugged" gem prior to 0.24.0 lacked the ability to automatically "prune" branches from a local repo that no longer existed in the matching remote repo after a fetch. To work around this issue, r10k included code that would manually remove/recreate branches during a fetch. Since "rugged" 0.24.0 is now widely available, r10k has been updated to use the built-in "prune" option during a fetch and the workaround code has been removed.
NOTE: If you use the "rugged" gem with r10k, you will need to manually upgrade it to a version >= 0.24.0 to take advantage of the new functionality. If you are using a "rugged" version less than 0.24.0, r10k will now issue a warning every time it fetches from a remote git repository.
2016/11/15
(#669) Updated the behavior of the rugged based git provider to handle unexpected behavior around checkout and resets regarding file permissions, specifically, when resetting to an already checked out SHA the executable bit on files would not update.
(#664) Added to the proxy error message for changes made to libcurl which gave new, surprising errors about unsupported proxy schemes.
2016/08/23
(RK-266) Fixed an issue where the "puppetfile install" action was encountering an error when operating on a Puppetfile with "local" content declarations.
2016/08/22
(RK-265) The "puppetfile install" action will no longer overwrite local modifications to managed Git content. Instead, a message will be logged at the "WARN" level indicating that the content was skipped. Note: The "deploy" actions will still overwrite local modications. For more background on this change, see below:
In 2.4.0 a change was made to r10k's behavior when it encounters local modifications during "deploy" operations. Previously, r10k would log an error and skip updating the modified content. As of 2.4.0, local modifications will be overwritten and a warning will be logged. This change was considered a bug fix but was originally omitted from the changelog for that release. This change also inadvertently modified the behavior of the "puppetfile install" action. A command line flag to control this behavior more explicitly will likely be added in a future version.
2016/08/11
(#634) Fix "undefined variable" error in "deploy module" action. (Special thanks to Andreas Ntaflos (antaflos) for the fix.)
(#635) Reword some documentation around environment level purging.
2016/08/10
(RK-222) New "install_path" option for Git/SVN content.
This feature allows you to specify where inside an environment each item from the Puppetfile should be installed to. See the Puppetfile documentation for details.
(RK-246) New "environment" level purging and configurable purge levels.
You can now configure how r10k purges unmanaged content after a deployment. The default behavior should be unchanged but there is a new "purge_levels" configuration option that can be used to enable new behavior or de-activate certain existing behaviors. See the relevant configuration documentation for more details.
(RK-223) Ability to track control repo branch from content declarations.
Puppetfile content sourced from Git can now be configured to attempt to track the branch name of the control repo branch being deployed. For example, if r10k is deploying the 'production' branch of your control repo, it will try to also deploy the 'production' branch of a given Puppetfile content repo. See the documentation for more details.
All user-facing strings generated by r10k have been externalized to enable future iternationalization (i18n) and localization work.
(RK-258) Symlinks inside of Forge modules will no longer cause r10k to exit non-zero. This situation used to raise an error but will now generate a WARN level log message instead.
(#483) Local modifications to managed content will now be overwritten during "deploy" actions. (Note: This change inadvertently also affected the "puppetfile install" action in 2.4.0 and 2.4.1. This was fixed in 2.4.2. A command line flag to control this behavior more explicitly will likely be added in a future version.)
(#616) Ensure that Forge module version strings are valid semantic versions. (Special thanks to Patrick Robinson (patrobinson) for the fix.)
(#622) Fix typos in workflow docs. (Special thanks to Yury Frolov (mrdracon) for the fix.)
2016/05/17
(RK-236/RK-237) Added HTTP proxy support for Git operations.
Previously, r10k only supported the use of HTTP proxies for connecting to the Puppet Forge. With these changes, r10k can now be configured to use an HTTP proxy for both Forge and Git operations. Configuration can be specified globally, for Forge or Git only, or on a per-Git repository basis. See configuration documentation for more details.
(RK-238) When r10k encounters and ignores invalid file types in a module archive, it will now log the message at the DEBUG1 level instead of at WARN.
(RK-243) In certain cases, when using the "rugged" Git provider, specifying invalid HTTP credentials for a repository could result in an infinite loop. Authentication retry for HTTP repositories is now capped at 50 attempts which matches the existing behavior for SSH.
2016/04/18
(RK-241) "deploy display" action does not properly format wrapped exceptions
The "deploy display" action was not capturing and logging exceptions in the same way as other related actions. This meant that in many cases, when an error occurred, the underlying cause was not being shown. Specifically, the "deploy display" action was not benefitting from the improved error messaging for unreadable SSH keys which was added in r10k 2.2.0 as part of RK-220.
2016/04/14
(RK-229) Setting Forge proxy options breaks PE Authentication
A bug in the interaction between r10k and the puppet_forge gem was preventing the correct Authorization headers for Puppet Enterprise-only modules from being included with requests to the Puppet Forge when r10k was also configured to use a proxy. This bug has been resolved by adding new functionality to the puppet_forge gem and updating r10k to use the new version.
2016/03/08
(RK-154) Per-repo config for Git sources
Git repository options, such as the SSH private key, can now be set indepdently for each repository when using the Rugged provider. See documentation for details.
(RK-220) Improved error message for unreadable SSH keys
r10k will now check to ensure that the configured SSH private key for a given repository is readable before attempting to connect. This will result in a clearer error message in situations where the key file is not readable.
(CODEMGMT-453) Support for running under JRuby 1.7 with shellgit provider
r10k should now run successfully under JRuby 1.7.x when using the "shellgit" provider.
(MAINT) Documentation fixes
Various errors and inconsistencies in the documentation have been fixed thanks to contributions from Paul Tobias, Rob Nelson, and David Danzilio. Thanks!
2015/11/12
(CODEMGMT-440) Defer git alternates setup
The fix for RK-175 that updated the Git alternates file for repositories was happening too early, and could cause issues when multiple r10k processes were running concurrently. This has been fixed so that the alternates file update is deferred till the first time the git repository is actually accessed.
(RK-187) Consider thin repos with a .git file (not directory) to be mismatched
If a given Git thin repository had a .git file where r10k expected there to be a directory it would behave badly; this has been fixed so that if r10k encounters this case it treats the repository as mismatched.
(RK-181) Correctly set baseurl/proxy with shared PuppetForge URL
A combination of some odd connection handling behavior in the puppet_forge gem combined with some bad assumptions in r10k prevented users from being able to actually set a custom forge baseurl; this has been fixed.
2015/10/28
Thanks to the following contributors for their work on this release:
- Abel Paz fixing some broken links in the Git Environments documentation
- Alex Rowley for surfacing the --puppetfile and --module options for
r10k puppetfile install - Austin Blatt (our summer intern!) for extracting the vendored Puppetforge code and pushing it into the upstream puppet_forge gem.
- Branan Riley for updating the quickstart guide for Puppet 4.
- Darrell for adding documentation for the :local module type
- David Pashley for documenting potential issues with Rugged/SSH on Ubuntu
- Dennis for clarifying path information in the quickstart guide
- E. Dunham for adding an entry to the FAQ to explain the name of r10k
- Kirill Kondratenko for contributing the :local module type (RK-149)
- Louis Mayorga for finding and reporting RK-143, and testing fixes for that issue.
- Mark McKinstry for fixing some broken links in the FAQ
- Patrick Robinson for submitting GH-516 to catch and handle ArgumentErrors raised when loading Puppetfiles
- Thomas Lapinski for fixing a bug where switching from a Git module to a Forge module would incorrectly leave the Git module version installed.
- Thomas Mueller for updating the r10k homepage link in the gemspec.
(RK-70) Expose Forge baseurl setting in r10k.yaml
The Puppet Forge URL can now be changed from the default forgeapi.puppetlabs.com to a user specified value. This allows users to use private/testing Puppet Forge instances. See https://github.com/puppetlabs/r10k/blob/master/doc/dynamic-environments/configuration.mkd#baseurl for documentation on using the baseurl setting.
(RK-96) Correctly switch from Forge modules to Git modules
If a Git version of a module had been installed via a Puppetfile and the Puppetfile entry was updated to
use the Forge version of the module, r10k would only check the metadata - and if the metadata of the Git
version matched then r10k would assume that the Forge module was installed. This has been fixed so that
when checking the status of a Forge module r10k specifically looks for a .git directory and considers
the module mismatched when the directory is present, which means that it will correctly replace a Git
version of a module with the Forge version.
(RK-142) Add machine readable output for r10k deploy display
The output format for r10k deploy display was a YAML-like, janky hand rolled version; this has been
replaced with the ability to specify an actual data format for the output. The default output type is
now YAML.
(GH-477) Add --fetch option for r10k deploy display
When r10k deploy display is run it might be useful to see which environments don't yet exist; the
added --fetch option allows r10k to update the environment sources to check for missing environments
when displaying environments.
(RK-143) Use argument vector when executing commands on Windows
R10k used the Open3.capture3 call, and used to concatenate the string to execute and join it with
spaces. However Windows loves including spaces in file names, which creates all sort of weird behavior
especially when running shell commands over SSH to Windows. This has been fixed to use the argument
vector form of capture3 to avoid these issues.
(RK-149) Add :local module type
The new :local module type allows users to add modules to the Puppetfile that are included in a control repository, so that modules can be kept in the Puppetfile moduledir without them being destroyed.
(CODEMGMT-345) Write deploy signatures to a file
If the code deployed by r10k is copied to another location without the cached repos, it becomes impossible
to use Git to interact with the repository and see which version of code r10k deployed. R10k now creates a
.r10k-deploy.json file that records the time and SHA of the last code deployment.
(RK-80) Support non SSH key based authentication for rugged provider
An error in how the Rugged Git provider provided SSH credentials prevented HTTPS urls requiring authentication to function. This has been fixed so that only SSH urls are provided ssh private key credentials, and HTTPS urls can provided a username and password in the URL for authentication.
(RK-90) Don't raise an error when forge modules are duplicated
A bug in how r10k Forge modules created caused r10k to crash when two forge modules tried to create the same directory. This has been fixed and restores the old behavior of letting the last module specified win.
In the long term trying to specify two modules with the same name will produce an error, but in the mean time this fix restores existing behavior so that r10k won't roll over and die on error.
(RK-120) Enable --config as top level command option
R10k was originally built with environment deployment in mind, but the r10k puppetfile subcommand made this
assumption faulty. However the way the config file was loaded only worked with r10k deploy subcommands,
which prevented users from using a custom cachedir or use other settings when running r10k puppetfile. This has
been corrected so that r10k can always read a config file when running any subcommand.
(RK-175) Don't crash on unresolvable Rugged ref
When r10k was using the rugged provider, checking out an unresolvable ref would throw a TypeError because r10k didn't always ensure that the ref was resolvable. This has been fixed so that r10k explicitly ensures that refs can be resolved before it checks them out.
(RK-174) Always ensure alternates file is up to date
When r10k created a Git working repository it permanently linked the repository to a cached Git repository via the Git alternates file. However if the cachedir setting changed, existing repositories would still reference the old path which could very badly break Git. This has been fixed so that r10k always ensures that the alternates file is up to date before any Git operations can happen, so that changing the cachedir will not break Git repositories.
(RK-169) Print validation failures for invalid configs
R10k used to have very lax handling of config file input, which would cause crashes during r10k runtime. This has been fixed so that r10k validates all configuration before it runs and prints out all configuration validation errors when validation fails.
(RK-21) Indicate error when deploying non-existent environment
When r10k was deploying a specific list of environments, it would skip any existing environment that didn't match the desired list of environments. However this meant that deploying a non-existent environment would not deploy any environments and r10k would silently exit with an exit code of 0. This has been fixed so that when r10k deploys a list of environments it makes sure that each environment can be updated, and when requested environments are missing it logs an error and exits with a non-zero exit code.
(RK-163) Add deploy/write_lock setting
Users can now prevent r10k deploy commands that change environments and modules from running via a config
option; this allows users to lock out code deployments at certain times (code freezes and times outside of
maintenance windows.)
(GH-516) Handle ArgumentError when loading Puppetfiles
The r10k puppetfile check command didn't output a very useful error when a module was given the wrong number
of arguments; this has been fixed so that r10k catches ArgumentErrors raised while loading a Puppetfile
and wraps it so that more information is added to the error message.
(GH-469) Add --puppetfile and --moduledir options to r10k puppetfile install subcommand
The r10k puppetfile install subcommand was able to set a custom puppetfile path and moduledir location
via environment variables to match librarian-puppet, but they accidentally didn't match the librarian-puppet
semantics - and environment variables aren't very nice to expose for configuring and application. The
r10k puppetfile install subcommand now supports command line flags to set these options.
(CODEMGMT-339) Add command line option to set cachedir
There are some scenarios where r10k needs to be run with a common config file, but on a specific basis
may need to set a custom cachedir. To make this work r10k deploy now supports a --cachedir setting
for these temporary overrides.
2015/8/13
This is a bugfix release that resolves a critical issue issue in installing PE only modules.
(RK-156) PE-only modules cannot be installed
The mechanism used to load PE license information in r10k was preventing r10k from not being able to locate the pe-license Ruby library in PE 2015.2.0; this has been resolved by actually trying to load the relevant files instead of probing for a gem and then conditionally loading it.
2015/06/18
This is a maintenance release that improves error messages around installing modules from the Puppet Forge.
(RK-109) Add context to connection failure errors
If a connection to the Puppet Forge failed for any reason, the resulting exception would indicate the error type but not the host or proxy host. This made it hard to understand why connections were failing. This has been fixed so that r10k will include the host and optional proxy host in error messages when connections fail.
(RK-121) Improve error handling for nonexistent Forge modules
The r10k Puppet Forge connection error handling reports when HTTP requests fail, but would simply print the HTTP status code on failure. For cases where a nonexistent module or module release was queried, r10k now specially handles HTTP 404 status codes and indicates that the module/module release is missing instead of just throwing a generic HTTP error.
2015/06/09
This release fixes a couple of issues and defects found in 2.0.0.
Thanks to Tim Meusel (https://github.com/bastelfreak) and ktreese (https://github.com/ktreese) for reporting GH-443 and GH-447.
(RK-117), (GH-443), (GH-447) Add minitar as runtime dependency
Minitar is a hard runtime dependency of r10k as part of the new Forge module implementation, but was only added as a development dependency which means that r10k could be installed without all of the required runtime dependencies. This oversight has been corrected; r10k will now pull in minitar at installation time.
(RK-118) Readd '/etc/r10k.yaml' to config search path
The '/etc/r10k.yaml' config path was deprecated in r10k 1.5.0, but this deprecation was only a soft deprecation and was easy to miss. While 2.0.0 is a backwards incompatible release it was too aggressive to remove this entirely, so '/etc/r10k.yaml' will continue to be respected/read in r10k 2.x. If this file is present and used then a deprecation notice will be logged. Apologies for the churn on this!
2015/06/08
This is a backwards incompatible feature release, but as major releases go this is a pretty small one. Some changes introduced into master included some breaking changes and SemVer dictates that we do a major release in this case. Actual changes that will affect end users should be limited; the only one that should have big impact is the removal of Ruby 1.8.7 support. Any other issues encountered should be treated as bugs and will be fixed.
(GH-1) Native support for installing modules from the Puppet Forge
R10k can now directly install modules from the Puppet Forge, rather than shelling out to the Puppet module tool. This will allow for later optimizations like caching module downloads to speed up installing module across multiple environments.
(RK-83) Allow '-' as a module name separator
(RK-53) Remove '/etc/r10k.yaml' from config file search path.
R10k 1.5.0 added '/etc/puppetlabs/r10k/r10k.yaml' to the paths checked while looking for a config file, in order to keep in convention with the rest of the Puppet Labs config files. In 2.0.0 the old location, '/etc/r10k.yaml', has been removed.
(RK-57) Notify users of purgedirs key deprecation
The purgedirs key was used in r10k 0.0.9 but has not been used in the 1.x release series; if this setting is given then r10k will generate a warning indicating that it is not used.
(RK-47) Remove support for Ruby 1.8.7
Given that Ruby 1.8.7 has been EOL for nearly two years, it's time for r10k to drop support for 1.8.7 as well. The Puppet 4 all in one package ships with Ruby 2.1.6, so even if you're on a platform that doesn't have Ruby 1.9 or greater you can install r10k into the Puppet collection environment and used the bundled Ruby.
(RK-54) Remove deprecated subcommands
R10k 1.0.0 reorganized a number of subcommands but retained the old subcommand names for compatibility with 0.0.x; since it's been over 2 years since 1.0.0 has been released these commands have finally been removed.
(RK-113) Remove deprecated Task classes and namespaces
The R10K::Task namespace turned out to be unwieldy in practice and has been replaced with the R10K::Action namespace. Use that for running r10k as an application.
(RK-114) Remove deprecated git classes
The reorganization of the Git code in 1.5.0 rendered a number of classes obsolete; they've been removed.
2015/04/09
Thanks to all the users that helped track down RK-86, and Zack Smith in specific for tracing the source of the bug.
(RK-62) Warn when Rugged is compiled without SSH or HTTPS transports
If Rugged/libgit2 was compiled without libssh2, trying to access a Git repository via SSH throws a fairly cryptic error. It's not terribly easy to specially handle the error message that's being logged, but in lieu of that r10k now checks the Rugged gem to make sure it's compiled with support for SSH and HTTPS when the Rugged Git provider is used.
(RK-79) Bump minimum required version of faraday_middlware-multi_json
Faraday 0.9.0 changed the API for middleware plugins, which made it incompatible with the faraday_middleware-multi_json plugin. That plugin supported the new API in version 0.0.6 but the minimum required version in the r10k gemspec was not updated, allowing r10k to be installed with incompatible versions of faraday and faraday_middleware-multi_json. This has been fixed by requiring the minimum compatible version of faraday_middleware-multi_json.
(RK-86) Git modules don't properly track branches
R10k 1.5.0 added smarter syncing for Git caches to reduce network traffic, but it accidentally caused branches to stop tracking changes to the remote branch. This has been fixed and the pre-1.5.0 behavior has been restored.
2015/04/02
Ruby 1.8.7 has had a good run, but it's time for r10k to think about moving on. As of r10k 1.5.0, support for Ruby 1.8.7 is officially deprecated. Issues affecting Ruby 1.8.7 will still be fixed, but will be of lower priority. Support for Ruby 1.8.7 will be dropped entirely in r10k 2.0.0.
Thanks to the following contributors for their work on this release:
- Brett Swift for adding custom prefix values for sources (RK-35)
- Eli Young for adding additional debug
information to
r10k version(RK-37) - Pete Fritchman for removing a warning generated under Ruby 2.2 (RK-55)
- Theo Chatzimichos for updating the dependency versions for r10k
- ETL, Ben S, and Robert Nelson for their documentation contributions
(GH-57) Git based modules fail to detect presence of git binary
As part of RK-17, r10k now checks to make sure that the git binary is present
and executable before trying to run any commands. If Git is missing, r10k will
fail hard instead of trying to run and failing on the first failed command.
(GH-195) (RK-35) Allow sources to specify a custom prefix string
For r10k deployments that have separate sources for Puppet code and Hiera data,
the created environment paths must line up between the code and data repos. With
basic prefixing where the source name is used as the prefix string, it's not
possible to use prefixing and commonly named environments. This has been fixed
so that the source prefix option can be given true, false, or a string which
is a custom value to use for prefixing.
(RK-17) Add rugged/libgit2 based Git implementation
The libgit2 library and Ruby rugged gem provide a native interface to Git for Ruby that is faster and more consistent than shelling out to Git. An additional rugged based Git implementation has been added along the original 'shellgit' implementation, and the implementations can be swapped out as needed.
(RK-24) Add '/etc/puppetlabs/r10k/r10k.yaml' to config search path
In order to be more consistent with the rest of the Puppet ecosystem, r10k will now check for configuration in a standard location inside of '/etc/puppetlabs'. The old location, '/etc/r10k.yaml' is still respected so no configuration change is needed at this point, although the old location will be deprecated in the future.
(RK-25) Warn if both '/etc/r10k.yaml' and '/etc/puppetlabs/r10k/r10k.yaml' exist
(RK-31) Remove undocumented search for 'r10k.yaml'
Early versions of r10k would try to search for 'r10k.yaml' in parent directories, in the same manner that programs like Git and Bundler recursively search for configuration. However this functionality isn't terribly useful, is a bit surprising, and adds a lot of complexity. This behavior has been removed in 1.5.0.
(RK-32, RK-33, RK-38) Provide configuration options for Git providers
The r10k Git providers can now be selected and configured via configuration in r10k.yaml. See the Git specific documentation for more information on these settings.
(RK-37) Support higher verbosity levels for r10k version
If r10k version is run with the --verbose flag, diagnostic information about
r10k will be included along with version information.
(RK-39, RK-66) Improved logging/messaging
R10k now logs more information about what's going on and is more consistent about which levels it displays messages.
(RK-74) Improved log formatting
The log formatting that r10k uses was added in some distant point in the past where there were few users, and wasn't really designed with users in mind. The formatting has been improved so that at lower log levels the formatting will include the log level and nothing else, and at higher levels it will include timing information.
(RK-75) Optional log coloring
At higher log levels r10k can produce a lot of logging information at a rapid
rate, and in general errors and warnings are not very visually distinct and easy
to miss. To help solve these problems r10k 1.5.0 now has optional colored
logging to help provide more information/context at a glance. Coloring can be
enabled with the --color option.
(GH-265) (RK-11) Unix commands with > 64KiB of output no longer deadlock
Unix pipes have a maximum buffer size of 64KiB, and if the pipe buffer fills then subsequent writes will block. Since r10k used to wait for a subprocess to finish before reading from the attached pipes, if the process filled the buffers then the processes would deadlock.
This has been fixed by continually reading from the subprocess pipes to prevent the buffers from filling up.
Unfortunately implementing this correctly requires reading from the pipes until EOF, which breaks the ssh ControlPersist functionality. Supporting both the ControlPersist functionality while preventing deadlock introduces too many potential race conditions, so unfortunately workarounds for this issue can't be accepted.
See https://bugzilla.mindrot.org/show_bug.cgi?id=1988 for more information about the ssh ControlPersist bug.
(GH-142) Use Forge v3 API
R10k was using the old and deprecated v1 Forge API to determine the latest
version of Forge modules when using the :latest module version. This has been
fixed so that the v3 API is used.
(RK-16) Decouple Git platform dependent and independent code
The original code that interfaced r10k and Git was very tightly coupled; there was no defined interface and it was expected that r10k would always shell out to Git. This has been fixed by extracting the shellout specific code to a library and defining a common interface for Git classes so that different Git implementations can be used. The existing shellout implementation has been renamed to 'shellgit' and should be functionally equivalent to the Git implementation in previous versions of r10k.
(RK-55, GH-355) Prevent warnings on Ruby 2.2
Ruby 2.2 generates warnings when comparing values and #<=> raises an
exception; the code triggering this behavior has been cleaned up to prevent a
warning from being logged.
(RK-65) Switch to using the semantic_puppet gem
R10k is switching from the old vendored copy of 'SemVer' that was stolen from Puppet to the 'semantic_puppet' gem, which is the library that Puppet now uses for version comparisons and parsing.
(RK-48) Ignore deleted versions when fetching latest module version
When looking up the latest version of a module on the Forge, if the latest version had been deleted r10k would try to install that deleted version anyways. This has been fixed so that all deleted module releases will be ignored.
2015/03/13
The r10k ticket tracker is moving to the Puppet Labs issue tracker; new issues should be filed on the R10K project. The GitHub issue tracker will remain online for the near future but is deprecated in favor of JIRA. Issues from the GitHub tracker will be prefixed with "GH-"; issues from JIRA will be prefixed with "RK-".
(RK-4) Raise meaningful errors on missing sources
If the 'sources' key in r10k.yaml was left unset, was misspelled, or was empty, r10k try to blindly iterate through it as a hash and would subsequently raise an error. This has been remedied so that if the value is missing or empty an error will be raised.
Note that this doesn't handle the case where r10k.yaml is empty or malformed; that issue is being tracked as RK-34.
(GH-310, RK-36) r10k deploy display -p --detail fails on Ruby 1.8.7
Ruby 1.8.7 does not implement the comparison operator on Symbols, which was being used by the display command to ensure that hashes were printed in a consistent order. This has been fixed by backporting the Ruby 1.9 Symbol sorting to 1.8.7.
Thanks to Eli Young for reviewing PR 337 and providing helpful feedback.
2015/01/09
(GH-254) Puppetfile subcommands use non-zero exit codes on errors.
The Puppetfile install and purge commands would always exit with an exit
code of 0, regardless of if any errors occurred or if there was no Puppetfile
available. This has been now corrected so that runtime errors cause r10k to exit
with a non-zero exit code.
(GH-260) Normalize deployed environment names on the command line.
Version 1.4.0 removed the environment name normalization needed to deploy environments that had their directory names normalized; this has been remedied so that environment names on the command line are also normalized to match the corrections that r10k will make to the environment names.
(GH-269) Improved error messages when trying to use non-existent Git refs.
If a Git module tried to use a Git ref that did not exist, it would output a particularly unhelpful error message that didn't indicate what actually failed. This has been fixed so that if an invalid ref is used, r10k will actually report that the ref could not be used. What a brave new world we inhabit!
(GH-271)/(GH-275) Report the name of invalid module names.
The fix for GH-92 released in 1.4.0 added better handling and parsing of module names, but also added stricter parsing of module names and disallowed invalid characters that Puppet itself could not use. However when r10k encountered such an invalid module name, it would not report the module with the invalid name, making debugging harder than needed. In 1.4.1 r10k when r10k encounters an invalid module name it reports the invalid module name in the error message.
Thanks to the following contributors for their work on this release:
- Lex Rivera for discovering and reporting the r10k puppetfile exit code bug (GH-254).
- Eli Young for discovering and fixing the environment normalization bug (GH-260).
- Clayton O'Neill For adding better error reporting of invalid module names (GH-275).
Additional thanks to all those in #puppet and #r10k for endlessly helping new users of r10k, your assistance is invaluable!
2014/12/2
(GH-40) Display expected and actual module versions
When displaying the state of a deployment, modules would report a version but
would not indicate if that was the expected version or desired version. This has
been changed so that both the expected and actual module version information is
displayed. Because determining the actual version for modules can be slow the
--detail flag must be passed to display this information.
(GH-43) Using a relative moduledir in the Puppetfile is relative to the Puppetfile
The moduledir setting in the Puppetfile could be used to set a custom
directory for Puppetfile modules, but if a relative path was used it was
relative to the current working directory. As of 1.4.0 a relative moduledir
setting will be expanded to the Puppetfile, which should make it much easier to
use a directory other than 'modules' for the Puppetfile installed modules.
(GH-68) Add alias for r10k deploy display
The r10k deploy display subcommand had unfriendly syntax, and now has an
alias of r10k deploy list.
(GH-92) Support mod 'owner/module' for Git and SVN modules
The original implementation of Git and SVN based modules assumed that the module
name was only the name component, and did not include the owner component of the
module. Because of this the full module name was added to the path, which meant
that a Git module or SVN module called foo/bar would be created as
$moduledir/foo/bar, after which r10k would check for stale modules, see a
module called foo, and delete it.
This has now been corrected so that all modules may have an owner and name, and only the name will be used when constructing the module path.
Issues also closed as part of GH-92:
- GH-78
(GH-96) Provide output from Git command failures
When r10k encounters an error while running a command, it will always log the failed command and the output of the command. This should make it dramatically easier to diagnose issues with the underlying commands used by r10k without having to re-run the failing r10k command with a myriad of command line flags.
Issues also closed as part of GH-96:
- GH-46
- GH-94
(GH-121) Support for calling an arbitrary script after deployment
Users can now specify a postrun setting in r10k.yaml to run an arbitrary
command after an environment deployment finishes. The current implementation is
fairly simple and isn't passed additional information about the deployment and
is mainly meant to notify other services that the deployment has completed.
Future versions of r10k will provide more information to this command so that
more complex actions can be taken on success and failure.
(GH-155) Allow SVN credentials to be added for modules/environments, disallow interactive SVN
When interacting with SVN modules, r10k could run SVN commands that could try to prompt the user for input but were not provided access to stdin. R10k is meant to be non-interactive so credentials need to be provided in some manner, but cannot be provided directly by the user.
As of 1.4.0 SVN environments and modules can supply a username and password to
be used when interacting with the SVN remote, and SVN commands are run with
--non-interactive to prevent commands from trying to grab stdin.
Note: SVN credentials are passed as command line options, so the SVN credentials may be visible in the process table when r10k is running. If you choose to supply SVN credentials make sure that the system running r10k is appropriately secured.
(GH-169) Perform non-blocking reads on stderr in Subprocess
When using SSH with a Control Master, the first time an SSH connection is launched it will persist and will hang onto stderr from the parent process. R10k was using blocking IO to read from child processes and assumed that the file descriptors would be closed when the launched process exited, and the persistent SSH process would cause r10k to hang.
The subprocess code has been updated to perform nonblocking reads of stderr; when the child process exits it assumes that even if stderr is held open nothing else should be written to it, drains the buffered pipe content and returns with that.
This is working around buggy behavior in SSH, but this problem doesn't look like it'll go away so the best course of action is to incorporate this fix into downstream.
(GH-180) Don't leak FDs when executing subcommands
R10k was not explicitly closing all file descriptors, and on Ruby 1.8.7 these would not be closed by the garbage collector, causing file descriptors to leak. This has been fixed and all file descriptors should be closed after each subprocess is invoked.
Major thanks to Jeremy Asher for discovering and fixing this.
(GH-193) Don't purge environments if environment sources can't be fetched
The original behavior for deploying environments with r10k was to fetch sources, deploy environments from those sources, and then clean up any orphaned environments. If a source had been fetched before but could not be reached then r10k would use the previously fetched branch information to update environments. In normal cases this would provide a reasonably robust failure mode.
However, this meant that if no sources had been be fetched or the source remote information was typoed, r10k would have no source information, could enumerate no environments, and then enter HULK SMASH mode where it would delete all unmanaged environments - AKA everything. This is an uncommon failure mode but could bite environments that were setting up r10k for the first time.
To resolve this issue, r10k will try to fetch all sources and if any source fails to be fetched then r10k will abort the entire deployment. This means that r10k will fail very early before any changes have been made which is a safe time to fail. If the errors were transitory then r10k can be run again, and if the failures are permanent then it's hard to safely update anything and extremely dangerous to try to delete any environments.
(GH-202) Different environments cannot manage the same path
R10k allowed for multiple sources to create environments but did not have semantics for environments from different sources managing the same path. If this happened, the resulting behavior would be undefined and could do any number of strange things. The approach to this was by convention and prefixing was recommended to avoid this, but it's still not a great approach.
This was resolved by looking for environment collisions before deploying; if collisions exist then r10k will abort the deployment.
Note: The commit that fixed this referenced GH-123 instead of GH-202.
(GH-214) Rescue SyntaxError and LoadError when evaluating Puppetfiles
Since Puppetfiles are just a Ruby DSL it's possible to have multiple Puppetfiles
that are aggregated by using require or load. However these methods raise
exceptions that don't descend from StandardError, so the normal error handling
would not catch them, so a malformed Puppetfile could catastrophically crash
r10k.
Because the Puppetfile is not an actual source file in the conventional sense we can recover from these errors and continue, so r10k will now recover from these errors. If a SyntaxError or LoadError is raised while evaluating a Puppetfile, r10k will rescue them and wrap them in an R10K::Error and then raise that, which can be caught and handled as a normal, non-critical exception.
(GH-221) Only deploy modules once for each environment deploy
If an environment was deployed for the first time and --puppetfile was
specified, the initial creation would create the environment and then deploy
modules, and then the modules would be deployed again because --puppetfile was
separate code. The deploy logic has been refactored to consider --puppetfile
being passed or the environment being created to be equivalent and will only
deploy modules once.
R10K::Environment::Base#sync is no longer recursive; callers are expected to
handle recursive updates if needed.
The R10K::Action namespace has been added to provide an API to r10k
functionality. These should be used by any code that wants to interact with r10k
and should provide an alternative to shelling out to r10k or using R10K::CLI
directly. The individual action objects should be called through
R10K::Action::Runner so that application setup and teardown is handled.
Thanks to the following contributors for their work on this release:
- Robert Nelson for his work on documenting
workflows and best practices with r10k and adding convenience aliases for
r10k deploy display - Wolf Noble for homogenizing markdown file extensions
- Thomas Bartelmess for updating the required version of cri
- Theo Chatzimichos for correcting the license format to conform to SPDX
- Richard Raseley for writing a quickstart guide for r10k
- Matthew Haughton for fixing documentation typos
- Markus Frosch for fixing a regression in how duplicate environments are checked for, before the regression was ever released. Good catch!
- Jeremy Asher for fixing file descriptor leaks and hanging on open file descriptors in the Subprocess module. Great sleuthing on this!
- Guzman Braso for adding Debian support and fixing some bugs in the quickstart documentation.
- David Schmitt for fixing markdown syntax errors so that link URLs would render properly in GitHub
- Christophe Bliard for fixing the Puppetfile moduledir setting to treat relative paths as relative to the Puppetfile
- Christoph Föhrdes For fixing a copy/paste error in the Puppetfile documentation
In addition to those who contributed to the code base, thanks to all those that reported and commented on issues; user input makes it much easier to make r10k a better project!
2014/11/13
(GH-212) Force use of json_pure on Ruby 1.8.7
Ruby 1.8.7 does not ship with a JSON library, so r10k has depended on json_pure to ensure that there's always a JSON library available. However there is a quirk in multi_json in how it probes for JSON implementations and may load the wrong gem, which percolates up and breaks the JSON parsing code used when querying for forge module versions. To resolve this, json_pure is always used on ruby 1.8.7.
2014/09/14
This bugfix release incorporates all fixes added in 1.2.4.
2014/09/11
(GH-178) Failing to fetch a source git repo can wipe out environments
When updating Git sources at the beginning of a deployment, if the fetch was interrupted r10k could cache an empty list of environments. This could cause r10k to remove all environments for that source. This was due to a method directly using a value that was supposed to be lazily evaluated and memoized. It has been fixed so that even if r10k cannot fetch a source it will still be able to deploy modules for environments, clean up removed environments, and correctly remove unmanaged environments.
(GH-186) Rescue SyntaxError when checking Puppetfile syntax
When a Puppetfile with invalid syntax is parsed it raises a SyntaxError,
and the r10k puppetfile check code was not specifically handling that.
Thus when checking an invalid file r10k was actually crashing and not
gracefully handling the error. This was fixed to cleanly rescue the SyntaxError,
optionally print stacktraces, and print an all ok message on success.
2014/07/27
This bugfix release incorporates all fixes added in 1.2.3.
2014/07/16
(GH-161) Deployments fail where a branch has \W in the git branch name
In 1.3.0 environment naming was partially reworked to allow better handling of per-environment deployment, but unfortunately this caused a regression where environments would be differently named in 1.3.0. This fix changes the environment deployment on a per-name basis to use the normalized name instead of the raw Git branch name.
This bugfix release also incorporates all fixes added in 1.2.2.
Thanks to Chris Spence for his work on this release.
2014/06/07
R10k can now dynamically generate enviroments based on SVN repositories. SVN repositories must SVN repositories must conform to the conventional SVN repository structure with the directories trunk/, branches/, and optionally tags/ in the root of the repository. The trunk/ directory is specifically mapped to the production environment, branches are created as environments with the name of the given branch.
Please note that since SVN support for environments should be considered preliminary and may still have some kinks to be worked out, so use it with caution in production.
When deploying modules with r10k deploy module <modules>, users can specify
the -e <environment> flag to update modules in a single environment.
Git sources can now tune how r10k behaves when it encounters a git branch with a non-word character. Valid values are 'correct_and_warn' which emits a warning and sanitizes the environment name, 'correct' which silently corrects the environment, and 'error' which emits an error and ignores the environment.
There are a lot of internal APIs in r10k that need to be improved or overhauled, and making changes in a backwards compatible manner has been impeding development on a number of important features. There's no indication that there are any consumers of the r10k internal APIs, and if that's the case then r10k doesn't help anyone by maintaining SemVer for its internal APIs.
As of 1.3.0, r10k is dropping guarantees about API compatibility for most of
the core functionality. The R10K::Task* classes were designed to be the
primary interface for external use and those will remain backwards compatible
for 1.x. However any code around configuration parsing, deployments, sources,
environments, and modules may have API changes in minor versions.
That being said, if you are using any of these APIs and you experience breakage, you're not out of luck. If an API change actually does affect you please report it as a bug and those specific APIs can probably be fixed up. Hopefully this will ease development of r10k while not making the lives of external developers too painful.
This allows the groundwork for allowing users to implement plugins for sources and environments. A real API specification for sources and environments has been started, and sources and environments can be defined at runtime. In the long run r10k will add a plugin system for loading additional code from Rubygems and other sources, so r10k will be extensible without requiring modifications to the source.
2014/09/14
(GH-188) Call puppet module tool install with --force for downgrades
2014/07/27
(GH-173) Fixed a bug with Ruby 1.8.7 with Pathname objects.
2014/07/16
(GH-165) r10k puppetfile only consumes handled command line options.
Previously, passing -v or other commands when running r10k puppetfile *
could result in this error:
r10k puppetfile install --help --trace
Error while running: #<RuntimeError: Unrecognized options: help>
This was due to overly greedy code passing in all options from the command line to the TaskRunner. This has been fixed so only known options are passed along, and options that aren't relevant (such as :verbose) will be ignored.
(GH-158) Log levels are now documented in the command line --help pages.
(GH-137) Git remotes are now correctly updated.
A regression in the Git remote handling meant that git remotes would never be properly updated when switching Git environments and modules from one remote to another, and the git alternates file was never updated properly. This has been fixed so that when the Git remote is updated, all references to the remotes and alternates will be updated.
(GH-163) All Git tags are deleted when switching Git remotes
Git tags cannot necessarily be transferred from one Git repository to another, so when a Git repo has its remotes changed all tags are deleted to prevent stale tags from overwriting tags from the new repo.
2014/04/21
(GH-93) r10k deploy subcommands now respect the --help flag.
(GH-100) The addition of a faster command execution library was POSIX centric, but there were a number of users that were running r10k on Windows, which brike their environments. Support for Windows has been re-added and Windows is now a supported platform for using the r10k puppetfile commands. Please note that r10k now requires Ruby 1.9.3 on Windows to function.
Thanks to Sam Kottler and Daniel Dreier for their their work on this release.
2014/03/07
Preliminary support for Puppetfile modules from SVN sources. SVN repositories can track the latest available revision or may be pinned to a specific revision.
Forge modules can now track the latest available version. This can be enabled by
setting the module version to :latest.
Git based Puppetfile modules can now be specified as branches, tags, and commits. When tags and commits are specified r10k can perform optimizations when updating the given repositories to reduce network accesses.
Command execution has been greatly improved. The old library for executing commands (systemu) had very high overhead and was 50 - 100 times slower than %x[] or fork/exec. It's been replaced with a custom process execution implementation.
Modules can swap out sources. When an existing module is changed from Forge to Git, for instance, the existing module will be removed before the new module is installed. (GH-30)
2014/02/27
Git repositories were not tracking their upstream remotes, repos should now properly update upstream changes.
Git reference clones now fetch their cache remotes immediately after the initial clone.
2014/02/08
Release Candidate 1 for 1.2.0
2014-02-24
This is a backwards compatible bugfix release.
- (GH-90) Multiple environments with the same name but with different sources were previously colliding and some environments were being ignored. This has been fixed and all environments should be deployed when updates are run.
Thanks to the following contributors for their their extraordinary patience and help in for chasing down GH-90:
- Andreas Ntaflos (antaflos)
- Igor Galić (igalic)
2014-01-26
This is a backwards compatible maintenance release.
- (GH-82) Added all git managed files, including README.markdown, CHANGELOG, and LICENSE to the gemspec for better compatibility with non-gem packages.
2014-01-06
This is a backwards compatible maintenance release.
- If Puppet and r10k are required in the same namespace, it's possible for the vendored copy of SemVer to conflict with the Puppet version. This was fixed by renaming the copy vendored in r10k and putting it under a namespace.
2013-12-11
This is a backwards compatible bugfix release.
- (GH-48) Environment prefixing always defaults to off. Users were already using r10k with multiple sources but in different directories, and prefixing breaks this behavior. Since this was a backwards incompatible change this has to be rolled back.
- (GH-64) Multiple sources in a single directory no longer purge each other.
Thanks to the following contributors for their help in 1.1.1:
- Stig Sandbeck Mathisen
- Gabriel M Schuyler
2013-09-30
This is a backwards compatible bugfix and feature release.
(GH-35) Puppetfiles can now specify a path to the moduledir, instead of assuming
'/modules'. It can be set with the Puppetfile moduledir directive. Note that
this is not compatible with librarian-puppet.
(GH-53) Multiple environment sources can now be specified in a single
directory. When multiple sources are specified, each environment will be
prefixed with the source name. This can be enabled and disabled with the
source prefix option.
(GH-45) Documentation has been greatly expanded.
(GH-56) New subcommand: r10k puppetfile check allows you to validate the
syntax of a Puppetfile.
(GH-66) Initial clones use git checkout when switching to a new branch
instead of just git reset; without this change it would look like the wrong
branch was checked out.
(GH-59) r10k can now pull from Pulp repositories for Forge based modules.
(GH-70) Handle unset HOME - in case that HOME is unset, assume that the current
user is root. This mainly occurs when r10k is being run as the prerun command
under Puppet.
The method mocking framework has been switched from mocha to rspec-mocks. Mocha is notoriously bad about breaking changes between versions and rspec-mocks is very robust, so Mocha has been ripped out and dropped as a dependency.
Rspec expectations now use the expect(thing).to syntax instead thing.should
A quasi settings framework has been extracted to make application settings less bad. In the long term a general application framework will be extracted from r10k to handle generic 'application' problems like this, but for now the settings framework is the way to handle singleton data.
R10K:Git::Cache object memoization has been extracted into a standalone class instead of being grafted onto the class. All hail the single responsibility principle!
R10K::Module code has been refactored. There's now a real base class instead of a hacky mixin with some metadata magic on top.
Thanks to the following contributors for their help in 1.1.0:
- Alex Linden Levy
- Abhay Chrungoo
- Adam Vessey
- Chuck Schweizer
- Elias Probst
- Greg Baker
- Jochen Schalanda
- Theo Chatzimichos
2013-05-30
This is a backwards incompatible bugfix and feature release.
The configuration file format of 0.0.9 should be compatible with 1.0.0, and any issues with that should be considered a bug.
A longstanding issue was confusion between symbols and strings in r10k.yaml (GH-18). To resolve this, symbols and strings will be treated equally and should produce the same behavior. In the long run, symbols will probably be deprecated for the sake of conformity.
A number of commands have been renamed. They still but will emit a deprecation
warning and will redirect to the new command implementation. The only
exceptions is the are the r10k environment cache and r10k environment stale
commands, but they were pretty much useless anyways.
Log level verbosity can now be specified by level name instead of level number.
If --verbose is passed without a level, it will set the log level to info.
r10k can be used to deploy modules from a standalone Puppetfile. See
r10k puppetfile for more information.
Modules without a version in the format of 'foo/bar' will be assumed. (GH-21)
r10k handles versioning according to SemVer; since this is a major release this is a backwards incompatible API change. It's unlikely that this has had any extensions written on top of it, but if you have, then heads up. However, all versions of 1.x should be backwards compatible.
A number of bugs were due to underlying architecture flaws. Part of 1.0.0 has been a significant architectural overhaul, so on top of all of the above changes there should be a lot of other bugs that have been fixed.