pyvenv-mode doesn't deactivate the virtualenv in buffers where pyvenv-activate isn't unset

[xenophonf]

I want to be able to open my Python projects in Emacs and automatically set up a virtual environment—but just for project-related buffers! To accomplish that, I've enabled the pyvenv minor mode by customizing the Emacs Lisp variable pyvenv-mode, and I've put the following code into my Python project's .dir-locals.el:

((nil . ((eval . (progn
                   ;; install or activate the development environment                                                                                                                        
                   ;; (requires pyvenv-tracking-mode)                                                                                                                                        
                   (set (make-local-variable 'my-project)
                        (locate-dominating-file default-directory ".dir-locals.el"))
                   (set (make-local-variable 'my-project-venv)
                        (concat my-project ".venv"))
                   (if (not (file-exists-p my-project-venv))
                       (let ((cwd default-directory))
                         (cd my-project)
                         (async-shell-command "make dev-infra")
                         (cd cwd)
                         (message "Please re-open this file/directory after the \"make dev-infra\" command finishes."))
                     ;; must be set project-wide to pre-commit work                                                                                                                          
                     (set (make-local-variable 'pyvenv-activate)
                          my-project-venv))))))
 (python-mode . ((eval . (progn
                           ;; sort imports, then style code                                                                                                                                  
                           (add-hook 'before-save-hook #'py-isort-before-save nil t)
                           (add-hook 'before-save-hook #'elpy-black-fix-code nil t))))))

It works beautifully. I can open files and folders in one Python project directory, and my directory-local variables automatically install my virtual environment or, if it's already installed, activate it. Opening files/folders related to another project automatically switches to the second project's virtual environment. Smooth as silk.

But if I want to open files/folders outside a project directory, i.e., in buffers where the local variable pyvenv-activate isn't set, I must manually run pyvenv-deactivate.

How do I configure pyvenv-mode to deactivate a virtual environment outside of the relevant buffers?