Making scimax quicker?

[hummuscience]

Inspired by a recent reddit post about the speed of doom emacs, I was wondering if some of the options shipped with doom could also help out startup speed of scimax.

https://github.com/hlissner/doom-emacs/blob/665b627b7c07c8d29ec8d334588cecc2ba308248/docs/faq.org#how-does-doom-start-up-so-quickly

[jkitchin]

I would be happy to see scimax startup speed up. At the moment I am trying to transition to emacs-jupyter, and that will probably keep me busy for a few months. I would be happy to look at pull requests to speed things up though.

[shrysr]

I hope to look a little deeper into this, as the claim of Doom emacs has always fascinated me! However, fwiw : on my Mac: (scimax + my 'bloated' config) startup in ~10 seconds. I tangle 100+ babel source blocks as a part of my config each time. The scale made me think whether further optimisation effort was truly worth it, additionally considering it becomes somewhat better using the daemon + client arrangement. But we are all young.. of course it would add up and in the least, would certainly be so cool and efficient if scimax started up 'instantly' :) Perhaps, one generic direction would be some kind of delayed / deferred load of org-ref and all the heavier packages. I'd bet a lot of them are already deferred. Scimax's LOB probably adds a second or two. Weirdly, (emacs-init-time) always shows me 0.3 !? Perhaps something to do with GC ? Iirc there are separate packages to profile Emacs though, and will have to dig.

[hummuscience]

Currently my setup is taking 25s to load. I am wondering what is taking so long. Running some analysis tells me that my user.el file is taking 10 seconds to load while the rest of scimax is taking 15s.

Any idea how I can check what is taking so long for my user.el to load?

[hummuscience]

Here is my esup output:

Total User Startup Time: 22.102sec     Total Number of GC Pauses: 3     Total GC Time: 0.240sec

packages.el:544  9.599sec   43%
(when (and
scimax-load-user-dir
(file-exists-p (expand-file-name "user.el" scimax-user-dir)))
(load (expand-file-name "user.el" scimax-user-dir)))

packages.el:495  1.628sec   7%
(use-package scimax-lob
:ensure nil
:load-path scimax-dir)

packages.el:177  0.973sec   4%
(use-package elpy
:config
(elpy-enable))

packages.el:174  0.928sec   4%
(use-package elfeed)

packages.el:291  0.912sec   4%
(use-package mustache)

packages.el:421  0.886sec   4%
(use-package scimax-org
:ensure nil
:load-path scimax-dir
:bind
("s--" . org-subscript-region-or-point)
("s-=" . org-superscript-region-or-point)
("s-i" . org-italics-region-or-point)
("s-b" . org-bold-region-or-point)
("s-v" . org-verbatim-region-or-point)
("s-c" . org-code-region-or-point)
("s-u" . org-underline-region-or-point)
("s-+" . org-strikethrough-region-or-point)
("s-4" . org-latex-math-region-or-point)
("s-e" . ivy-insert-org-entity)
:init
(require 'scimax-org))

packages.el:308  0.886sec   4%
(use-package pdf-tools)

packages.el:320  0.874sec   3%
(use-package org-ref
:ensure nil
:load-path (lambda () (expand-file-name "org-ref" scimax-dir))
:init
(add-to-list 'load-path
(expand-file-name "org-ref" scimax-dir))
(setq bibtex-autokey-year-length 4
bibtex-autokey-name-year-separator "-"
bibtex-autokey-year-title-separator "-"
bibtex-autokey-titleword-separator "-"
bibtex-autokey-titlewords 2
bibtex-autokey-titlewords-stretch 1
bibtex-autokey-titleword-length 5
org-ref-bibtex-hydra-key-binding (kbd "H-b"))
;; (define-key bibtex-mode-map org-ref-bibtex-hydra-key-binding 'org-ref-bibtex-hydra/body)
;; (global-set-key (kbd "H-b") 'org-ref-bibtex-hydra/body)
)

packages.el:51  0.538sec   2%
(use-package auto-complete
:diminish auto-complete-mode
:config (ac-config-default))

init.el:100  0.530sec   2%
(let ((default-directory scimax-dir))
(shell-command "git submodule update --init"))

bootstrap.el:8  0.500sec   2%
(package-initialize)

bootstrap.el:8  0.500sec   2%
(package-initialize)

packages.el:256  0.486sec   2%
(use-package hydra
:init
(setq hydra-is-helpful t)

:config
(require 'hydra-ox))

packages.el:244  0.288sec   1%
(use-package helm-projectile)

[jkitchin]

I am surprised org-ref loads so fast. pdf-tools especially is very slow for me.

I don't have a very sophisticated way to check load times. There is a function (scimax-profile) but it is just a thin wrapper around esup.

My startup is about 20 seconds. Of that: pdf-tools is almost 4 seconds, my user-dir is about 3 seconds, and scimax-lob is almost 3 seconds.

I would have thought you could run esup on the file that loads your user directory too. Does that work for you?

[hummuscience]

Good point. I just tried and I get the following error:

ERROR: the child emacs had the following errors:
  ERROR(profile-sexp) at /usr/local/Cellar/emacs-mac/emacs-26.3-z-mac-7.8/share/emacs/26.3/lisp/calendar/calendar.elc:50 with sexp (byte-code

This seems a common issue with other emacs-kits such as spacemacs (jschaf/esup#48) and doom (doomemacs/doomemacs#788).

I do not, however, understand how to make it work for scimax as my elisp is weak. Any idea?

[hummuscience]

It is also an older issue in esup itself (jschaf/esup#46)

I revived that issue, lets see what comes of it.

[shrysr]

Actually, I just tried a few of the suggestions outlined in the Doom readme. In specific - I finished adding the gc-cons-threshold tricks. The only variation from the doom config required is the symbol doom-gc-cons-threshold, which has to be set explicitly to the previous gc-cons-threshold hook. This shaved off 1.2 seconds from my start up via esup (though I am not using vanilla scimax atm). From what I've studied of the remaining tricks - they seem written in a format that can be directly implemented in ~ any configuration, including Scimax.

Another note - Scimax's init.el already has the variations of gc-cons-threshold (start high and then scale down) implemented though the values are different. So - possibly unlikely to receive any benefit from this.

Actually, I guess the following can be plugged into scimax's init.el, replacing the gc-cons-threshold commands that are currently set in Scimax's init.el

(setq gc-cons-threshold most-positive-fixnum ; 2^61 bytes
      gc-cons-percentage 0.6)

(add-hook 'emacs-startup-hook
  (lambda ()
    (setq gc-cons-threshold 16777216 ; 16mb
          gc-cons-percentage 0.1)))

;; Raising gc-cons threshold when the minibuffer is active

(defun doom-defer-garbage-collection-h ()
  (setq gc-cons-threshold most-positive-fixnum))

(defun doom-restore-garbage-collection-h ()
  ;; Defer it so that commands launched immediately after will enjoy the
  ;; benefits.
  (run-at-time
   1 nil (lambda () (setq gc-cons-threshold 16777216 ; 16mb
          gc-cons-percentage 0.1) )))

(add-hook 'minibuffer-setup-hook #'doom-defer-garbage-collection-h)
(add-hook 'minibuffer-exit-hook #'doom-restore-garbage-collection-h)

Of particular note in the snippet above is the adjusting the gc-cons-threshold when the mini-buffer is active and immediately reducing it thereafter. This has removed the very slight lag I experienced while calling ~some helm commands like helm-M-x (even on Scimax) making helm almost comparable in response to counsel. In fact, my overall mini-buffer response seems to be much better !

Another specific note is the caching function implemented for packages.el approach. I am currently using straight.el which apparently does this by default.

I will try out the above in vanilla scimax and report.

[shrysr]

Unfortunately, esup does not seem to work for me with Scimax on Emacs 27 !? However, the load up time did seem marginally faster atleast. Anyway - I can confirm that the above snippet can be plugged into Scimax's init.el replacing the existing gc-cons-threshold commands without any apparent error.