Update: Factual corrections to Robin Templeton’s work.
Update: Added an extra set of benchmarks for Guile 3 in a Debian Sid Docker container.
Disclaimer: I don’t use Guile. I hardly know it. There are other Scheme implementations I know far better. But since Guile Emacs is a hot topic with much hopes and unproven claims, I experiment with it every now and then. All “benchmark” results here are to be taken with caution, they’ve been created using Guile 2.2.6 and Emacs 26.3 on a Thinkpad X230t running Arch Linux.
With that out of the way, laurus from #emacs reminded me that one of the reasons why Guile Emacs was possible in the first place is Guile’s language tower, with Emacs Lisp being one of the supported languages. But what does that mean? How complete is the Emacs Lisp support? What can it be used for? These and a few other questions are the topic of this blog post.
In need of a spec
Standardized programming languages have the great benefit of being based on a specification one can consult whenever in doubt of how things are supposed to behave. This allows several competing implementations to be developed, with their own unique strengths and benefits. if you adhere to the standard, switching between implementations isn’t hard and can help shaking out bugs, for example when compiling your C programs with different compilers.
Things get considerably harder if your chosen language decided to forego this approach and the correct behavior is defined by it, yet this didn’t stop people from writing alternative implementations for programming languages such as Python and Ruby. Emacs Lisp got into a similar situation ever since Guile was extended to the degree of supporting Emacs Lisp as an additional language. Provided your version of Guile is new enough, you can evaluate trivial code in the REPL:
scheme@(guile-user)> (define foo 1) scheme@(guile-user)> foo $1 = 1 scheme@(guile-user)> ,L elisp Happy hacking with Emacs Lisp! To switch back, type `,L scheme'. elisp@(guile-user)> (defvar bar 2) $2 = bar elisp@(guile-user)> bar $3 = 2
So far so good. But how much of Emacs Lisp is supported? Not much apparently, many common functions like message and error are unbound. It doesn’t seem possible to do anything with buffers or files either. This greatly limits the possibilities of writing useful scripts in Emacs Lisp. One way of determining what exactly is supported would be consulting the source code, but where’s the fun in that when you could instead test it programmatically, thereby creating an executable spec…
Generating the spec
The usual test approaches fail me. Reading test inputs via stdin with read-string? Accessing the arguments with argv? Reading from a file with insert-file-contents? Obtaining an environment variable with getenv? None of that is supported. At least you can print to stdout with princ. I went for a slightly different approach instead, obtain a list of functions/variables in a minimal Emacs environment, then generating a test file that checks their existence and prints a test summary. Here’s the code generation part:
(defun printf (fmt &rest args) (princ (apply 'format fmt args))) (printf ";; elisp spec adherence test (defvar passed 0) (defvar failed 0) (defun test-sym (pred sym) (if (funcall pred sym) (setq passed (1+ passed)) (setq failed (1+ failed)))) (defun test-fun (sym) (test-sym 'fboundp sym)) (defun test-var (sym) (test-sym 'boundp sym))\n\n") (mapatoms (lambda (atom) (when (fboundp atom) (printf "(test-fun '%S)\n" atom)) (when (and (not (keywordp atom)) (boundp atom)) (printf "(test-var '%S)\n" atom)))) (printf "\n") (printf "(princ \"Passed: \")\n") (printf "(princ passed)\n") (printf "(terpri)\n") (printf "\n") (printf "(princ \"Failed: \")\n") (printf "(princ failed)\n") (printf "(terpri)\n")
Assuming it’s been saved as gen-elisp-spec.el, the executable spec itself is generated with emacs -Q --batch --script gen-elisp-spec.el > elisp-spec.el. Here’s a test session using Emacs and Guile:
[wasa@box ~]$ time emacs -Q --batch --script elisp-spec.el Passed: 9567 Failed: 2 emacs -Q --batch --script elisp-spec.el 0.10s user 0.02s system 99% cpu 0.117 total [wasa@box ~]$ time guile --language=elisp elisp-spec.el Passed: 137 Failed: 9432 guile --language=elisp elisp-spec.el 77.62s user 0.27s system 103% cpu 1:15.04 total
This is kind of surprising. I didn’t expect Emacs to fail its own test and didn’t expect Guile to implement this little either. Most surprising is the abysmal speed of the test, I’m looking forward to anyone being able to explain that part to me. Here’s one more test using the official Debian Sid Docker image with Emacs 26.3 and Guile 3.0.2:
root@d27668492764:/# time emacs -Q --batch --script elisp-spec.el Passed: 9108 Failed: 2 real 0m0.104s user 0m0.097s sys 0m0.007s root@d27668492764:/# time guile --language=elisp elisp-spec.el Passed: 137 Failed: 8973 real 6m20.950s user 10m32.294s sys 0m7.846s
This is not exactly an improvement. At least the numbers are small enough to print out the offending symbols, for Emacs it’s atom and printf (which polluted the test environment), for Guile I’ve taken the liberty of annotating the list:
;; binding let let* ;; functions lambda apply funcall ;; evaluation eval load eval-and-compile eval-when-compile ;; sequences aref aset make-vector nth ;; sequencing progn prog2 prog1 ;; iteration dolist while ;; control flow if when unless cond ;; short-circuiting or and not ;; explicit nonlocal exit throw catch ;; exceptions signal condition-case unwind-protect ;; input read-from-minibuffer ;; output prin1-to-string print princ send-string-to-terminal terpri ;; cxr car cdr caar cadr cdar cddr car-safe cdr-safe nthcdr ;; associations assoc assq ;; search member memql memq ;; destructive list processing nreverse setcar setcdr rplaca rplacd ;; other list processing cons list make-list ` mapcar mapc append concat reverse length ;; symbols defconst defvar defun defmacro get put fset set setq setplist symbol-function symbol-name symbol-plist symbol-value intern make-symbol fmakunbound makunbound quote function ;; plist plist-get plist-put lax-plist-get lax-plist-put plist-member ;; strings string string-match substring upcase downcase ;; predicates zerop floatp stringp numberp integerp wholenump boundp fboundp functionp symbolp consp listp nlistp atom null ;; math 1+ 1- fceiling ffloor ftruncate fround float abs min max ;; comparators > < >= <= /= = eq eql equal string-equal string= ;; numerical operators + - * % ;; misc random
Some notable omissions and differences:
- No division. Most likely incompatible with Scheme’s numeric tower.
- Input is read with read-from-minibuffer, not read-string.
- send-string-to-terminal is unusual to have, but most likely the primitive output function.
- string-match is nice to have, but of limited use without match-string.
- prin1-to-string exists, prin1 doesn’t.
- print doesn’t add newlines and behaves like prin1 should.
To do anything outside of textbook exercises you’d need to define extra primitives. Guile’s module/language/elisp/boot.el shows how to apply a band-aid on some of the previous shortcomings:
(fset '/ (@ (guile) /)) (fset 'read-string 'read-from-minibuffer) (fset 'prin1 (@ (guile) write)) (defun print (object) (prin1 object) (terpri))
What does this mean for Guile Emacs?
What I’ve shown above is barely sufficient to bootstrap an Emacs on top of it. Guile Emacs requires a customized version of Guile and Emacs, then loads up the supporting Emacs Lisp files to do the rest. There are more incompatibilities, like called-interactively-p being stubbed out. Extending the presented rudimentary spec to contain actual tests would help with tracking progress and usability. It might even improve the overall quality of GNU Emacs itself, provided that the core developers are on board and believe in the idea. I’ve briefly searched emacs-devel for previous discussion on the topic, but only found bikeshedding about Guile Emacs itself, so anyone who feels strongly about the subject, feel free to start a discussion there!
With regards to Guile Emacs itself, the situation is trickier. The above repositories have not been touched for five years, with Robin Templeton being the sole contributor for five Google Summer of Code events. Even though the work is far from complete, it is impressive what a college student managed to do under supervision of Guile’s maintainer Andy Wingo and Ludovic Courtès. Further advancements require similarly motivated individuals to participate in the Guile community and become part of the effort, much like with other free software projects. It’s tempting to take a shortcut like donating to other developers, but unless they’ve figured out a way of converting that money into equivalent work, there will be little connection between what you give away and what they do in return. This again is a topic worth discussing, preferably with the people that can make a change.
|||laurus did some research as well, you can find an interesting discussion on the #guile channel: http://logs.guix.gnu.org/guile/2020-05-16.log|
|||At least you could now solve SICP in Emacs Lisp with less footguns: You have bignums, lexical scoping by default and TCO!|
|||This isn’t exactly correct, what’s tested for is whether the symbol has its function/value slot bound which may contain other things, for example macros and keywords.|
|||Consider that people like to advocate for Guile Emacs with the argument that it will make for a faster Emacs. While this may hold true in the long term, it’s nowhere near close to that yet. Here’s hoping that Guile 3 will alleviate some of the pain…|