Fixing My #1 Annoyance With Emacs Lisp

26/08/2018

Ah, Emacs Lisp. There are many reasons for loving and hating it. I disagree with most people name when they argue why the language sucks[1], for me it’s mostly two things that end up mattering in practice:

  1. The APIs are terrible. Font-locking is an enigma. It’s common for packages to use synchronous APIs because it’s far easier to do than The Right Thing™. Moving through buffers and editing them makes for incomprehensible and stateful code. I could go on, but most of these can be mitigated by writing your own APIs as you figure things out. This is not what this blog post is about.
  2. There is no namespace or module system. This means that every global identifier could end up clashing with another one unless you emulate namespacing by adding a unique prefix. While this could be fixed, it’s unlikely to happen[2]. Interestingly enough this situation is similar to C, but worse as there’s no visibility control, only the convention of using a double dash for global identifiers not considered public. This annoys me as I have to type out a potentially long prefix every time. This is what this blog post is about.

I initially considered one of the namespace packages. It would make for as little typing as possible, however this would require an additional dependency and break my existing workflows. Therefore I went for the alternative route, writing a command that inserts the package prefix of the current buffer at point. Bind that command to an easily reachable key binding and you’d save nearly as much effort with typing.

(defvar-local my-current-package-prefix nil)

(defun my-ensure-trailing-dash (string)
  (if (and (not (zerop (length string)))
           (not (= (aref string (1- (length string))) ?-)))
      (concat string "-")
    string))

(defun my-guess-current-package-prefix (arg)
  (save-excursion
    (goto-char (point-min))
    (if (and (not arg)
             (re-search-forward "^(defgroup \\(\\w+\\)" nil t))
        (setq my-current-package-prefix
              (my-ensure-trailing-dash (match-string 1)))
      (setq my-current-package-prefix
            (my-ensure-trailing-dash
             (read-string "Package prefix: "
                          my-current-package-prefix))))))

(defun my-insert-current-package-prefix (arg)
  (interactive "P")
  (when (or (not my-current-package-prefix) arg)
    (my-guess-current-package-prefix arg))
  (insert my-current-package-prefix))

(with-eval-after-load 'elisp-mode
  (define-key emacs-lisp-mode-map (kbd "C-.")
              'my-insert-current-package-prefix))

Guessing the prefix is done by looking for a (defgroup ...) form which is a good enough indicator for a prefix[3]. In case it’s not given, the above code prompts for a prefix and allows resetting it with a prefix argument. The trickiest part is ensuring the prefix ends with a dash. You could optimize this even further by looking whether a prefix has already been inserted, but honestly, undoing the change is simple enough.

Let’s see whether this reignites my drive to write more Emacs packages…

[1]Who cares if it’s slow? Who cares about the lack of regex literals? Yes, it’s not <insert your favorite language>. Despite all of this people wrote lots of it, far more than any of the haters would. Feel free to dream about an Emacs rewritten in something else, but it’s going to stay a pipe dream if that’s all you do. The topic deserves a separate blog post because it’s a common phenomenon in the Emacs community to place irrational hopes in a re-implementation to succeed the status quo.
[2]The topic came up on emacs-devel before, the main problem is that the tooling would need to be updated. Simple workflows the core team is used to (such as grepping the qualified name) would completely break apart.
[3]An even better indicator would be the :prefix option inside (defgroup ...), but let’s not go overboard.

Installing Custom Certificates on Arch Linux

23/08/2018

One day a 1337 h4xx0r at the hackerspace[1] asked me whether I could relay a message to the hackint IRC network. This had one catch, for some reason the hackint operators insist that you only connect via TLS and use their self-signed intermediate and root certificates. I did a cursory verification[2], then looked into how to do a system-wide installation of the two .crt files. To my surprise, there is no Arch Linux wiki article on the topic, so here’s a short guide.

% cp *.crt /etc/ca-certificates/trust-source/anchors
% update-ca-trust

Merely copying the files into the destination directory isn’t enough, you need to run update-ca-trust to pound them into a form OpenSSL and friends can deal with. If everything went correctly, you’ll find new symlinks in /etc/ssl/certs with the issuer’s name in them. FWIW, something similar happens on every system update thanks to Pacman’s update-ca-trust hook. Check /usr/share/libalpm/hooks/update-ca-trust.hook for details.

[1]As hard as it is to believe, I occasionally meet actual hackers there. The kind that studies exploits, knows their way around shell code and breaks into computer systems.
[2]They’re quick to point out that comparing hashes doesn’t help you in case you get MITM’d. I tried verifying the certificate’s signature with GPG, but failed as I didn’t have anything useful imported in my local trust store.

Fixing My #1 Annoyance With Clojure

03/08/2018

Clojure is quite something. Immutable by default, functional style programming being encouraged and with many useful libraries written by a nice community. There’s ugly sides to it as well, like the error reporting being terrible, dealing with Java things and waiting for your tooling to boot. However, my greatest annoyance with it is that my preferred debugging workflow, printing out the problematic thing, is far more painful than it should be[1]. Suppose you have the following piece of code and want to check the result of the last form in the let.

(defn frobnicate [x]
  (let [y ...
        z ...]
    (filter foo? (map bar (baz x)))))

This won’t do as the function would return nil:

(defn frobnicate [x]
  (let [y ...
        z ...]
    (prn "XXX" (filter foo? (map bar (baz x))))))

This works, but what if the expression has a side effect or is computationally expensive?

(defn frobnicate [x]
  (let [y ...
        z ...]
    (prn "XXX" (filter foo? (map bar (baz x))))
    (filter foo? (map bar (baz x)))))

This is quite good, but annoying to type out and undo after it’s no longer needed:

(defn frobnicate [x]
  (let [y ...
        z ...
        xxx (filter foo? (map bar (baz x)))]
    (prn "XXX" xxx)
    xxx))

I’d love to have this:

(defn frobnicate [x]
  (let [y ...
        z ...]
    (dbg (filter foo? (map bar (baz x))))))

The idea is borrowed from Smalltalk, more specifically Super Collider where you can just call .debug on an object to see its value printed. Unlike a regular print function this one returns the object and does therefore allow inserting into a call chain just fine. Bonus: It accepts an optional argument for printing a prefix so that you can identify the debug output easily. Translating this to Clojure is as easy as it gets[2]:

(defn dbg
  ([thing]
   (dbg "XXX" thing))
  ([prefix thing]
   (println prefix)
   (print (with-out-str (clojure.pprint/pprint thing)))
   thing))

Now, this is useful. You can surround anything useful with (dbg ...) and if you wish, add a prefix as first argument. Removing the debug print is as easy as raising the enclosed S-Expression[3]. But how do you get this into a Clojure session? The thing is that once you eval this in a namespace, the function belongs to that namespace and to use it in another one you’d need to either import it from that namespace or define it in the other namespace. Another issue is that you’d need to edit the sources of a Clojure project to make use of this helper. Surely you can do better than that?

Studying the Leiningen documentation and the official Clojure docs on namespaces I learned about two more things:

  • Leiningen supports the :injections key which evaluates a vector of forms in the project context[4]
  • The lowest-level function to manipulate a different namespace than the current one is intern[5]

Put both together and you get the following snippet for your ~/.lein/profiles.clj:

{:user {:injections [(defn dbg ...)
                     (intern 'clojure.core 'dbg dbg)]}}

This is close to perfect. It will only work for projects using Leiningen obviously and displays a warning as the code is run twice, but it works nicely in any namespace!

[1]I’ve got to admit, this is quite petty. If I managed learning how to read ugly Java backtraces and studied the wonders of the Java class path, how could printf-debugging annoy me to this extent? I believe the conventional wisdom that it’s hardly worth automating tasks with little time savings misses the point, fixing long-standing annoyances however is a worthy goal. Better be happy than bitter about your setup.
[2]The eagle-eyed reader will notice that you could get by with a mere (clojure.pprint/pprint thing). The reason for the above is that I’ve encountered rather discomforting behavior in a codebase where pretty-printed output ended up interleaved with logging output. The easy workaround is making the pretty-printing atomic by collecting it into a string, then printing it out.
[3]If you use Paredit or Smartparens, it’s as easy as hitting M-r with point on the form you want to replace the outside one with.
[4]This doesn’t say anything about how often it’s actually evaluated, so better put an idempotent expression there.
[5]intern in Emacs Lisp does merely convert a string to a symbol. In CL it does the same, but allows specifying what package the symbol should belong to. In Clojure it takes a namespace, a name and a value…

Highlight Text Manually In LaTeX Slides

29/04/2018

Sometimes I’ve been in the situation that I have a text snippet where a splash of color would explain things in a far easier way than having to create a visualization or use a laser pointer. Imagine something like a backtrace, with different parts highlighted in different colors. This thing can be done pretty easily, even when using LaTeX indirectly, like when compiling it from Org in Emacs.

The following trick relies on using the minted package for highlighting. It supports an option for embedding LaTeX escapes into your code snippets. The documentation shows off a mathematic formula in a comment, however we can do far more, like using the \textcolor command from the xcolor package to insert colored text. Have a silly example:

\setminted{escapeinside=||}
\definecolor{green}{HTML}{218A21}

...

\begin{minted}[]{text}
|\textcolor{red}{RR}|
|\textcolor{green}{GG}|
|\textcolor{blue}{BB}|
\end{minted}

In an Org file you’d have to do a bit less typing (assuming you customized Org to always use minted):

#+LATEX_HEADER: \setminted{escapeinside=||}
#+LATEX: \definecolor{green}{HTML}{218A21}

...

#+BEGIN_SRC text
|\textcolor{red}{RR}|
|\textcolor{green}{GG}|
|\textcolor{blue}{BB}|
#+END_SRC

Bridging the Ancient and the Modern

23/03/2018

I tried out some new social networks lately. Mastodon I quite like (it’s like what I’ve wanted Twitter to be), Discord, not so sure. So, if you’ve wondered about my reduced presence on IRC, that’s why.

Writing an IRC bot is one of the classic programming exercises that can be done in pretty much every programming language offering you some way to open TCP sockets and manipulate strings. I started doing one in Emacs Lisp long time ago, although not from scratch (rather by leveraging an existing IRC client) and wondered whether there is anything to learn from doing the equivalent with a “modern” IM platform like Discord. Could it be still be done from scratch? What else is different about it?

First I had to find a meaningful thing for the bot to do. I chose Eliza, the classic demonstration of a chatter bot that managed fooling people into having prolonged conversations with them. The version I’m familiar with is M-x doctor which is part of Emacs. So, first of all, I wrote some code to interface with that command in a REPL-style fashion. A companion shell script boots up Emacs in batch mode for interfacing with the doctor from the shell. Much like the original mode, you terminate your input by pressing RET twice. This is an intentional design decision to allow for multi-line input as seen on Discord (but absent from IRC, where you could get away with making it single-line input).

I briefly entertained the thought of writing the rest of the bot from scratch in Emacs Lisp, but abandoned it after learning that I’d need to use websockets with zlib compression to subscribe and respond to incoming messages. While there is an existing library for websocket support, I’d rather not figure out its nitty-gritty details, let alone with the lack of zlib compression. It doesn’t help that Discord’s official API docs are inconclusive and fail answering questions such as how you can set your current status (and more importantly, why it fails getting updated). So, an officially recommended Discord library it is.

The choice on which one it’s going to be depended on whether the programming language it’s implemented with allowed me to communicate with my shell script. I tried out discord.js first, battled a fair bit with Node.js, but gave up eventually. There doesn’t seem to be a way to spawn a child process and read from / write to its stdout / stdin pipes as you see fit. Instead you can only add a callback for the process output and good luck if you want to figure out what piece of output corresponds to the input you wrote earlier. This is why I went for discordrb instead, wrote some glue code for subprocess communication and started figuring out their API to react to incoming messages.

There are a few lessons to be learned from their API:

  • Allow adding as many event handlers as you want for specific events, with convenience options for narrowing down to commonly needed situations (like messages starting with a prefix)
  • Inside these event handlers, provide an object containing all context you’d need, including the information who to respond to
  • Keep the bot alive when an event handler errors out

Now, to test the bot I needed to unleash it on a server. It turns out that unlike on IRC bot accounts are handled specially. You must:

  • Register an application and obtain an ID for authorization purposes
  • Enable a bot account and obtain an authorization token
  • Generate an URL for inviting the bot to a server
  • Share that URL with someone wielding the necessary permissions
  • Hope for the best and wait for them to invite the bot

This meant that I had to create my own test server to check whether my code worked at all. For this reason I haven’t been able to get it running on the server I was invited on. If you want to, you can try it on your own server, the sources are on GitHub as always.