Small-time Patching


Update: Added the improved backtrace

Update: Merged!

Today #emacs reminded me of an oddity in Emacs I’ve sort of learned to live with: Backtraces are, well, see for yourself:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
  +(1 t)
  eval((+ 1 t) nil)
  eval-expression((+ 1 t) nil)
  call-interactively(eval-expression nil nil)

I can live with errors printed as a list. What I can’t live with is none of the call stack lines being printed as a S-Expression… To fix this, one must dive a bit deeper than usual as only the debugger porcelain is implemented in Emacs Lisp. Its workhorse is backtrace in eval.c:

DEFUN ("backtrace", Fbacktrace, Sbacktrace, 0, 0, "",
       doc: /* Print a trace of Lisp function calls currently active.
Output stream used is value of `standard-output'.  */)
  union specbinding *pdl = backtrace_top ();
  Lisp_Object tem;
  Lisp_Object old_print_level = Vprint_level;

  if (NILP (Vprint_level))
    XSETFASTINT (Vprint_level, 8);

  while (backtrace_p (pdl))
      write_string (backtrace_debug_on_exit (pdl) ? "* " : "  ");
      if (backtrace_nargs (pdl) == UNEVALLED)
          Fprin1 (Fcons (backtrace_function (pdl), *backtrace_args (pdl)),
          write_string ("\n");
          tem = backtrace_function (pdl);
          Fprin1 (tem, Qnil);   /* This can QUIT.  */
          write_string ("(");
            ptrdiff_t i;
            for (i = 0; i < backtrace_nargs (pdl); i++)
                if (i) write_string (" ");
                Fprin1 (backtrace_args (pdl)[i], Qnil);
          write_string (")\n");
      pdl = backtrace_next (pdl);

  Vprint_level = old_print_level;
  return Qnil;

Despite the rather unusual look and weird naming, it’s not too hard to find the culprit. Most of the time is spent inside a loop that walks through the call stack, accesses the top-most function and args with backtrace_function and backtrace_args, prints a lisp object with Fprin1 (which is just another way to use prin1 from C code) and writes out normal strings with write_string. Qnil refers to the global nil symbol, tem is a naming convention for a temporary variable. It should be sufficient to print the opening paren first, then the function, a space and proceed normally from that point:

tem = backtrace_function (pdl);
write_string ("(");
Fprin1 (tem, Qnil); /* This can QUIT.  */
write_string (" ");

Time to recompile Emacs with make, boot the binary with src/emacs -Q, and trigger a backtrace with M-: (+ 1 t). Unfortunately that does not yield a prettier backtrace yet, but rather a Search failed: "\n debug(". As the debugger is busted, I had to resort to ag to find where exactly the breakage occurs. It’s not too hard to fix, mind you, all you have to do is to patch debugger-setup-buffer in debug.el to search for "\n (debug" instead.

The result is the following teensy patch:

diff --git a/lisp/emacs-lisp/debug.el b/lisp/emacs-lisp/debug.el
index 22a3f39..4020620 100644
--- a/lisp/emacs-lisp/debug.el
+++ b/lisp/emacs-lisp/debug.el
@@ -279,7 +279,7 @@ That buffer should be current already."
   (goto-char (point-min))
   (delete-region (point)
-              (search-forward "\n  debug(")
+              (search-forward "\n  (debug")
               (forward-line (if (eq (car args) 'debug)
                                      ;; Remove debug--implement-debug-on-entry
                                      ;; and the advice's `apply' frame.
diff --git a/src/eval.c b/src/eval.c
index 72facd5..e32e7a1 100644
--- a/src/eval.c
+++ b/src/eval.c
@@ -3409,8 +3409,9 @@ Output stream used is value of `standard-output'.  */)
      tem = backtrace_function (pdl);
-     Fprin1 (tem, Qnil);   /* This can QUIT.  */
      write_string ("(");
+     Fprin1 (tem, Qnil);   /* This can QUIT.  */
+     write_string (" ");
        ptrdiff_t i;
        for (i = 0; i < backtrace_nargs (pdl); i++)

And an IMHO vastly improved backtrace:

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p t)
  (debug error (wrong-type-argument number-or-marker-p t))
  (+ 1 t)
  (eval (+ 1 t) nil)
  (eval-expression (+ 1 t) nil)
  (funcall-interactively eval-expression (+ 1 t) nil)
  (call-interactively eval-expression nil nil)
  (command-execute eval-expression)

Not sure whether to bother submitting this… Let me know what you think!

Haunted by Hermann Zapf


Linux gives me, for worse or better, the ability to coerce nearly every application into picking any font. The respective component in its font stack is the aptly named Fontconfig software. This is good because some applications insist on fonts I don’t like and bad because sometimes my configuration does the wrong thing, resulting in bizarre text rendering problems.

One of those has plagued me for months. When using i3lock, the unlock indicator used a swirly, cursive font from the TeX Gyre project instead of my default sans-serif font. I’ve eventually identified it as TeX Gyre Chorus which appears to be a libre variant of ITC Zapf Chancery. Why would i3lock ever get the idea to use that of all the fonts? It didn’t make much sense, the mere act of making TeX fonts available to the rest of the system shouldn’t do something this drastic…

Eventually I’ve had the idea to use the FC_DEBUG environment variable with 4096 as value, this revealed that the query was empty or in other words, no font was set at all. For some inexplicable reason, fc-match "" returns something entirely different.

Thanks to Cairo and its simple API, it didn’t take me long to figure out a fix. While it recommends to use Fontconfig directly for serious usage, I’d rather not. Just see for yourself at FcPatternBuild and recoil in horror.

Lisping the ChucKian Way


I did it again! For the uninitiated, ChucK is a special-purpose programming language intended to be used for sound synthesis. While working on a college assignment, it occurred me that the language had just the minimum amount of features to be leveraged for my second MAL implementation (previously): Console/File I/O, an object system reminiscent of both Java and C++, regular expressions and arrays. How hard could it possibly be to write the probably most advanced ChucK program in existence?

This turned out to be a good deal more annoying than my first implementation. The major problem besides the language giving you minimal support for non-audio programming was that clearly nobody else did use it for a project of this size. I came to this conclusion after reporting a number of bugs that should have been absolutely obvious to anybody using the standard library for anything else than creating music.


I didn’t expect the first obstacle to be the very act of loading code from another file, something that should surely be supported well for a system also used in live programming. It turns out that live programming is a very stretchable term; what they do is more akin to hotswapping invidual code units in terms of files which is cute, but not what I need. There is a Machine.add(file) facility available from code, however it does not immediately load code from the specified file and instead post-pones loading after the current file has been loaded up.

I’ve pondered whether to create a file solely consisting of these instructions via make, then decided against it in favor of an upfront loading approach where a runner script extracts “magic” comments indicating the dependencies and boots chuck with all of them in that order and the file they originate from. While this isn’t ideal[1], it works surprisingly well.

I/O and time

Getting user input was also tricky. Initially I tried out the HID example just to find out that it would only work for special input devices as opposed to console input. Therefore I tried out the ConsoleInput class which gives a “hacked” thing to read from. Due to it having some oddities such as not accepting C-d (and therefore only being terminable with C-c which brings down the entire process), behaving incorrectly when nested and adding an extra space to the prompt, I hacked my own thing together with the KBHit class, an ASCII table and some flushing to standard output.

Speaking of printing, you would expect the manual to tell you how one does that… Instead you are told about the debug output syntax <<< foo >>>;, I’ve had to consult the VERSIONS file to learn that one can send strings to chout. Another wonderful gotcha was that due to some RtAudio bug, you can get spurious errors about your audio stream still running which mess up the prompt. The only way to get rid of them is starting the process with --silent which just runs everything as fast as possible resulting in 100% CPU usage. Fun.

Finally, I’ve hunted for a way to measure time for the performance tests, but learned that ChucK’s notion of it is more about coordination of sound. In silent mode, it is therefore useless. However not all hope is lost as you can shell out to date and retrieve its output in a hacky way: Std.system(command) doesn’t return anything useful, so you need to redirect to a file and read from it instead…


ChucK is somewhere between a traditional compiled and interpreted language. I did run into a good amount of exceptions, but was surprised that I couldn’t throw, create or catch any. This is saddening me as it forces one to return errors and check for them very often, be it in form of integers and out parameters (hello, C!) or a dedicated error object.


I’m pretty used to have at least some way to pass functions around and expected ChucK to be the same considering that the debug syntax had a way of printing functions. However the language doesn’t have any function type, so while you can coerce one into an Object, it won’t do you any good. Therefore I went for the C# solution and implemented functors, that is, classes with a call(args) method which one can instantiate and pass around. Yuck.


While one can do Java-like OOP, it is severely limited. Considering that there are no generics, no super, no interfaces, no unions, no casting to arrays, no self-references, no automatic boxing or boxed primitives, an impoverished static keyword and no private, the resulting code is clumsy, yet has a certain air to it which I’d call “ChucKian”, like a few other people did on the chuck-users mailing list. The most impressive collection I’ve found so far is LicK.


There is a strict distinction between reference and value types which require using either the @=> or => operators. While the manual insists strings are reference types, one can use => just fine on them…

While there are no hashmaps, one can use an array with string keys and store or retrieve objects of the array type. What doesn’t work though is retrieving or even iterating over the keys. For this reason I wrote a terrible hash table implementation using regular arrays.


I’m not impressed by the compiler. It doesn’t catch things like missing returns, blatant type system abuse and OOP mistakes that result in segfaults or thrown assertions. While many hate Java backtraces, not having any is worse. Scoping is most certainly not lexical and forced me to pick more unique names in a few places.


I’ve dragged out this project far too long, but have been happy to learn that it is possible to implement MAL in a language as weird as ChucK. Next time I’ll hopefully pick a more featureful language, like SuperCollider

[1]The most obvious reason is that files may not contain cyclical dependencies, a less obvious one is that due to the approach of at most one class per file, one ends up with comically large command lines.



I’ve released the third and last egg in my series of wrappers for GUI libraries! Maybe there will be more, but none I’ll work on full-time.

This time I haven’t had the joy of working with a well thought-through API and ran into more issues than before. I’ll focus on the memory-related ones as it’s a topic I haven’t found much about online. To summarize, I’ve identified at least three different ways of managing memory in my wrapper:

  1. Foreign-managed pointers to heap-allocated memory:

    This is the easiest case. Some foreign function is called and it returns a pointer towards the data you’re interested in. Fortunately, you don’t need to clean up afterwards as it firmly remains under its control.

    (define widget-table (make-hash-table))
    (define (dispatch-event! widget* type)
      (match-let* ((widget (hash-table-ref widget-table widget*))
                   (handlers (widget-handlers widget))
                   ((handler . args) (hash-table-ref handlers type)))
        (apply handler widget args)))
    (define-external (libui_WindowClosingHandler (uiWindow* window*) (c-pointer data)) bool
      (dispatch-event! window* 'closing))
  2. Self-managed pointers to heap-allocated memory:

    Slightly bit harder. A foreign function expects a pointer to a long-lived object that you maintain yourself. First you allocate heap memory with good ol’ malloc, then expose free explicitly, implicitly with a finalizer or in a way combining both:

    (define (new-area-handler draw-handler mouse-event-handler mouse-crossed-handlerdrag-broken-handler key-event-handler)
      (let* ((area-handler* (allocate uiAreaHandler-size))
             (_ ((foreign-lambda* void ((uiAreaHandler* handler))
                   "uiAreaHandler *h = handler;"
                   "h->Draw = libui_AreaDrawHandler;"
                   "h->MouseEvent = libui_AreaMouseEventHandler;"
                   "h->MouseCrossed = libui_AreaMouseCrossedHandler;"
                   "h->DragBroken = libui_AreaDragBrokenHandler;"
                   "h->KeyEvent = libui_AreaKeyEventHandler;")
             (area-handler (make-area-handler area-handler* draw-handler mouse-event-handler mouse-crossed-handler drag-broken-handler key-event-handler)))
        (hash-table-set! area-table area-handler* area-handler)
        (set-finalizer! area-handler area-handler-free!)))
    (define (area-handler-free! area-handler)
      (and-let* ((area-handler* (area-handler-pointer area-handler)))
        (free area-handler)
        (area-handler-pointer-set! area-handler #f)))

    The latter was a bit difficult as libui is too smart and attempts detecting memory leaks on its own when the uiUninit function is called. If that precedes the finalizers, the application crashes. Not too hard to hack around though:

    (define (new-path #!optional alternate?)
      (let* ((flag (if alternate? uiDrawFillModeAlternate uiDrawFillModeWinding))
             (path* (uiDrawNewPath flag)))
        (set-finalizer! (make-path path*) path-free!)))
    (define (path-free! path)
      (and-let* ((path* (path-pointer path)))
        (uiDrawFreePath path*)
        (path-pointer-set! path #f)))
    (define (uninit!)
      ;; run all pending finalizers
      (gc #t)
  3. Self-managed pointers to stack-allocated memory:

    Now things get weird. In C, the default mode of operation is using the stack for allocating data. There isn’t really an equivalent in the vast majority of dynamic languages where nearly everything is a heap-allocated object. I’ve tackled this problem previously by using the free/malloc strategy, later by writing code that unpacked the stack-allocated struct into simple values, then reassembled them to a struct at call-time. The former ignores the semantics of stack-allocation and the latter is limited as it falls apart when the data needs to live on for longer than a function call. Clearly I needed a different solution!

    It turns out that you can use CHICKEN’s garbage collector in your favor. It is fairly fast when it comes to allocating more memory as it is using the C stack and alloca to store Scheme objects. Therefore, all we’d need is a way to use a Scheme object in foreign code as it will be subject to Scheme’s scoping rules and can be reclaimed once it’s no longer accessible. For this you create a blob of the right size, then hand a locative pointing to it to the foreign function expecting a pointer and fill your data in:

    (define-record brush storage)
    (define (brush-pointer brush)
      (make-locative (brush-storage brush)))
    (define uiDrawBrush-size (foreign-type-size (struct "uiDrawBrush")))
    (define (new-solid-brush r g b a)
      (let* ((brush (make-brush (make-blob uiDrawBrush-size)))
             (brush* (brush-pointer brush)))
        ((foreign-lambda* void ((uiDrawBrush* br) (double r) (double g) (double b) (double a))
           "br->Type = uiDrawBrushTypeSolid, br->R = r, br->G = g, br->B = b, br->A = a;")
         brush* r g b a)

    An alternative way is storing the locative inside the record and omitting the pointer procedure. While this variant requires a bit less code, and avoids creating locatives on demand, it’s not entirely safe to use due to bug #1293.



It’s been two weeks since my last blog post and I’ve released my second CHICKEN egg! Like the previous one, it’s a wrapper for a GUI library, albeit a hugely different one as it follows the immediate mode (as opposed to the retained mode) approach. This design decision resulted in a number of differences:

  • Very little hidden state
  • No caching involved, all widgets are redrawn at each frame
  • The UI is heavily integrated into your program and builds upon its data
  • No callbacks, widgets that can be triggered return interesting values instead
  • It becomes ridiculously simple to create custom widgets
  • It’s trivial to add custom behavior to widgets

For examples of the last two, see the calendar widget, the password input box and the user-customizable layouts in the overview demo. Reimplementing these in a traditional UI toolkit would have been considerably harder, if not even impossible. Now consider the challenge of implementing a GUI editor. With this approach it is a matter of creating new data structures on demand that get rendered as widgets and represent their internal state, then save it to disk when asked to. I find this way of thinking incredibly liberating and would recommend anyone to give it a try, even if just in the form of a data-driven game.

Unlike with the kiwi egg, I had to step up my C binding skills. My old trick to use helper functions that stack-allocate data, then invoke the real function, did not work as they needed to live across multiple function calls. Instead I learned that one can use blobs as storage managed from Scheme and use a locative to it for work from C. As long as one doesn’t do funny things like using the results as hash table keys anywhere it works surprisingly well. Other than that I’ve used the C API here and there to pass around Scheme values that were impossible to represent directly with the FFI.

The greatest problem I’ve encountered was a lack of documentation. Fortunately I’ve figured out everything necessary myself, but I still have the impression that I’m missing out on something. I hope to get in contact with upstream regarding the bugs I ran into, otherwise I’ll have to give dear imgui a try…