Robot Testing With Emacs
I have written about assess previously [@url:www.russet.org.uk/blog/3135] it is a tool which provides predicates, macros and functions to support testing for Emacs. It is actually agnostic to the test environment, although has specialised support for ERT.
My new release of assess (v0.3.2) includes one significant change, and
two new features. I have updated the call capture functionality --- the
first version stored all the call data in a global variable, which was
quick and easy, but clearly not a log term solution. It now uses
closures instead which means that several functions can be captured at
once. This also allows the first new feature, which is the ability to
capture calls to hooks, with the function
which takes a hook and a
lambda, and returns any calls to the hook
lambda is evaluated. As an example usage, from
(should (equal '(nil nil) (assess-call-capture-hook 'assess-call-test-hook (lambda () (run-hooks 'assess-call-test-hook) (run-hooks 'assess-call-test-hook)))))
This is good functionality and should be very useful. The API could be
improved a bit; a macro version would avoid the explicit
example. And returning a list of
nil means this function also works
with hooks with args, but is a bit ugly for hooks without (which are the
The second area that I wanted to address has come about because of my
hacking into the Emacs undo system. This is hard to test automatically;
I have often found myself writing things like this test from
(should (with-temp-buffer (setq buffer-undo-list nil) (insert "hello") (member (current-buffer) undo-auto--undoably-changed-buffers)))
This is okay, but it's painful to write; I am trying to robotize Emacs, and it's not easy. Some times it's hard to work out exactly what set of functions you need to call. It would be much easier just to type a key sequence and have Emacs run this for you.
Fortunately, Emacs has special support for this in the form of keyboard
macros; you can remember, store and save any set of keypresses and run
them, rerun them, automate them or, most importantly, save them to a
file as a lisp expression. This example, for instance, comes from
(kmacro-call-macro nil nil nil [left ;; Delete "c" backspace left left left ;; Delete "a" backspace ;; C-/ or undo 67108911])
This is okay, but it's still not ideal. I have had to add comments to
make the test clear by hand. It's not easy to read, and what is that
67108911 about? It comes from somewhere in Emacs and is stable into
the future. But, you only have my word for it that this is
would be all too easy to get this wrong, to have the wrong comment.
Tests need to be readable.
Fortunately, Emacs provides a nice solution, in the form of
edmacro --- this is a major-mode for editing macros after they have been
created. It also defines a human readable version of a macro. We can
parse this and then execute it directly. This example comes from
(execute-kbd-macro (read-kbd-macro " C-c C-o ;; latex-insert-block RET ;; newline C-/ ;; undo "))
The advantage of this is that I didn't actually write this string; I recorded the macro, the edited it and copied the contents of the edmacro buffer.
This is still not easy enough, though; I want an easier way of editing
the macro as it appears in the test. This is, unfortunately, difficult
edit-kbd-macro is not easy to invoke programmatically --- it
absolutely hard-codes user interaction (I did even try invoking
edit-kbd-macro using a keyboard macro!). So, I have given up with that
approach in the short term. Instead, I have written a function
assess-robot-execute-macro that combines
execute-kbd-macro, but also sets the macro as the last macro, making
it easy to edit. I've also added a keybinding to edmacro to copy the
macro to the kill-ring. And here is a test using it:
(should (assess= "hello" (assess-robot-with-switched-buffer-string (assess-robot-execute-kmacro " hello ;; self-insert-command * 5 "))))
This also demonstrates the final quirk. Keyboard macros work in which
ever buffer is selected --- not the one which is current. We cannot use
with-temp-buffer to select on temporarily and run the macro in it. So
I have added macros to display a buffer temporarily instead.
As with many parts of assess, the back end is quite convoluted and complex, as many parts of Emacs were not written with testing in mind (that is they predate the whole idea of unit testing by many years!). But, I hope that the API that assess provides is simple, clear and flexible.
Feedback is, as always, welcome.