## Archive for the ‘Emacs’ Category

### Robot Testing With Emacs

I have written about assess previously ; it is a tool which provides predicates, macros and functions to support testing for Emacs. It is actually agnostic to the test environment, although has specialised support for ERT.

My new release of assess (v0.3.2) includes one significant change, and two new features. I have updated the call capture functionality — the first version stored all the call data in a global variable, which was quick and easy, but clearly not a log term solution. It now uses closures instead which means that several functions can be captured at once. This also allows the first new feature, which is the ability to capture calls to hooks, with the function assess-call-capture-hook, which takes a hook and a lambda, and returns any calls to the hook when the lambda is evaluated. As an example usage, from assess-call-tests.el:

 (should (equal '(nil nil) (assess-call-capture-hook 'assess-call-test-hook (lambda () (run-hooks 'assess-call-test-hook) (run-hooks 'assess-call-test-hook)))))

This is good functionality and should be very useful. The API could be improved a bit; a macro version would avoid the explicit lambda, for example. And returning a list of nil means this function also works with hooks with args, but is a bit ugly for hooks without (which are the majority).

The second area that I wanted to address has come about because of my hacking into the Emacs undo system. This is hard to test automatically; I have often found myself writing things like this test from simple-test.el.

 (should (with-temp-buffer (setq buffer-undo-list nil) (insert "hello") (member (current-buffer) undo-auto--undoably-changed-buffers)))

This is okay, but it’s painful to write; I am trying to robotize Emacs, and it’s not easy. Some times it’s hard to work out exactly what set of functions you need to call. It would be much easier just to type a key sequence and have Emacs run this for you.

Fortunately, Emacs has special support for this in the form of keyboard macros; you can remember, store and save any set of keypresses and run them, rerun them, automate them or, most importantly, save them to a file as a lisp expression. This example, for instance, comes from viper-test.el.

 (kmacro-call-macro nil nil nil [left ;; Delete "c" backspace left left left ;; Delete "a" backspace ;; C-/ or undo 67108911])

This is okay, but it’s still not ideal. I have had to add comments to make the test clear by hand. It’s not easy to read, and what is that 67108911 about? It comes from somewhere in Emacs and is stable into the future. But, you only have my word for it that this is undo. It would be all too easy to get this wrong, to have the wrong comment. Tests need to be readable.

Fortunately, Emacs provides a nice solution, in the form of edmacro — this is a major-mode for editing macros after they have been created. It also defines a human readable version of a macro. We can parse this and then execute it directly. This example comes from simple-test.el.

 (execute-kbd-macro (read-kbd-macro " C-c C-o ;; latex-insert-block RET ;; newline C-/ ;; undo "))

The advantage of this is that I didn’t actually write this string; I recorded the macro, the edited it and copied the contents of the edmacro buffer.

This is still not easy enough, though; I want an easier way of editing the macro as it appears in the test. This is, unfortunately, difficult as the edit-kbd-macro is not easy to invoke programmatically — it absolutely hard-codes user interaction (I did even try invoking edit-kbd-macro using a keyboard macro!). So, I have given up with that approach in the short term. Instead, I have written a function assess-robot-execute-macro that combines read-kbd-macro and execute-kbd-macro, but also sets the macro as the last macro, making it easy to edit. I’ve also added a keybinding to edmacro to copy the macro to the kill-ring. And here is a test using it:

 (should (assess= "hello" (assess-robot-with-switched-buffer-string (assess-robot-execute-kmacro " hello ;; self-insert-command * 5 "))))

This also demonstrates the final quirk. Keyboard macros work in which ever buffer is selected — not the one which is current. We cannot use with-temp-buffer to select on temporarily and run the macro in it. So I have added macros to display a buffer temporarily instead.

As with many parts of assess, the back end is quite convoluted and complex, as many parts of Emacs were not written with testing in mind (that is they predate the whole idea of unit testing by many years!). But, I hope that the API that assess provides is simple, clear and flexible.

Assess is available at github and MELPA.

Feedback is, as always, welcome.

## Bibliography

### Emacs Testing with Assess

Assess is my new package supporting testing in Emacs. It has grown out of my frustration with the existing framework while building the lentic package .

For quite a while, the only testing framework in Emacs has been ERT (the Emacs Regression Testing tool) which is part of core. More recently, there have been a number of new ones arriving. For example, buttercup and ecukes both provide behaviour driven testing, rather like Jasmine or Cucumber respectively. Both worth looking at — I’ve used Ecukes for testing Cask, and it’s nicely implemented and very usable. Assess is rather less radical than this though. It focuses on providing a general set of tools for testing, mostly in terms of some macros and predicates that should be useful.

For example, a recurrent problem during the development of lentic was ensuring that no buffers were left around after a test. Particularly when a test fails, this can lead to unexpected failures in later tests. For this purpose, I have added a macro called assess-with-preserved-buffer-list. Any buffers at all created inside this macro will be removed after. Hence this:

 (assess-with-preserved-buffer-list (get-buffer-create "a") (get-buffer-create "b") (get-buffer-create "c"))

Which preserves the buffer state. a, b and c will be removed after the macro exits.

Assess also provides some handy predicates for testing. So, we can compare the contents of strings, buffer and files easily. For example:

 ;; Compare Two Strings (assess= "hello" "goodbye") ;; Compare the contents of Two Buffers (assess= (assess-buffer "assess.el") (assess-buffer "assess-previous.el")) ;; Compare the contents of Two files (assess= (assess-file "~/.emacs") (assess-file "~/.emacs"))

Again, this has all been done carefully to avoid changing state. The last example, should work whether ~/.emacs is open already or not, and will not result in new buffer creation.

As well as string comparison, we can also check that indentation is working correctly. This example, for instance, takes a indented string, un-indents and re-indents according to a specific mode, then checks that nothing has changed.

 (assess-roundtrip-indentation= 'emacs-lisp-mode "(assess-with-find-file\n \"~/.emacs\"\n (buffer-string))")

Likewise, we can check fontification with the assess-face-at= function. In this case, we are checking that three words get highlighted correctly.

 (assess-face-at= "(defun x ())\n(defmacro y ())\n(defun z ())" 'emacs-lisp-mode '("defun" "defmacro" "defun") 'font-lock-keyword-face)

Finally, in terms of test functions, I have recently added two new pieces of functionality. Call capturing was an idea I stole from buttercup — this is a non-interative equivalent to the function tracing, returning parameters and return values to a function call.

 (assess-call-capture '+ (lambda() (+ 1 1))) ;; => (((1 1) . 2))

And discover provides a drop in replacement for ert-run-tests-batch-and-exit except that it automatically finds and loads test files, based on a set of heuritstics. I’ve already started to use this instead of ert-runner, as it requires no configuration.

The final thing I wanted to address was better reporting. While everything described so far as test environment agnostic, I’ve only managed to advance reporting for ERT. All of the functions that I have written plug into ERT, so produce richer output. So instead of:

 F temp (ert-test-failed ((should (string= "a" "b")) :form (string= "a" "b") :value nil))

When comparing "a" and "b", this output is fine, but if the strings are more complex (say, for example, the a short piece of code that you expect to indent in a certain way), it is hard to work out what the differences are.

So, assess extends ERT so that that it now calls diff on the strings. We now get this explanation instead:

 F test-assess= (ert-test-failed ((should (assess= "a" "b")) :form (assess= "a" "b") :value nil :explanation "Strings: a and b Differ at:*** /tmp/a935uPW 2016-01-20 13:25:47.373076381 +0000 --- /tmp/b9357Zc 2016-01-20 13:25:47.437076381 +0000 *************** *** 1 **** ! a \\ No newline at end of file --- 1 ---- ! b \\ No newline at end of file "))

Verbose for sure, but very useful for identifying issues, especially white space related. By default, this uses the diff command, but this is also extensible, and has a simple fallback in its absence.

There is more functionality to assess than that shown here: it can create multiple temporary buffers in a single scope; it can create “related” temporary files, prevent conflicts if files are already open; it can re-indent the contents of files rather than buffers, and so on. I think assess is a nice addition to the testing capabilities of Emacs.

More is needed and some of this involve changes to Emacs core — packages which are noisy, enforce interactivity, and so forth. I would also like to add support for “robotized” tests running keyboard macros, tests for checking hooks, output to the message buffer, and testing for asynchronous call backs. But assess is quick, easy to use and makes testing of many features of Emacs much easier. I’ve started to use it in my own packages, and will eventually use it in all of them.

My plans for the future are to move assess to ELPA, and then eventually to Emacs core after 25.1, probably as ert-assess. I hope that along with my restructuring of the Emacs unit tests files, this will make testing of Emacs core simpler and more straight forward; if it does this should make the merge hassles (mostly to other people, sadly) caused by file moves worthwhile. Emacs will be easier to develop, and simpler to change.

Feedback is, as always, welcome.

## Bibliography

### lentic 0.9

Lentic is a package which implements lenticular text — two Emacs buffers that contain the same content, but are otherwise independent. Unlike indirect-buffers, which must contain absolutely identical strings, lentic buffers can contain different text, with a transformation between the two.

It was not my original plan to have another release so soon after the last release . However, the work that I had planned for that release turned out to be very-straightforward.

For this release, introduces a new form of buffer which is an unmatched block buffer. The details do not matter — the practical upshot is that with, for example, org-mode it is now possible to have more than one style of source block. In my examples directory, I have an org-mode file with “hello world” in three different languages (Clojure, Python and Emacs-Lisp). When lentic, you get four views, each in a different mode, and syntactically correct. Not a use I think I would suggest, but a nice demonstration.

Lentic is now available on MELPA, MELPA stable and github.

## Bibliography

### lentic 0.8

Lentic is a package which implements lenticular text — two Emacs buffers that contain the same content, but are otherwise independent. Unlike indirect-buffers, which must contain absolutely identical strings, lentic buffers can contain different text, with a transformation between the two.

This has several uses. Firstly, it allows a form of multi-modal editing, where each lentic buffer shows the text in a different mode. For example, this can be used to edit literate Haskell code. This should work with indirect-buffers, but in practice does not because the buffers share text-properties. These are a feature of the buffer strings in Emacs, and are used by some modes for their functionality; when two modes work on the same string, each tends to reset the text properties of the other.

It is possible to take this form of multi-modal editing further, where the different buffers contain different syntax. So, for example, one buffer might be in fully valid Emacs-Lisp, while the other might be a fully-valid org-mode buffer. This allows a literate programming technique even without specific support for this form of programming in the language. Taken to the extreme, it is even possible for the buffers to contain completely different strings; I have not found a good practical use for the extreme yet, but lentic now supports a rot-13 transformation which demonstrates its capabilities.

Lentic can also be used to create persistant views of the same text. For example, lentic could be used to maintain a view of the imports of a java file, or the namespace form in clojure, or the preamble of a latex document. Unlike a second window, this view persists even if it is not visible. Alternatively, one view could use very small text, and the other could contain larger text, allowing rapid navigation.

Lentic 0.8 contains a number of new features since the 0.7 [release] . The biggest change is that it is possible to produce any number of lentic buffers, rather than just two as previously. This means that its multi-modal and persistant view capabilities can be used at the same time.

Lentic is now available on MELPA, MELPA stable and github.

## Bibliography

### m-buffer 0.11

m-buffer is a package which provides a high-level convienience API for interacting, searching and changing the contents of an Emacs buffer. It abstracts away from Emacs’ use of global state, and provides list-orientated operations removing the need for explicit looping.

Hence, we can replace:

 (while (re-search-forward "foo" nil t) (replace-match "bar"))

with:

 (m-buffer-replace-match (m-buffer-match (current-buffer) "foo") "bar")

m-buffer also protects global state, so the first form actually needs to be something closer to:

 (save-excursion (goto-char (point-min)) (save-match-data (while (re-search-forward "foo" nil t) (replace-match "bar"))))

There have been a few changes since the 0.10 release . The interface has been improved slightly (in a backward compatible way). In addition some error-checking has been added to pick up what I think was counter-intuitive behaviour. So, previously:

 (m-buffer-match-page (current-buffer) :regexp "this")

would return all matches to “this” rather than pages.

The biggest change, however, has been the full use of lenticular source documentation , which provides a richer explanation of the code base. At the moment, I think that this is very usable, although I am aware that it is less clean than, for example, the documentation for dash, but it still is a nice demonstration of how lenticular text can work.

## Bibliography

### Lentic Doc — self documenting source for emacs

Emacs, of course, has always described itself as self-documenting because it has an integrated documentation system. With my new creation, lenticular text, though the world changes slightly. I can now write rich documentation embedded into the code base of my package, and can do this without comprimising either the documentation or coding environment.

For the 0.7 release of lentic , I added support for generating this documentation automatically on the fly from the source code. This can be viewed either internally in Emacs with EWW, or externally with a browser. In the latter case, I have used org-info.js which gives an info-like and navigable experience to the manual.

As yet, this process is not perfect, and the output is not ideal, but it still seems to me to be very nice. It is certainly something that I wanted to use for m-buffer . So I’ve extended lentic so that its documentation system can work for any package. Much to my amazement, it’s already is use by others , which has to be a good sign after such a short time span.

The main issue in making lentic-doc generic is working out how to compile the completed org-mode files into HTML. I had already considered trying to the “main” file for a package (so, lentic.el for lentic or m-buffer.el for m-buffer). But, as I discovered with lentic this does not work that well; instead I use a single top level file which imports everything else. To enable this lentic-doc uses the following technique: first it looks for a variable called package-doc describing how to compile the package documentation; if that fails, lentic looks for a file called package-doc.org in which case that is used. The main entry points are two functions lentic-doc-external-view or lentic-doc-eww-view which show the documentation in the respective browser.

There has been a lot of discussion about documentation format on the Emacs-devel mailing list recently. One common complaint was that texinfo 5 is rather too slow, taking a considerable time to compile for Emacs. Now, at first sight it would appear that moving to a lentic/org mode solution would make this worse, as the combination of the two will be much slower than even texinfo. However, lentic and org have one advantage over texinfo — they are entirely implemented in Emacs lisp, so anyone using Emacs also has the ability to generate the documentation from source; this means that the documentation can be built up lazily, on-the-fly as it is requested.

This works straighforwardly for a single package at a time — on request Emacs generates the documentation for lentic or m-buffer and displays. This still leaves the problem of cross-links between the different packages, of course. If I want to hyperlink from lentic to m-buffer, then I need to ensure that the m-buffer documentation is generated. Not so easy if the documentation is being viewed inside a browser.

Fortunately, Emacs provides a solution. Actually, two, as there are now two packages implementing a webserver for Emacs. By using an Emacs web server, I can handle any requests for package documentation, generating them on-the-fly if necessary. It turns out to be relatively simple too, as you can see here: https://github.com/phillord/lentic-server.

Of course, lentic-server is just a work in progress and far from complete, but it is pleasing none the less. The use of lentic has allowed me to tie together Org and Emacs-lisp, and to make rich documentation environment from technology that already exists.

The versions of lentic and lentic-server described here are available on MELPA .