When you make a standalone application intended as a command line tool you may want integration tests to exercise the application through the normal command line user interface.

There are tools that can do this testing on OS executable applications but sometimes it is useful to run these tests as part of the test suite without having to first build a standalone binary.

The goal then is to test a function that only interacts via standard input and output. This code demonstrates how it can be done:

(defun application ()
  (let* (name greet)
    ;; Prompt 1
    (format *query-io* "Name: ")
    (setf name (read-line *query-io*))
    ;; Prompt 2
    (format *query-io* "Greeting: ")
    (setf greet (read-line *query-io*))
    ;; Result
    (format t "~&~A ~A!~%" greet name)))

(def-test greet ()
  (let* (;*STANDARD-INPUT* must contain all the responses at once.
         (*STANDARD-INPUT*
          (make-string-input-stream (format nil "John~%Hi~%")))
         (*STANDARD-OUTPUT* (make-string-output-stream))
         (*QUERY-IO*
          (make-two-way-stream *STANDARD-INPUT* *STANDARD-OUTPUT*))
         result)
    (application)
    (setf result (get-output-stream-string *STANDARD-OUTPUT*))
    ;; RESULT =>
    ;; "Name: Greeting: 
    ;; Hi John!
    ;; "
    (is (string-equal
         "Hi John!"
         (nth 1 (split-sequence:split-sequence
                 #\newline result))))))

The example’s method only works for functions that have a fixed request/response sequence, in other words, the test does not need to change its response dynamically, all the responses can be prepared before the function is executed.

The reason for this limitation is that make-string-input-stream creates streams with fixed content. Even if the original string changes, the stream content does not. Without the ability to change the stream content after creation there is no way to implement dynamic responses to application prompts.

While it does not seem possible to create interactive output-to-input streams with Common Lisp primitives, UIOP:RUN-PROGRAM can provide such output-to-input streams to external processes. This at least indicates that it can be done for some particular case. On CCL this is accomplished with file descriptor streams. I have not investigated the detail of the implementation nor have I looked how it is done for other implementations, so it may ultimately be a dead end.

It appears as if there is no easy and uncomplicated solution for setting up a *STANDARD-OUTPUT* to *STANDARD-INPUT* pipe that will enable test functions to apply logic in-between responses. If you need such a solution some avenues to investigate are:

  • Test the application as a standalone external process with UIOP:RUN-PROGRAM. This definitely works but it is exactly what we did not want to do.
  • Look for a library that implements some kind of pipe function. cl-plumbing may be an option.
  • Try to implement piping with files or sockets.