The Speed of eval in Some
Lisp Implementations

Kazimir Majorinc
November 17th, 2013.
The times of the evaluation of four simple and similar expressions give some impression about speed of eval in few popular Lisp. The first two expressions do not contain eval et all, the third one contains simple although non-trivial eval application, and the fourth contains one trivial eval application. Hence, the most important is the third of four numbers; others are for comparison. All tests are done on the same PC with Windows operating system, and Linux implementations running in Sun's VirtualBox virtual machine. Programs inside "virtual" OS-es run about 10% slower than in native OS-es.
Newlisp: 
(time (begin (setq i 1)
             (do-while (< 1000000) 
                       (setq i (+ i 1))
                       (begin (setq x 0)
                              (setq x (+ x 1))))))
(time (begin (setq i 1)
             (do-while (< i 1000000) 
                       (setq i (+ i 1))
                       '(begin (setq x 0)
                       (setq x (+ x 1))))))
(time (begin (setq i 1)
             (do-while (< i 1000000) 
                       (setq i (+ i 1))
                       (eval '(begin (setq x 0)
                              (setq x (+ x 1)))))))
(time (begin (setq i 1)
             (do-while (< i 1000000) 
                       (setq i (+ i 1))
                       (begin (setq x (eval 0))
                              (setq x (+ x 1))))))
Scheme: 
(define x 0)
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (begin (set! x 0)
                 (set! x (+ x 1)))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          '(begin (set! x 0)
                  (set! x (+ x 1)))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (eval '(begin (set! x 0)
                        (set! x (+ x 1))))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (begin (set! x (eval 0))
                 (set! x (+ x 1)))))
Common Lisp: 
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (progn (setf x 0)
                 (setf x (+ x 1)))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          '(progn (setf x 0)
                  (setf x (+ x 1)))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (eval '(progn (setf x 0)
                        (setf x (+ x 1))))))
(time (do ((i 0 (+ i 1)))
          ((= i 1000000))
          (progn (setf x (eval 0))
                 (setf x (+ x 1)))))
Clojure: 
(time (do (def i 1)
      (while (< i 1000000)
             (def i (+ i 1))
             (do (def x 0)
                 (def x (+ x 1))))))
(time (do (def i 1)
          (while (< i 1000000)
                 (def i (+ i 1))
                 '(do (def x 0)
                      (def x (+ x 1))))))
(time (do (def i 1)
          (while (< i 1000000)
                 (def i (+ i 1))
                 (eval '(do (def x 0)
                            (def x (+ x 1)))))))
(time (do (def i 1)
          (while (< i 1000000)
                 (def i (+ i 1))
                 (do (def x (eval 0))
                     (def x (+ x 1))))))

The results:


Implementation   time1   time2   time3   time4
PicoLisp 3.0.1, Linux 64 bit (VirtualBox) 0.163 0.070 0.174 0.174
Eligis Open Lisp 9.1.0 (32 bit) 0.139 0.070 0.212 0.166
Eligis Open Lisp 9.1.0 (64 bit) 0.195 0.078 0.231 0.227
PicoLisp 3.0.1, Linux 32 bit (VirtualBox) 0.251 0.135 0.269 0.250
Newlisp 10.0.0. 0.544 0.275 0.611 0.636
Newlisp 9.3.12. 0.578 0.265 0.656 0.656
GNU CL 2.6.7. 1.040 0.510 1.110 1.120
Lispworks CL 5.0 compiled 0.015 0.000 1.187 0.015
XLISP-PLUS 3.04 1.280 0.780 1.500 1.390
CLisp CL 2.41 1.672 1.047 1.906 1.859
Lispworks CL 5.0 without compiling 2.578 1.500 2.828 2.656
Allegro CL 8.1. compiled 0.016 0.015 3.703 0.094
Clozure CL 1.6-r14468 (WindowsX8664) 0.025 0.004 3.991 0.103
Clozure CL 1.5-r13651 (WindowsX8664) 0.025 0.004 4.189 0.106
Ufasoft CL 4.2.21.0 4.095 2.889 4.393 4.416
ABCL 0.20 interpreted 3.762 0.900 4.911 4.623
ABCL 0.18 compiled 0.069 0.039 5.117 1.388
Clozure CL 1.4-r13122 (WindowsX8664) 0.031 0.015 5.556 0.064
ABCL 0.18 interpreted 3.808 1.280 5.796 5.586
Petit Chez Scheme 7.4. 0.109 0.063 10.125 0.859
SBCL 1.0.34, Linux 32 bit (Sun Virtual Box) 0.011 0.004 11.962 0.390
Allegro CL 8.1 without compiling   11.875   7.922 13.906 12.704
Prime-Lisp 2.1 10.569 5.802 13.821 11.762
Gambit 4.2.6. Scheme interpreter 0.609 0.344 19.828 2.016
Chicken 4.6.0 (compiled, VirtualBox) 0.050 0.017 25.150 0.870
Petit Chez Scheme 8.0. (nonthreaded) 0.259 0.078 32.513 2.865
Chicken 4.6.0 (interpreted, VirtualBox) 0.960 0.620 35.150 2.050
Gauche Cygwin 0.9 Scheme 0.095 0.061 45.693 8.975
PLT Scheme 372 (debugging and profiling off) 0.140 0.063 50.406 6.953
Petit Chez Scheme 8.0. (threaded) 0.270 0.078 50.761 9.713
PLT Scheme 4.13 (debugging and profiling off)   0.047 0.004 77.127 19.609
Racket 5.1 (debugging and profiling off) 0.032 0.004 102.292 28.108
Racket 5.00 (debugging and profiling off) 0.016 0.000 104.396 30.358
SXM (CXEMA) Version 1.1 Dec 10 2001 0.780 0.470 165.844 45.781
Clojure 1.2. 2.946 0.873 181.235 11.764
Corman 3.0.1. (VirtualBox) 0.005 0.003 250.137   94.427
Clojure 1.3. 7.156 2.106 291.523 20.451
Clojure 1.1. 4.110 1.549   304.426 23.023