Are (let) and (lambda) equivalent in Common Lisp

Question

I'm reading On Lisp by Paul Graham, trying to better understand the functional style of programming. In Chapter 3, he mentions that functional programs "work by returning values" rather than performing side effects. I don't quite understand the implications this statement makes on how to manipulate and represent intermediate results of a procedure. If a functional program returns a value, is capturing it using (let) equivalent to using a (lambda) function?

Here's an example showing the definition of a function, (group), using a lambda function to capture the intermediate result. (group) takes a list, src, and a number, n, and subdivides the list elements into k groups of size n.

(defun group (src n acc)
  (funcall
    #'(lambda (back)
       (cond 
         ((consp back)
          (group back n (cons (subseq src 0 n) acc)))
         (t
          (reverse (cons src acc)))))
    (nthcdr n src)))

The last line of the function grabs the back partition of the list by taking the (nthcdr n src). Then, this result is passed as an argument to a lambda function which decides how to process src conditioned on the argument. This is purely functional code, and has no side effects. On the other hand I could have defined (group) in the following way:

(defun group (src n acc)
   (let ((back (nthcdr n src)))
     (cond ((consp back)
            (group back n (cons (subseq src 0 n) acc)))
           (t
            (reverse (cons src acc))))))

where we use (let) to first bind the variable back to the (nthcdr n src). I'm not sure how functional this version of (group) is because (let) binds variables to values in an imperative-like way.

Answer 1

Well, a way to regard let is that it is a more syntactically convenient form of a particular usage of lambda.

To clarify some notation, in Common Lisp several forms are equivalent.

• (function (lambda (...) ...)) can be written as #'(lambda (...) ...) since #' is a reader macro.
• #'(lambda (...) ...) can then be written as (lambda (...) ...), since lambda is a macro whose expansion is (function (lambda (...) ...)).
• Finally (funcall (lambda (...) ...) ...), which is equivalent to (funcall (function (lambda (...) ...) ...) from above, can be written as ((lambda (...) ...) ...), as a special case of a compound form (see 3.1.2.1.2.4).

None of this is necessary in a Lisp-1, but in CL it is.

Below I am going to write ((lambda (...) ...) ...) rather than the clunky (funcall #'(lambda (...) ...) ...) that I think Graham probably uses.

So, now, the important point:

(let ((x ...) ...) ...) is entirely equivalent to ((lambda (x ...) ...) ...).

Although it is not implemented this way in CL (let is a special operator), you can think of let as if it were a macro defined in terms of lambda like this. In particular here is a definition for a macro called allow which is like let (you can't redefine let itself in CL of course, hence this name):

(defmacro allow (bindings &body decls/forms)
  `((lambda ,(mapcan (lambda (b)
                       (typecase b
                         (null '())     ;elide ()'s in binding list as special caee
                         (symbol (list b))
                         (cons
                          (unless (eql 2 (list-length b))
                            (error "mutant binding ~S" b))
                          (list (first b)))
                         (t
                          (error "what even is ~S" b))))
                     bindings)
      ,@decls/forms)
    ,@(mapcan (lambda (b)
                (typecase b
                  (null '())
                  (symbol (list 'nil))
                  (cons (list (second b)))
                  (t (error "what even is ~S" b))))
              bindings)))

And now, for instance, using a macroexpansion tracer:

> (allow ((x 1) (y 2)) (+ x y))
(allow ((x 1) (y 2)) (+ x y))
 -> ((lambda (x y) (+ x y)) 1 2)
3

As I said, in CL let isn't defined like this as it is a special operator, but it could be, and there could be corresponding definitions of let* and so on. Here is a definition of allow* which is let*:

(defmacro allow* (bindings &body decls/forms)
  (if (null (rest bindings))
      `(allow ,bindings ,@decls/forms)
    `(allow (,(first bindings))
       (allow* ,(rest bindings) ,@decls/forms))))

(And at this point I get to laugh at people who say 'macros with recursive expansions baaad, baaaad'.)

So from the perspective of the semantics of the language there is no difference at all: (let (...) ...) is entirely equivalent to ((lambda (...) ...) ...). In particular, since any program involving let can trivially be rewritten to one using only lambda, and lambda is a purely functional construct, then let is also a purely functional construct.

There are then two differences in practical terms.

Readability. This is the important one. Programs are not just ways of instructing a machine to do something: they are a way of communicating your intent to other human beings. For almost everyone (and I think, probably actually for everyone) it is easier to read code which says, in English

let x be ..., and now ... things involving x ...

Rather than something which is extremely hard to even write in natural language, but might be

x will have a value in here, and now ... things involving x ..., and the value is ...

And that's even worse when comparing

let x be ... and y be ..., and now ...

with the really awful

x and y will have values in here, and now ..., things involving x and y ..., and the values are ... and ...

That's just an awful way of writing something: the values are widely-separated from the variables being bound, there is no indication which value belongs to which variable when you finally reach them, and finally there's a serial dependency which is generally harder for humans to parse.

If you came across natural-language text written like this you'd correct it, because it's awful. Well, that's what let does: it turns the hard-to-read lambda form to a much more understandable one, where the initial values are next to the variables.

Possible historical ease of implementation. It is possible, I think, that it was once easier for a compiler if let was treated as a special case rather than as simply a macro which expands into a lambda form. I am not sure about this, especially as I am fairly sure I have read somewhere I can't find just now that let originated as a macro, but it seems plausible as a reason that let is a special operator in CL. Certainly I find it hard to imagine that it was not possible for a compiler to see ((lambda (...) ...) ...) and compile that form in some optimal way (ie don't compile a function at all), even a very long time ago when compilers were made of mud and goose fat.

I think it is safe to ignore this second reason today.

Answer 2

Many points to answer your questions :

First,

In computer science, an operation, function or expression is said to have a side effect if it modifies some state variable value(s) outside its local environment -- Wikipedia

So having local variables to process inputs doesn't mean you're not doing functional programming. What's important is the deterministic nature of functions you develop: for any set of inputs, functions should always return the same output corresponding to the inputs.

When you want to remove any use of local state, you're trying to do purely functional programming :

Purely functional programming, a subset of functional programming which treats all functions as deterministic mathematical functions, or pure functions. When a pure function is called with some given arguments, it will always return the same result, and cannot be affected by any mutable state or other side effects. -- Wikipedia

What's important here, is to know why you'd want to do functional programming :

Proponents of purely functional programming claim that by restricting side effects, programs can have fewer bugs, be easier to debug and test, and be more suited to formal verification. -- Wikipedia.

I would also add important things like lazy evaluation and graph reduction (which should lead to automatic optimizations) :

Lazy evaluation does not evaluate function arguments unless their values are required to evaluate the function call itself.

The usual implementation strategy for lazy evaluation in functional languages is graph reduction. Lazy evaluation is used by default in several pure functional languages, including Miranda, Clean, and Haskell. -- Wikipedia

Common Lisp can do lazy evaluation but using a framework like CLAZY.

With functional programming, concurrency is easy because this paradigm avoids mutable state.

So to come back to your case, both group implementation use a local variables (the argument to your lambda is a local variable when you funcall it). Both use tail-recursion which is a good thing and remove a dependance to the capacity of the call stack.

The problem is that the local variable back in both implementations make an implicit dependance of your code to the limits of the system they run into, because back could be quite large.

If you know the maximum size of your input in advance and that it can be stored entirely in memory, then no problem to use those implementations. But if your input is some exabytes of data, you'd better use streams and generators and reduce the local state to an integer (the number of elements in the current group you're building) instead of storing entire sequences and subsequences. That way you can distribute the load and have concurrency, thanks to Functional Programming.

Answer 3

It is a lot personal taste, I would say. Since let itself is mostly anyway implemented by lambda. lambda arguments are local variables just as let is one.

(let ((a 1))
  a)

;; is equivalent to:
((lambda (a) 
   a)
 1)
 

I think it becomes rather interesting, if you use let*, since now you can sequentially use the stored and renamed intermediate values. Nesting lambdas in contrast would be very bulky and difficult to understand. Because as you see - the input for the lambda comes at the very end of the lambda call ...

I like to use let especially, when mutating a variable or variables using setf over a procedure or loop.

(let ((result '()))
  (loop for i in '(1 3 7 11)
        do (setf result (cons i result)))
  result)
;;=> (11 7 3 1)

Of course, this case is a very silly example (and could be expressed much more succinct without using let) but the procedure which is used for setfing could be much more complicated.

This silly example would be expressed as lambda - tail-call recursively (I will use defun to name the lambda):

(defun myfun (lst &optional (acc '()))
  (cond ((null lst) acc)
        (t (myfun (cdr lst) (cons (car lst) acc)))))

(myfun '(1 3 7 11))
;;=> (11 7 3 1)

I think let is less verbose and is useful especially when one has several of such variables which will be mutated.

Probably when the variable which holds the results should be prominent, a let expression is better, while if the procedure should be more emphasized - and be reusable - then the lambda or defun should be used - which gives the procedure a name.

But of course you could wrap a lambda/defun around a let expression to give the procedure a name ...