Chris Houser

Raghavan Muthu wrote:6. The wikipedia article also says that clojure works on "code as data" principle. The term is fascinating but I could get very little of it. can you just brief about this?

"Code is data" is indeed fascinating, with broad but often subtle implications.

Several languages provide syntax for some kinds of literal data -- often array and hash tables, like JavaScript. So this is textual syntax that maps quite simply to in-memory data structures:
{ name: "Chouser", kids_ages: [3, 6, 8] }
But they don't use this syntax for most of their code, even if parts of it look similar (such as curly braces):
if(should_print(a)) { print(a); }
Now undoubtedly inside the parser or compiler this is translated to an in-memory data structure, sometimes called an abstract syntax tree. But it's hard to guess exactly what shape it takes, and easy to imagine that it is clumsy to manipulate compared to the core array and hash-table data structures of the language.

So why not define a language's syntax to exactly match it's literal data
syntax? We could then perhaps rewrite the if statement above something like this:
{ if: {should_print: {var: "a"}}, then: {print: {var: "a"}} }
Now it's quite clear what data structure will be constructed for that syntax, and we can easily imagine code that generates or manipulates structures like that, because our programs already create and use data like that when it's representing data.

This is "code is data". Of course the syntax for lists is a bit different, but Clojure has syntax for lists, hash maps, vectors, and sets, and this syntax is used both for its data and for its code. As with most LISPs, code is primarily made of up lists:
(if (should-print a) (print a))
This is the key to macros being not just possible but convenient to write and use. It means it's easy to pass around not just objects and functions, but unevaluated code or code fragments. It means reading in, manipulating, and then saving code back out to files is fairly straightforward. It may not be immediately obvious how this helps, but I continue to be surprised by subtle ways my programs can benefit from the fact that code and data are described and manipulated exactly the same way.

Congrats on writing and running some real Clojure code of your own!

In the latest Clojure 1.3 pre-release snapshot, my error looks like this:
ClassCastException java.lang.Boolean cannot be cast to clojure.lang.IFn user/f (tmp.clj:6)
The name of the file (tmp.clj) and line number (6) causing the error are right there. From the message itself I can see I was trying to use a boolean value (true or false) as if it were an IFn, which is short for Function Interface. This almost always means there's an expression at the beginning of a set of parens that isn't actually a function. And of course that's the problem here -- the result of (zero? ...) is a boolean, and since that was wrapped in another set of parens, Clojure was trying to call the boolean, as if it were a function.

If that's not sufficient detail, you can use (pst) afterwards at the REPL and you'll see something like this:
ClassCastException java.lang.Boolean cannot be cast to clojure.lang.IFn user/f (tmp.clj:6) user/eval179 (repl-init.clj:4) clojure.lang.Compiler.eval (Compiler.java:6222) clojure.lang.Compiler.eval (Compiler.java:6189) clojure.core/eval (core.clj:2680) user/my-repl/fn--151 (repl-init.clj:25) clojure.main/repl/read-eval-print--5619 (main.clj:180) clojure.main/repl/fn--5624 (main.clj:201) clojure.main/repl (main.clj:201) user/my-repl (repl-init.clj:21) user/eval159 (repl-init.clj:39) clojure.lang.Compiler.eval (Compiler.java:6222)
This doesn't help much in our case, but what it's showing here is the full stack trace, from the function that cause the error at the top, all the way back to [tt]eval[\tt] function in the repl that started it all. Note that I'm running a slightly customized repl, so my stack trace may look a bit different from yours. Anyway, this kind of full stack trace is frequently useful in larger programs.

If you're using Clojure 1.2, things aren't quite so pretty:
java.lang.ClassCastException: java.lang.Boolean cannot be cast to clojure.lang.IFn (repl-init.clj:39)
Here it's pointing to a file and line number that is not helpful, so looking at the full stack trace may be worth while. To do that in Clojure 1.2, you must write (.printStackTrace *e), which presents you with this loveliness:
java.lang.ClassCastException: java.lang.Boolean cannot be cast to clojure.lang.IFn (repl-init.clj:39) at clojure.lang.Compiler.eval(Compiler.java:5440) at clojure.lang.Compiler.eval(Compiler.java:5391) at clojure.core$eval.invoke(core.clj:2382) at user$my_repl$fn__291.invoke(repl-init.clj:25) at clojure.main$repl$read_eval_print__5624.invoke(main.clj:183) at clojure.main$repl$fn__5629.invoke(main.clj:204) at clojure.main$repl.doInvoke(main.clj:204) at clojure.lang.RestFn.invoke(RestFn.java:578) at user$my_repl.invoke(repl-init.clj:21) at user$eval299.invoke(repl-init.clj:39) at clojure.lang.Compiler.eval(Compiler.java:5424) at clojure.lang.Compiler.load(Compiler.java:5857) at clojure.lang.Compiler.loadFile(Compiler.java:5820) at clojure.main$load_script.invoke(main.clj:221) at clojure.main$init_opt.invoke(main.clj:226) at clojure.main$initialize.invoke(main.clj:254) at clojure.main$repl_opt$fn__5650.invoke(main.clj:262) at clojure.main$repl$fn__5627.invoke(main.clj:193) at clojure.main$repl.doInvoke(main.clj:192) at clojure.lang.RestFn.invoke(RestFn.java:422) at clojure.main$repl_opt.invoke(main.clj:262) at clojure.main$main.doInvoke(main.clj:354) at clojure.lang.RestFn.invoke(RestFn.java:437) at clojure.lang.Var.invoke(Var.java:373) at clojure.lang.AFn.applyToHelper(AFn.java:169) at clojure.lang.Var.applyTo(Var.java:482) at clojure.main.main(main.java:37) Caused by: java.lang.ClassCastException: java.lang.Boolean cannot be cast to clojure.lang.IFn at user$f.invoke(tmp.clj:6) at user$eval305.invoke(repl-init.clj:2) at clojure.lang.Compiler.eval(Compiler.java:5424) ... 26 more
That's pretty daunting, but don't panic -- the details you want are in there. As we describe in the book, start with the root cause near the bottom of that listing, the last place it says "Caused by". Now, look at the following line and there's the information pointing you to exactly which line of your file was causing the problem -- tmp.clj line 6.

You can see some improvements have been made in this arena in recent versions of Clojure.

Michael Swierczek wrote:So you're saying this, macros, that, macros, and the other thing, macros, and also macros?

Exactly!

But note that's merely why I came to Clojure. As it turns out, I don't use them all that often, and there are other aspects of Clojure that are at least as compelling and perhaps more, depending on the task at hand. Like many language features, it's hard to see how they're useful until you find yourself in a situation that would truly benefit from them. So macros get their due coverage in Joy of Clojure, hopefully in a way that will help you understand both how they work and why they're useful.

If it works for Java, there's a very good chance it'll work for Clojure. Have you tried using settings.xml?

I think mutable data structures are less likely to need laziness internally in order to achieve their desired performance. On the other hand, laziness at it's core is just delayed execution, which is fairly simple to build in any language that supports closures. It would be possible, for example, to build a lazy seq like Clojure has in JavaScript which doesn't exactly encourage immutability at the language level.

Do you mean internal algorithms? Clojure's persistent sorted map is built on a persistent version of a red-black tree, which has many similarities to a mutable red-black tree, so in this case the internal algorithms are very similar. Clojure persistent hash map, however, has a very different structure internally compared to most mutable hash tables, and likewise very different internal algorithms.

On the other hand, the performance provided by Clojure's persistent collections are close to what you'd expect of their mutable cousins, so the algorithms and situations in which one would use Clojure's hash or sorted maps are likely very similar to where you'd use mutable versions in an imperative program.

Leiningen is a solid recommendation. For a more purely maven approach, see https://github.com/talios/clojure-maven-plugin

As John said above, Clojure is not an academic language. I feel confident saying this for a couple reasons. One is that it makes no claim that any one aspect of it is new or unique enough to be interesting "academically". Another is that in studying the decisions made in the design of Clojure itself, one can see that time after time when given the choice between something clever or unique on the one hand, and something useful or practical on the other, Clojure invariably goes with the practical solution.

For why Clojure might be worth learning and using, see my answer to question 2 over there.

When you read the word "MEAP", do you immediately think of the Road Runner from Looney Tunes?

I do. Every time.
I even hear in my head the "popping" sound that immediately follows.

Michael Fogus wrote:

I am curious what features drew you to Clojure over Scala.

I actively use both, so maybe Chouser can chime in regarding his choice. :-)

Initially, the dramatically simpler type system and lack of required type declarations attracted me, as well as macros. Later, macros. Then the seq abstraction kinda blew my mind, and also the macros.

3. Clojure's basic syntax for function and macro calls is the same as any LISP. In fact the way macros are defined and operate in Clojure is almost identical to Common LISP. Also, the dynamic nature of the development process is quite LISP-like.

4. The contradiction you see is in Clojure running on the JVM but not fully supporting OOP? Well, there are two points to make here. One is that Clojure considers some features of OOP to be harmful. For example, providing a concrete base class without a set of interfaces that fully describes it is now generally discouraged even in Java -- Clojure takes this a step further by not supporting it at all in its preferred polymorphic forms (multimethods and protocols).

The point other is that sometimes these less than desirable features are required for interop with some existing Java code, and so most of them are supported via gen-class and/or proxy. There is a bit of a performance cost to using both of these, in exchange for getting access to more of these undesirable OOP features as well as more dynamic behavior that Java alone provides.

Perhaps, though, I didn't understand your question. If I missed your point, please feel free to rephrase it.

5. Clojure embraces its host platform, adding power and flexibility to a virtual machine runtime that already exists. This allows Clojure to co-exist with other languages on the same platform. I think it's not at all uncommon for Clojure to be added existing Java code bases. The CLR port allows you to take advantage of Clojure's features on that platform as well, and to incorporate Clojure in existing C# or other .Net code bases.

I'll take a crack at your final question tomorrow.

My understanding is that InvokeDynamic won't help Clojure much if at all. Function calls and sufficiently hinted Java interop forms already compile to regular Java calls, so should already be faster than InvokeDynamic calls.

There is a separate proposal for tail call optimization which I believe is not currently planned for Java 7. Although TCO is common in LISPs and required by Scheme, Clojure isn't in desperate need of the feature as it has solutions for an overwhelming majority of situations where it might be required in other LISPs. The recur form, lazy seqs, trampoline and regular stack-consuming recursion together address nearly every practical recursion need. Still, it would be nice if the JVM were to fully solve the problem.

The exact mechanism used varies based on the kind of thing being reloaded.

Records and protocols (deftype, defrecord, and defprotocol) indeed do something like class reloading using a custom classloader provided by Clojure itself.

Most of the rest are built on top of Clojure Vars, which are a concurrency construct that allows for thread-safe mutation. Updating one of these amounts to making its Var point to the new definition.

A function data structure is one that doesn't support destructive updates, that is you can't change a value in the structure itself but instead create a new structure with your "change" applied to it. In order to do this efficiently, that is without copying the entire collection each time a "change" is made", a variety of techniques are used, including structural sharing (the new collection keeping a reference to part of the old collection) and laziness (not computing the exact details of some part of the structure until they're needed).

Persistent collections are the subset of functional collections whose performance does not degrade as you create newer and newer versions. That means it's safe and efficient for one part of your code to keep and use a collection while some other part starts with that collection and makes a series of "changes" to it, ending up with a very different new collection of it's own. Not all functional data structures can do this, but persistent ones can.

Functional languages generally provide functional versions of collection types you're already familiar with. Clojure provides persistent versions of maps (a.k.a. associative arrays) both sorted and hashed, as well as sets (sorted and hashed), vectors, lists, and queues.

It's not the case that some problems can only be solved with a functional language. These are all general purpose Turing-complete languages, after all. What's perhaps more interesting then is consider which problems are "better" solved in a functional language, for some measure of "better". Personally I find the biggest wins in code that deals with data -- functions that take data object, do computations, combinations, filtering, and in the end produce new or different data objects. This kind of code is often at the heart of a difficult problem -- the algorithmic core, or the main data-processing code. This is the kind of task where classic mutable objects are the most uncomfortable, where you have to worry about changing collections with iterators are open on them, where you'd like to use parallelism but are afraid to risk the potential deadlocks and race conditions. A functional approach to these tasks can be a real breath of fresh air.

Note that in other parts of a program where Clojure's functional toolset is perhaps less compelling, its macros often step in to still provide benefits compared to some other languages.

As for background, if you already know functional programming or a LISP, Clojure will be easier to pick up than if you don't. That said, I don't see why Clojure's not as good a place to start building such a background as any. :-)

Clojure and Scala have a lot in common. Both provide support functional-style programming and make some effort to address concurrency issues. Both provide compilation to .class files as well as interactive prompts (REPLs) for interactive development. Both enjoy support in various IDEs. Both compile to Java bytecode and run on the JVM, and both have had efforts to bring them to the .Net CLR. Both languages make it easy to use libraries provided by Java.

But of course there are also differences:

Scala's syntax is inspired by Java's, though I dare say it's got quite a bit that doesn't look that much like Java anymore. Clojure's syntax is inspired by LISP, but has its own extensions as well.

Clojure doesn't provide as much support for imperative-style programming as Scala, preferring to stay simpler and more firmly in the functional camp. In fact, simplicity and separation of concerns (not tangling different ideas into a single feature) is something you'll find throughout Clojure's design, a goal that I'm not sure Scala holds in such high regard.

Clojure never requires the types of arguments or locals to be specified in the code, falling back on runtime reflection when necessary. Scala on the other hand requires the type of everything to be known at compile time, though it does have some type inference so you don't have to write out the types of things quite as often as in Java. This is good because Scala's static type system is significantly more complex than Java's.

While I've never tried to use a Scala library from Java, my understanding is that it can be tricky and/or messy. However Clojure's gen-class and deftype macros make it easy to write very clean libraries for use in Java code that may have no idea the library it's using was written in Clojure.

I'm sure there are other significant differences, but I hope this gives you a feel for how the languages compare.

I made the jump from imperative to functional programming while solving Project Euler problems. They're fun and I continue to recommend them. They start start simple and get harder, while never really demanding you learn a lot of miscellaneous library functions (for networking, databases, GUIs, etc.) as you might for any "real world" project you might use as a learning vehicle. This keeps them relatively approachable.

Also, lots of other people have gone there before and once you've solved a particular problem you can check and see how others have solved it. This will help you learn even faster, giving you ideas to solve the next harder problem.