This just in: Google will host an Android press gathering at its global HQ on Jan. 5 …. Presumably, the event will have something to do with the company’s Nexus One, an Android phone that Google plans to sell on its own Web site and perhaps through T-Mobile as well.

via Digital Daily

One of the distinguishing characteristics of Unix is the philosophy that “everything is a file” (taken even further in Linux and Plan 9). Reading the interview with Ken Thompson in Coders at Work (page 465) sheds some light on why this is the case:

Seibel: So you basically wrote an OS so you’d have a better environment to test your file system.

Thompson: Yes. Halfway through there that I realized it was a real time-sharing system. I was writing the shell to drive the file system. And then I was writing a couple other programs that drove the file system. And right about there I said, “All I need is an editor and I’ve got an operating system.”

So Unix started life as a glorified test harness for Ken’s file system! Amusing…

C functions may be used recursively; that is, a function may call itself either directly or indirectly. Uninquiring souls may take this as just another peculiarity of those C folk, of whose ways their neighbours speak little to outsiders but much among themselves.

By Kernighan, Ritchie, and Lovecraft

I’ve been working on adding paredit.el like structural editing to the next version of the La Clojure plugin for Intellij IDEA. IDEA already does most of the paren matching stuff (automatically inserting a closing paren when you type an opening paren and so on). So far I’ve got the basic barf and slurp commands working, and splicing, as you can see in the screenshot below:

The next step is probably to make IDEA’s expand selection code be a little smarter in the face of s-expressions.

In related news: I found a good introduction to paredit.el on SlideShare which may be of interest.

I’ll try to get the guys at IntelliJ to push out a new version of the plugin after the 9.0.1 release is out (it’s in beta now).

I’ve been following Tim Bray’s excellent Concur.next article series covering approaches to concurrency in various languages, and currently (no pun intended!) covering Clojure. The latest article talks about a super efficient log parsing implementation done by Alex Osborne an includes the following comment:

“Lots of the performance wins come from dipping into Java-land (AtomicLongs, LinkedBlockingQueue), which is perfectly OK, but a Clojure purist would probably see those occasions as maybe highlighting gaps in that language’s coverage.”

I’d take issue with that, one of the real strengths of Clojure it that it has easy and fast access to the whole Java ecosystem. As Rich says:

“Clojure is designed to be a hosted language, sharing the JVM type system, GC, threads etc. It compiles all functions to JVM bytecode. Clojure is a great Java library consumer, offering the dot-target-member notation for calls to Java.”

That seems pretty clear to me. I wonder if the people claiming to be Clojure purists are all coming from a Lisp background rather than the Java world?

I’ve been loking at the early-access version of Manning’s forthcoming Clojure in Action book as well as some of the criticism of it. One of the complaints is that the current drafts describe macros as a run-time concept and that this is wrong. The confusion arises from the fact that Clojure (and Lisps in general) don’t follow the same path from source to execution as a more conventional programming languages like C and Java. I’ll compare four different languages to see how they differ: C, Java, a traditional Lisp compiler, and Clojure.

C Like Languages

This is what most people are used to, the traditional compiled language. Here, source code is read in and then parsed (this is normally broken out into multiple stages, e.g. a separate lexing stage, but for our purposes we can gloss over these details). The parser emits some form of intermediate representation, usually an abstract syntax tree (AST), this is then used by the compiler to generate executable code.

Again, this potentially glosses over some details: optimizers can world on the intermediate representation for example, or the compiler could require a separate linking stage to generate an executable. For our purposes though this is sufficient: we go from source to AST to executable.

Also, it could be argued that the C pre-processor operates on the source code before the parser sees it, but in practice the C macros system is so primitive it doesn’t really warrant being called out as a separate stage.

Java Like Languages

The Java like languages differ from C in that they run on top of a virtual machine rather than being executed directly by the OS; as a reult their compiler emits ‘bytecode’ rather than a finished executable. This bytecode is then executed by the virtual machine. In all modern desktop and server virtual machines this is means just in time compiling the bytecode down to native machine code.

Traditional Lisps

The Lisp view of things is a little different; it’s more complicated but also more powerful. The first thing to note is that Lisp code is already basically in the form of an AST - there is no (or not much) syntax getting in the way. Next, there are 2 types of macros which are applied to Lisp code: macros and reader macros. I’ll duscuss them in the opposite order to the way they are applied…

The standard type of Lisp macros are what most people rave about when extolling the virtues of the language: these are chunks of code that are executed after the source has been loaded into an AST (remember, Lisp source code is basically in this form already, so this just involves moving from a textual representation to something that the Lisp runtime can work with). If a node in the AST is a macro then it is evaluated as the code is loaded and the result of the evaluation is used to replace the macro node in the AST. Stop and think about that for a minute - this happens before the code is evaluated by the regular Lisp runtime, but yet at this point you already have access to the full Lisp programming language. All of this means that you can do some pretty cool tricks: how about writing your own control constructs? Writing a DSL compiler? Logging constructs that have zero runtime overhead when not used (but that can be switched on and off by users of the program, unlike #define DEBUG 0 in C)?

The second, and much less common, type of macro is the reader macro. Reader macros operate on the character stream as it is read in, before the AST is constructed. Basically, when the reader sees a specific character (usually #) it then looks at the next character and uses that as a key into a table of functions (the read table) that tell it how to handle future input. Using reader macros it’s possible to create DSLs that don’t use s-expression syntax (s-expressions are the paren enclosed lists that Lisp is (in)famous for); or do something as simple as allowing the use of brackets to write quoted lists without needing an explicit quote (i.e. writing [1 2 3] instead of '(1 2 3)).

Only once all of this has finished is a traditional lisp ready to let it’s compiler go to work turning the (now macro-free) AST into executable code.

Clojure

Clojure is very similar to the traditional Lisp model, with two main differences. The first difference is the fact that, like Java, the output is bytecode which is then loaded and executed by a standard Java virtual machine. The second difference is that while Clojure does have reader macros, the read table isn’t exposed to user programs; that is, while it operates in the same way as a traditional Lisp there is no way for user code to alter the behaviour of the reader. This is probably good thing as Clojure includes a relatively large number of predefined reader macros including a literal syntax for lists, sets, and maps, as well as lambda-expressions (anonymous functions) and metadata.

Summary

C and Java like languages have a huge amount of syntax baked in, but don’r provide any way to modify this or to manipulate the program before it is compiled. Lisp has almost no syntax but provides a mechanism for users to add their own, and provides a mechanism to manipulate programs before they are compiled. Clojure has some syntax (more than other Lisps, but way less than C/Java/&c.) and provides the same mechanism for program manipulation as other Lisps.

The OmniGraffle file for the images in this post is available here if anybody is interested.

Update: this post is intended to compare traditional C-style languages with Lisps, it doesn’t cover, for examples, so-called scripting languages such as Perl, Python, and Ruby.

Based on Alex’s comment on the Leiningen group, here’s my shiny new setup for working with Git & GitHub on OS X, I’m posting it here then it’ll be easy for me to refer back to in future…

Installing the Tools

1. install MacPorts if you don’t already have it;
2. get an up-to-date version of ruby and rubygems with sudo port install rb-rubygems;
3. install the github tool with sudo port install json github; and
4. upload your public key to your GitHub account.

Creating a New Project

Run github create project_name to create a public project, append --private to create a private project (you will need a paid account at GitHub for this).

Forking

Run github fork project_owner project_name, for example to create a fork of the Cucumber testing framework run github fork aslakhellesoy cucumber.

Making Changes

Make your changes in a branch, this took me a little while to get used to at first as it’s different from the normal way of working with version control systems like CVS.

$ cd project_dir
$ git checkout -b feature_name

Make your changes, then run git commit -a -m "Comment." to commit them. From time to time push the branch to github so others can see what I’m working on git push origin feature_name.

If the work is in a fork then you’ll also want to send a pull-request back to the original project with github pull-request project_owner feature_name.

All from the command line, neat huh? For more tricks, you can also browse issues with github issues {open|closed}.

A really nice set of visualisations of the old Choose Your Own Adventure books, I was a big fan of the English equivalents when I was a kid.

So there’s this blog post over on the Best in Class blog that talks about ceremony in programming languages and compares Clojure with Java on this basis. While I’d agree with the basic premise of the article (that there is less ceremony in Clojure), I’m less keen on the way it’s presented: by way of a needlessly verbose strawman example. To be fair the article does kind of admit that this is what is being done, but it’s still annoying.

With this in mind let’s see how well we can do with the Java version of the code, relying on a better coding style and a couple of freely available libraries (one of the platforms much touted strengths). For the original — 28 line — version of the code I’ll refer you to the original post (but warn you that it’s presented in that well known code storage format, PNG!).

The same code rewritten in a smarter manner, but still using only the core Java libraries. This gets it down to 10 lines of code and also makes the intent of the code clearer. There’s still a fair amount of ceremony about this however: the multiple imports, and all of the class and static main method boilerplate.

import java.util.Arrays;
import java.util.Set;
import java.util.HashSet;
 
class Distinct {
  public static void main(String... args) {
    Set<String> distinct = new HashSet<String>(Arrays.asList(new String[] {
        "foo", "bar", "baz", "foo"
    }));
  }
}

Let’s see if we can’t do a little better with the addition of some open source libraries. Enter Google Collections, a really neat library that improves the collections API from the JDK. We’re now down to 7 lines of code, and 2 of those are just closing braces! In any reasonably size program the class and main statements disappear into the noise, so we’re really saying that we have 2 import statements and a single line of code. That’s not too different from the Clojure version all things considered.

import java.util.Set;
import static com.google.common.collect.Sets.newHashSet;
 
class Distinct {
  public static void main(String... args) {
    Set<String> distinct = newHashSet("foo", "bar", "baz", "foo");
  }
}

It’s interesting to note that the second Java version weighs in at 10 lines of code, versus 8 for the equivalent clojure version; not much of a difference really.

I think that the benefits of Clojure come from it’s functional style, macro system, and excellent concurrency support — not from the fact that you can save a few lines of code here and there.

Peter Kriens of OSGi fame has posted some comments about the current EDR from JSR-294, the proposed Java language changes in support of module systems:

In Java 1..6 the language offered a pretty pure model that was mapped
to reality in the VM. With class loader tricks we could tweak the
perspective each JAR had of this pure world, solving many real world
problems. In JSR 294, we will for the first time introduce this messy
and complex runtime world in the language. Untold millions have been
spent to make Java run on hundreds of platforms, and with one simple
JSR we bring back the need for #ifdef …

Read the relevant posts on the mailing list, especially this one.

I generally agree with the OSGi camp here, this is a giant case of ‘not invented here’ syndrome from the Sun people. It’ll be interesting to see if the acquisition by Oracle has any effect on this (or the JCP in general) but I guess we’ll only find out about that after the deal goes through (i.e. months away yet).

Hat tip to Chris Aniszczyk.