Update: not actually a new set of talks, these have been online for about 6 months now, but it’s the first time that I’ve seen them…

Prevalent Databases

The idea behind prevalent databases has been around for a while being, if not ‘popularised’ exactly, at least pushed by the guys behind Prevayler. Here’s how they describe them:

Prevayler is an open source object persistence library for Java. It is an implementation of the Prevalent System design pattern, in which business objects are kept live in memory and transactions are journaled for system recovery.

Fleet DB

While Mark McGranaghan’s Fleet DB doesn’t use the term prevalent database, but right now that’s basically what it is. The core of Fleet DB is a Clojure based append-only log based database; it provides a native clojure query language (with built in query optimiser), schema-less records, indexes, and a server with a JSON based network protocol.

For a new new project Fleet DB also has a good set of documentation and it sounds like Mark has some big plans for it in the future. As an added benefit there are also clients for the network protocol in languages other than Clojure (Ruby officially, and a set of Python bindings in development).

Persister

Sergey Didenko’s Simple Persistence for Clojure project is a much less ambitious offering, but with the really cool feature of being a single (255 line, ~11KB) file that you could just drop into your project and start using - that’s pretty lightweight! From the read me file:

Simple Persistence for Clojure is a journal-based persistence library for Clojure programs. It follows “Prevalent system” design pattern.

The intended usage is assist you in making a prevalent system. Thus you work with your in-memory data and wrap every writing call into one of (apply-transaction*) macros.

A nice feature is that the log files are just Clojure code:

Transactions are logged as a valid Clojure code, so they are easy to read and run separately.

Given a sequence the squeeze function returns another sequence with any adjacent items which match a supplied predicate merged together using a supplied function. It’s probably easier to illustrate by example, suppose I have a sequence of strings and I want to merge them together when the trailing string starts with whitespace, I can squeeze them like this:

(squeeze #(and %2 (re-matches #"\A\s.*" %2))
         #(apply str (apply concat %&))
         ["hello" " world." "foo" " bar"])

Another example, given a sequence of characters (read from an InputStream for example), I could group them into words by squeezing then thusly (the first line is just to remind you that calling seq on a string produces a sequence of characters):

user=> (seq "Cheers, chars!")
(\C \h \e \e \r \s \, \space \c \h \a \r \s \!)
 
user=> (map str/trim
         (squeeze #(and %2 (not= \space %2))
                  #(apply str (apply concat %&))
                   (seq "I'm sorry Dave, I can't let you do that.")))
("I'm" "sorry" "Dave," "I" "can't" "let" "you" "do" "that.")

So how does it work? Well, here’s the interface:

(defn squeeze
  [pred merge-fn coll]
  (squeeze- pred merge-fn coll nil))

And here’s the actual function that does the work, it’s declared private because I don’t want to expose the matched parameter to the outside world.

(defn- squeeze-
  ([pred merge-fn coll matched]
    (lazy-seq
      (when-let [s (seq coll)]
        (let [f (first s)
              s (second s)
              rest (rest coll)]
          (if (pred f s)
            (squeeze- pred merge-fn rest (cons f matched))
            (let [next (if matched (merge-fn (cons f (reverse matched))) f)]
              (cons next (squeeze- pred merge-fn rest nil)))))))))

I should probably point out that all of this playing around with sequences was inspired by Sean Devlin’s excellent proposal for some new sequence functions for Clojure 1.2.

The full code for this is available here (it’s just the above, but with an added doc comment on the squeeze function definition).

So, here are some simple examples of working with Java arrays in Clojure…

Given some sample data:

(def my-list '(1 2 3 4 5))
(def my-vector [1 2 3 4 5])
(def my-map {:a "apple" :b "banana" :c "chopped liver"})

To convert to Java arrays:

(to-array my-list)
#<Object[] [Ljava.lang.Object;@962522b>
(to-array my-vector)
#<Object[] [Ljava.lang.Object;@37e55794>
(to-array my-map)
#<Object[] [Ljava.lang.Object;@52cd19d>

Note that this always returns Object[] regardless of the contents of the collection. Note also that the map isn’t flattened (the pp function used here is in clojure.contrib.pprint):

user=> (pp)
[[:a "apple"], [:b "banana"], [:c "chopped liver"]]

If the array is 2-dimensional there is a corresponding function:

user=> (def my-vec-2d [[1 2 3] [4 5 6] [7 8 9]])
#'user/my-vec-2d
user=> (to-array-2d my-vec-2d)
#<Object[][] [[Ljava.lang.Object;@3a42f352>
user=> (pp)
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
nil

If you need to use a specific type of array (e.g. to pass a String[] into a Java method) or need to use an array with more than 3 dimensions it’s a little trickier:

user=> (into-array my-list)
#<Integer[] [Ljava.lang.Integer;@60c0c8b5>
user=> (pp)
[1, 2, 3, 4, 5]
nil
user=> (into-array my-vector)
#<Integer[] [Ljava.lang.Integer;@2151b0a5>
user=> (pp)
[1, 2, 3, 4, 5]
nil
user=> (into-array my-map)
#<MapEntry[] [Lclojure.lang.MapEntry;@7ae0c7c3>
user=> (into-array (vals my-map))
#<String[] [Ljava.lang.String;@731de9b>
user=> (pp)
["apple", "banana", "chopped liver"]
nil

There, that should serve as a handy reference for myself for when I’m feeling forgetful…

This just in: Google will host an Android press gathering at its global HQ on Jan. 5 …. Presumably, the event will have something to do with the company’s Nexus One, an Android phone that Google plans to sell on its own Web site and perhaps through T-Mobile as well.

via Digital Daily

Seibel: So you basically wrote an OS so you’d have a better environment to test your file system.

Thompson: Yes. Halfway through there that I realized it was a real time-sharing system. I was writing the shell to drive the file system. And then I was writing a couple other programs that drove the file system. And right about there I said, “All I need is an editor and I’ve got an operating system.”

So Unix started life as a glorified test harness for Ken’s file system! Amusing…

C functions may be used recursively; that is, a function may call itself either directly or indirectly. Uninquiring souls may take this as just another peculiarity of those C folk, of whose ways their neighbours speak little to outsiders but much among themselves.

By Kernighan, Ritchie, and Lovecraft

The next step is probably to make IDEA’s expand selection code be a little smarter in the face of s-expressions.

In related news: I found a good introduction to paredit.el on SlideShare which may be of interest.

I’ll try to get the guys at IntelliJ to push out a new version of the plugin after the 9.0.1 release is out (it’s in beta now).

“Lots of the performance wins come from dipping into Java-land (AtomicLongs, LinkedBlockingQueue), which is perfectly OK, but a Clojure purist would probably see those occasions as maybe highlighting gaps in that language’s coverage.”

I’d take issue with that, one of the real strengths of Clojure it that it has easy and fast access to the whole Java ecosystem. As Rich says:

“Clojure is designed to be a hosted language, sharing the JVM type system, GC, threads etc. It compiles all functions to JVM bytecode. Clojure is a great Java library consumer, offering the dot-target-member notation for calls to Java.”

That seems pretty clear to me. I wonder if the people claiming to be Clojure purists are all coming from a Lisp background rather than the Java world?