2012

While I’m working on more radical ideas about Hackage and Cabal, I thought I’d stick out a couple more palatable ones now.

A relatively common issue we have dealing with pubsub services on the internet is reliability. Conceptually it should be simple: connect, subscribe and then data flows magically for the rest of time. Reality comes well short of our desires. We use a very large pubsub service to consume 140 character messages… and sometimes it just decides to stop sending new messages after a few days. They’re not alone either. The timeout function in base almost does what we want but falls short in one key area: extending the timeout duration.

I’ve terribly hacked glued together a few microbenchmarks comparing the LLVM primop madness against a more traditional FFI and a native Haskell deserializer. The FFI “parser” used here is a bit contrived as it’s primarly measuring Storable and FFI overhead and does no parsing, but it’s interesting all the same. Send pull requests with a more full featured implementation and I’ll update this post accordingly.

We use Protocol Buffers extensively, and from talking to some folks at BayHac’12 it may be time to revisit the state of protobuf in Haskell.

From time to time, I have the need to parse relatively small, fixed width binary messages. Like a ITCH 4.1 MoldUDP64 packet from our buddies over there at NASDAQ. Parsing should also be reasonably quick. And I’m lazy. Writing parsers by hand Maintaining handwritten parsers is no fun.