Blog

I’ve mentioned this project a couple times before…

Last year I spent some time exploring GHC.Generics as a language for describing Protocol Buffers messages. Steve and I pushed just hard enough to get a real implementation out the door and it’s finally available on Hackage.

Quite a few people have been asking about a follow up to an old post, specifically on dealing with passing return values on the stack from our C/LLVM based primop. I ended up with some unexpected free time today on the Caltrain waiting for a crushed tractor-trailer to get cleared off the tracks and dusted off a semi-functional feed parser in the style of Tsuru’s code sample. Sample data for is much easier to find than NASDAQ’s ITCH feed, I hope it’s also easier for you to get started with.

While I’m working on more radical ideas about Hackage and Cabal, I thought I’d stick out a couple more palatable ones now.

A relatively common issue we have dealing with pubsub services on the internet is reliability. Conceptually it should be simple: connect, subscribe and then data flows magically for the rest of time. Reality comes well short of our desires. We use a very large pubsub service to consume 140 character messages… and sometimes it just decides to stop sending new messages after a few days. They’re not alone either. The timeout function in base almost does what we want but falls short in one key area: extending the timeout duration.

I’ve terribly hacked glued together a few microbenchmarks comparing the LLVM primop madness against a more traditional FFI and a native Haskell deserializer. The FFI “parser” used here is a bit contrived as it’s primarly measuring Storable and FFI overhead and does no parsing, but it’s interesting all the same. Send pull requests with a more full featured implementation and I’ll update this post accordingly.

We use Protocol Buffers extensively, and from talking to some folks at BayHac’12 it may be time to revisit the state of protobuf in Haskell.

From time to time, I have the need to parse relatively small, fixed width binary messages. Like a ITCH 4.1 MoldUDP64 packet from our buddies over there at NASDAQ. Parsing should also be reasonably quick. And I’m lazy. Writing parsers by hand Maintaining handwritten parsers is no fun.

Someone on #haskell last night was looking for a tool to convert a cabal-install package description to JSON. I hacked something together a while back to do exactly this for the cabal-waf project. While it’s not worthy of a standalone release on hackage it might be useful for someone else out there.

I just published a little Cabal tool for waf to GitHub. We’ve been using it for quite a while for our builds, including a CI buildbot. Managing a moderate number of packages with cabal-install was getting to be a hassle… and supporting multiple languages (primarily concerning C++) in one build system was a looming requirement. cabal-dev might be an alternative for pure Haskell shops, though I haven’t used it.

I often have the luxury of finding all sorts of underdocumented compile errors in GHC, and today was no exception. This wasn’t the first time I’ve hit this one in particular but it’s the first time I didn’t have a easy work around.