Friday, January 23, 2015

Writing a low level graph database

I've been interested in graph databases for a long time, and I've developed several applications that offer an API close enough to a graph API, but with relational storage. I've also played with off the shelf graph databases, but I thought it would be fun to try an implement my own, in Haskell of course.
I found that in general literature is quite succinct on how database products manage their physical storage, so I've used some of the ideas behind the Neo4J database, as explained in the Graph Databases books and in a few slideshows online.
So I've written the start of a very low level graph database, writing directly on disk via Handles and some Binary instances. I try to use fixed length record so that their IDs translate easily into offsets in the file. Mostly everything ends up looking like linked lists on disk: vertices have a pointer to their first property and their first edge, and in turn these have pointers to the next property or edge. Vertex have pointers to edges linking to and from them.
I've also had some fun trying to implement an index trie on disk.
All in all, it's quite fun, even though I realize my implementations are quite naive, and I just hope that the OS disk caching is enough to make performance acceptable. I've written a small benchmark using the Hackage graph of packages as sample data, but I would need to write the same with a relational backend.

If anybody is interested in looking at the code or even participate, everything is of course on Github!

Monday, January 12, 2015

EclipseFP 2.6.3 released

A new version of EclipseFP, the Eclipse plugins for Haskell development, has been released. The release notes can be found here.

As usual install or update from Eclipse by using the update site http://eclipsefp.sf.net/updates.

Happy Haskell Hacking!

Wednesday, December 10, 2014

Using Ansi console for nicer console output in EclipseFP

A little tip from the trenches (-:. I use the Ansi Console plugin in EclipseFP to be able to see a nice color output from my Haskell executable runs.

For example, the Tasty example:

Works also for Yesod applications, etc.

Note that some support for Tasty is being added in EclipseFP 2.6.3, to at least provide some templates for the cabal test-suite section and test modules. I'm not too sure if it's worth to develop a runner for it, since contrary to HTF we won't have a link to the source, so the standard output is probably nice enough.

Happy Haskell Hacking!

Wednesday, December 03, 2014

Circling through ideas

I can't make my mind up about what to do in my free time (I have some, yes yes). I keep circling between the same ideas, thinking "that would be good", then jumping to the next one.

  • Keep on working on EclipseFP and other Haskell IDE related ideas. I would like for example to unify the storage of metadata between buildwrapper, scion-browser and eclipsefp, to have one database to would keep library information (definitions, documentations) like scion-browser, AST with types like buildwrapper, usage references like the usage DB in EclipseFP. It could be interesting to try to use that database to drive an IDE and have a clear repository of metadata. But then I'm tired of working on my own on EclipseFP, and when I see that Leksah really has also one active maintainer, I think people are really not interested in advancing Haskell IDEs, they must be happy with Emacs/vi and ghc-mod at a push, so why bother?
  • Then I think working on games is fun! I had fun writing Mazes of Monad, and other little games, and I do enjoy playing role playing or adventure games (I can really recommend the last one I've completed, the Longest Journey), so maybe writing a game the reactive way or even for Android like the guys at Keera Studios do would be a good use of my time. Then I remember I'm a programmer that sucks at graphics design and would probably suck as much at game design.
  • Games are too trivial, let's do something that will change the world, like work on AI! A few books I've read like Kurzweil's and Hawkins' have been truly inspirational, so maybe I could write some HMM neural temporal gizmo that would become sentient over night!! Then I wake up. People smarter than me and with more time than me are already working on that, so I would not contribute anything anyway. Why don't I help these people by providing a better programming experience, for example a Haskell IDE? Back to idea 1!
Ah well, I'll go back to browsing Reddit!

Sunday, November 23, 2014

EclipseFP 2.6.2 released!

I've just released EclipseFP 2.6.2. This is mainly a bug fixing release with a better handling of cabal sandboxes and related functionality. It wraps also an important change in scion-browser, which should now mean that Hoogle uses all the packages present in your sandbox, so should give better results.

Browse the release notes!

Install by pointing your eclipse to http://eclipsefp.sf.net/updates.

This release is brought to you by me, myself and I. I would love to see more people contribute, even if only to install the development versions to provide some testing before the release, or write some documentation. There's work that can be done on the Haskell side, on the Java side, etc. Contact me if you're not sure, or send me pull requests! The code is at https://github.com/JPMoresmau/eclipsefphttps://github.com/JPMoresmau/BuildWrapper and https://github.com/JPMoresmau/scion-class-browser.

Happy Haskell Hacking!!

Friday, September 19, 2014

A new Haskell IDE!

Well, that title is click-bait. It's only a proof of concept, so far (-:, sorry!

I wanted to play with Threepenny-GUI since it allowed to do UI the FRP way, without having the troubles of getting a proper UI library to work. And I was going through a period of frustration with EclipseFP, so I thought about something else for a while... It's got the romantic name of HWIDE!

So in fact, this is a very simple integration of CodeMirror and Threepenny-GUI, to be able to edit Haskell sources inside your browser. When you save your buffer, the data is written to disk, and if a cabal file could be found for that source file, a cabal build (in a sandbox) is attempted (with a configure if needed). The output is then parsed by code ripped off BuildWrapper, and the errors/warnings are displayed, so you can click on them and see the offending line in the code.

That's all, really, and even that is not 100% perfect, but it's a start. I could get to play a bit with Events and Behaviors, develop some little widgets. If some FRP experts want to have a look at the code and offer some advice, I'd be all ears!

I probably won't have much time to turn this into the next generation Haskell IDE, but please fork and hack to your heart's content! The repository is at https://github.com/JPMoresmau/dbIDE.

Happy Haskell Hacking!

Thursday, September 18, 2014

Learning Scala via unit tests

Last month I followed a big data training session, and we used Scala to write algorithms for Spark. I had looked at Scala some years ago but I think at the time the Eclipse support wasn't very good (the pot calling the kettle black, eh?), and it piqued my curiosity again. So I started looking at tutorials, and found Scala Labs. The idea is interesting: you get a project with sources and tests, and you need to make all the tests pass. The source code for both the code to change and the unit tests is heavily documented, and guides you through the language nicely. And seeing the green ticks appear always triggers the proper reward areas of my brain (-:

I had a little issue getting the code to compile using Eclipse, since there's a sbt-launch-0.13.0.jar library at the root of the project, and Eclipse used that as the top library, and it had a weird version of the List class, that wasn't genericized! I removed that jar from the class path and everything worked fine.

I'm not aware of a similar tutorial for Haskell, but that would be a good idea!