The place where I put my thoughts on software development, so I can remove them from my mind.

18 February 2009

Persistency solution decision algorithm

Here's my current strategy for picking a persistency solution:

IF rdbms is already in use THEN
  IF problem is trivial THEN
     Spring JDBC
  a graph DB

Yes, Hibernate/JPA is missing. As far as I'm concerned, the object/relational mismatch is troublesome enough to never choose an ORM at all. Face the facts: an RDBMS works with rows of tuples, not graphs of objects, and it is tightly tied to its query language: SQL. The only correct way to map this is in the way JDBC does. Trying to fix the mismatch gives complex solutions, and for what reason? There are other, simpler, faster, mature options now.

I'm not decided yet if an OODB like db40 is better than a graph database like neo4j. As someone who likes his code to be executed as is, without any byte code mangling, the indirection of graph databases looks more attractive. OODB's insist on persisting your objects directly, and magically reconstructing them again. Graph DB's are somewhat like a whole bunch of persisted property files with links between them, and those property files need a very shallow wrapper.

Anyway, it's time we took RBDMSes off their pedestal!


Peter Neubauer said...

Haha, nice code snippet, can I qupte you on that!
And then, with an e.g. "Java Ontology" meta layer on top of Neo4j you can in principle build your own OODB, which is probably easier than building a graph database out of db4o.


Emil Eifrem said...

And you made today's high score list! :)


Anonymous said...

Actually, I would go with db4o. A real object graph is powerful enough to express any application, you don't really need the parallel node/relationship handling (and you can also go the other way round and turn db4o into Neo4j if you really like that way of working). The indexes and related algorithms to fetch/retrieve a real object tree should be as fast as the ones in Neo4j (this is a commodity actually, nobody has a magic wand when it comes to optimizing the fecth/retrieval of object trees).
And one more thing, you don't need to touch your byte code to work with db4o (that's only something that you do if you want to use a couple of the latest features).
The goal of db4o is transparency, that's why it works with your object domain rather than something else.


Emil Eifrem said...

OODBs turn a programming paradigm into a data model. While there's merit to that, I agree with the founders of the relational model that data and logic should be kept separated. (I just disagree that tables is the appropriate abstraction of data in a semi-structured and networked world.)

Now, clearly you can add a graph db API on top of an OODB like Db4o. And with a meta model and some reflection you can turn Neo4j into an OODB. But I think that the ORM experiments of the world show that once you've commited to a persistence paradigm for your app, you're probably best off to choose a clean implementation of that paradigm rather than patching another one.

I think it boils down to choosing the right tool for the right job -- as well as a fair amount of personal taste.

I have to disagree with you on one thing though: Once you scale up in data set size, there's nothing commoditized about how to rapidly traverse deep graphs of data. I know few who do that well in the 100M+ nodes scenario, even fewer in the 1B+ range and none in the 100B+ range.

Disclaimer: I'm part of the Neo4j team.


Marc said...

I used a (rdbms (mysql) + ORM + node graph) solution in my latest project, and it turned out to work great (this far atleast). It's all open source so check it out: Sprout

Anonymous said...
This comment has been removed by a blog administrator.
Anonymous said...
This comment has been removed by a blog administrator.
Danny said...

It's been a while since I posted this. I have to update my opinion on this: the long term goal is not to get rid of RDBMSes. We want to fix them so they scale well.

Latest saved links

About Me

My Photo
Nieuwegein, Netherlands


Syntax Highlighter