12

Persistence in Java

Over the past years, I have gathered experience in the field of persistence abstraction in Java, using concepts such as EJB 2.0, Hibernate, JPA and home-grown ones. They seemed to me to have a steep learning curve and lots of complexity. In addition, as a big fan of SQL, I also thought that many abstraction models provide too much abstraction over SQL, creating concepts such as "criteria", "predicates", "restrictions" that are very good concepts, but not SQL.

The general idea of persistence abstraction in Java seems to be based on the Object-relational model, where RDBMS are somehow matched with the OO world. The ORM-debate has always been an emotional one, as there does not seem to exist a single solution that suits everyone - if such a solution can even exist.

jOOQ

My personal preference of how to avoid ORM-related problems is to stick to the relational world. Now the choice of data model paradigm should not be the topic of discussion as it is a personal preference, or a matter of what data model best suits a concrete problem. The discussion I would like to start is around my very own persistence tool called jOOQ. I designed jOOQ to provide most of the advantages modern persistence tools have:

  • A Domain Specific Language based on SQL
  • Source code generation mapping the underlying database schema to Java
  • Support for many RDBMS

Adding some features that few modern persistence tools have (correct me if I'm wrong):

  • Support for complex SQL - unions, nested selects, self-joins, aliasing, case clauses, arithmetic expressions
  • Support for non-standard SQL - stored procedures, UDT's, ENUMS, native functions, analytic functions

Please consider the documentation page for more details: http://www.jooq.org/learn.php. You will see that a very similar approach is implemented in Linq for C#, although Linq is not exclusively designed for SQL.

The Question

Now, having said that I'm a big fan of SQL, I wonder whether other developers will share my enthusiasm for jOOQ (or Linq). Is this kind of approach to persistence abstraction a viable one? What are the advantages / disadvantages you might see? How could I improve jOOQ, and what's missing in your opinion? Where did I go wrong, conceptually or practically?

Critical but constructive answers appreciated

I understand that the debate is an emotional one. There are many great tools out there that do similar things already. What I am interested in is critical but constructive feedback, based on your own experience or articles you may have read.

Improve this question
5
  • Does it handle transactions? Commented Dec 11, 2010 at 10:01
  • @Alison: No, it's just about SQL. Transaction handling is a very complex business that I think many other tools / frameworks (including EJB2/3) will handle better than jOOQ Commented Dec 11, 2010 at 10:27
  • I would strongly suggest a very solid and detailled use-case where you show your approach to be excellent for solving it. Commented Dec 12, 2010 at 19:26
  • Also note that editing THIS heavy will make answers more or less irrelevant and hence unusable for future readers. Ask a new, and better, question instead. Commented Dec 12, 2010 at 19:28
  • Hi Thorbjorn. You mean a use-case other than the ones shown on the examples page? Or would you like to see some examples in the question itself? About the edit: you're right in general. In this case, however, I thought the two answers I've got so far would have been somewhat the same... Commented Dec 12, 2010 at 21:42

4 Answers 4

Reset to default
5

I think you're on the right track and should continue with your solution. Some of the previous criticism is overly harsh, but some valid points are made.

I too have searched long and hard for a full-featured ORM solution that would fit my needs, but every solution I looked at came up short. Each has its pros and cons but none really did everything that I wanted. I agree with your criticism of these solutions - namely that they generally are complex and have a steep learning curve.

The top contender would have to be the de facto standard, Hibernate. It is full-featured, robust and well-tested. I respect it and appreciate it for what it is. Now, at the risk of offending half the programming community, I must also say that it is a bloated and complex piece of non-performant code. I've spent a fair amount of time poking around in its bowels, trying to debug and understand it, and I was not impressed with what I saw. Having said that, I would still recommend it as the starting point for anyone looking for a good ORM. It excels at simple CRUD, but I would not use it for performant bulk queries, such as data mining or number crunching.

Unfortunately, my application is more of the number crunching variety (though there is some CRUD too), so I decided to roll my own. I knew from the start that it would be sub par compared to the other solutions out there, but it would be good enough and give me the performance I needed. Now here's the really great part: it looks very much like your solution!

This is what I think you did most right: You stated your underlying beliefs as a kind of mission statement, and those beliefs are correct. I've obviously spent a fair amount of time thinking about this problem too, and it is a difficult one to solve. But as time goes on, I think the programming community comes to understand it better, and we will create better solutions. Kudos to all previous efforts (especially Hibernate), but I think we can do better.

Your solution better refines the problem, but it only solves part of it. I think that a layered approach may give us everything we want. What you have come up with/ IMO, is the foundation layer. As you have suggested, I think this layer is best automatically generated based on a database schema. It should be a very simple relational object model that maps directly onto the database and no more. You're right that data persists much longer than code, so at this level, data should drive code and not vice versa. It would support CRUD as well as performant bulk querying. I would probably implement some kind of L1 cache at this layer.

However, the thing that other ORM solutions excel at is the ability to define your objects in a manner that does not depend so much on the actual structure of the underlying database. For this functionality, I would create another layer. By necessity this layer becomes more abstract, hopefully simpler (at the expense of lost functionality), and builds on the previous layer. It might utilize an L2 cache.

I'm open to having more layers, but the key point for me is that by providing multiple entry points into the library, you can fulfill every need. For those that want a simple CRUD solution that maps directly to objects of their choosing, they can build on the top layer. For those looking for something more powerful and performant but willing to incur the added complexity, they can plug into the lower layer. I would not create a special query language, but instead would expose your query builder object for this purpose. And since the code is layered, that functionality naturally exists without special pass-through.

I think you do have an understanding of the space and aren't reinventing the wheel but rather have hit a slightly different niche. And frankly, this is a wheel that could use improvement anyway. You face an uphill battle contending against powerhouses of the past, but if your solution is good, and I think it's heading that way, then it will gain popularity on its own merits.

Improve this answer
1
  • Hi Dale, thank you for your encouragement! I like your phrasing about the layers and jOOQ being on the lowest layer with further abstraction possible on top of jOOQ. That is something I want to tackle when I have enough momentum with jOOQ. Interfacing with JPA and/or Hibernate will clearly be on the roadmap. As you put it, these tools excel in simplicity through abstraction and have many features that I don't want in my layer: transactions, sessions, L2 caches, etc. PS: Is your own solution in the public domain? I would be very interested! Commented Dec 24, 2010 at 8:55
8

"There are a lot of magic bullets out there, and no shortage of naive developers."

No offense, it appears that you don't entirely understand what's been done in this space already and therefore are reinventing some wheels - experience would tell us all that the wheels you invent, while neat and fun, aren't likely to be nearly as good or useful as the nicely refined wheels already available.

Improve this answer
3
  • 1
    Thanks for your feedback! Have you had a more in-depth look at my library? I precisely try to "abandon the O" how your article calls it. jOOQ wants developers to write SQL, not OR-Mapped code. I try to achieve what Linq does for .NET without being able to rewrite the Java compiler. Congratulations to Microsoft for Linq! And I dare say that I'm not naive, because I created jOOQ after I had not been satisfied with EJB 2.0 (quite some time ago), nor with Hibernate. Precisely for the reasons you point out in your article. I tried what's been done and it didn't suit my needs. Commented Dec 10, 2010 at 8:49
  • jOOQ wants developers to use your Criteria-like API. That's not SQL. It's a way to create SQL that in reality is uglier and more complicated than SQL with very little, if any, real benefit. "q.addCompareCondition(TAuthor.YEAR_OF_BIRTH, 1920, Comparator.GREATER);" I see nothing really different from any other class-per-table generation tool. Which means, as I said, there's an almost certain probability that a better one already exists. Template-generated string fields don't even come close to what lambda expressions enable. Commented Dec 10, 2010 at 17:14
  • 1
    It's a bit difficult for me to understand the motivation behind your answer/comment. I still think that you didn't have had a thorough look at the examples I provided (you quoted the very first example) and your comparison with competing products seems somewhat vague to me. The point of my question was to get constructive feedback, and I would appreciate just that. You mention "better tools" and "lambda expressions". Could you please elaborate? How does jOOQ compare to the tools you mentioned? How are lambda expressions implemented in Java 6 - before Oracle may add them in Java 8? Thanks again Commented Dec 10, 2010 at 23:26
2

One scenario that would make this kind of API well-suited is when you need a database-agnostic SQL builder which most of the ORM doesn't allow you to have. One big situation where I have needed it is in generating Reports. I have needed to build SQL in a object oriented way and going to run this sql on various databases for testing. Hibernate and other ORM are very dependent on configurations, thus limiting my sql construction such that I can't combine table where association doesn't exist in the xmls. Since any association is possible in the Report Module I am developing, I had looked for something different. Then I came across JOOQ. It just solves my problem. I have just needed a read only access, never encounter painful debugging issues like in hibernate.

I am actually planning to develop a different way of modelling data rather than the common POJO way and use JOOQ as one of the foundation which is a layer to build SQL. So on top of JOOQ, I am planning to create a more flexible way of modelling data/entities by using HashMaps. My design would make it possible to integrate front end and backend design easier in one entry point, though I would use various layers to complete the framework, just like what the above comments have said. JOOQ will really play a very good part like in my own project, it serves as one of the foundations or pillars.

So keep up the good work lukas!

Improve this answer
1

If I understand your tool correctly, it maps Objects directly to a conceptual SQL framework rather than mapping Objects that implement some mapping to SQL functionality (ORM), almost like a ORM-- abstraction for greater geeky control so that you can use more SQL features ORM's usually wouldn't give you?

Not sure what problem you're trying to solve. Since your tool requires in-depth knowledge of SQL, wouldn't people just use, well, SQL? Is it the glue-code you're trying to get rid of? Better validation of the SQL queries through API modelling instead of simple strings?

I have to admit that your JOOQ examples look like LISP versions of the SQL. Not that that is a bad thing :) and I do see some of the advanced stuff being interesting, but the advantages you list slowly disappear as you become increasingly advanced (more config, less abstract, more ingrained in the inner sanctum of the specific SQL engine, etc.)

One suggestion I'd like to see that I do miss in most ORMs is a framework that can deal with Objects in a non-relational way, translating Object interactions into whatever the backend might be (SQL, NoSQL, Topic Maps, RDF, etc.) which requires caching of the Model used rather than specifics of SQL interactions, dialects, and relational thinking.

IMHO, of course. :)

Improve this answer
1
  • Hi there, thanks for your feedback. You got it right. As qstarin put it, I try to remove the "O" from ORM and stay relational. Why not plain SQL? You mean JDBC? Have you ever queried your database with 5 nested selects and several joins, analytic functions with JDBC? Syntax errors, binding mismatch, etc. Unfortunately, jOOQ cannot be the right tool for you, because there is no possibility to map ANY model to objects. And I don't want that. jOOQ SHOULD be relational. For those developers that prefer that way of thinking. For the other one's I recommend JPA.. Or Linq if you're doing .NET Commented Dec 10, 2010 at 8:58

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.