Bookmark Topic Watch Topic
 
friki data migration
Ranch Hand
Posts: 772
1
  • Mark post as helpful
  • send pies
  • Report post to moderator
Avoid heavy processing in Java memory

A lot of Java developers load SQL data into memory, transform the data into some appropriate collection type, execute nasty maths on that collection with verbose loop structures (at least, before Java 8's Collection improvements).

But some SQL databases support advanced (and SQL standard!) OLAP features that tend to perform a lot better and are much easier to write. A (non-standard) example is Oracle's awesome MODEL clause. Just let the database do the processing and fetch only the results into Java memory. Because after all some very smart guys have optimised these expensive products. So in fact, by moving OLAP to the database, you gain two things:


  • Simplicity. It's probably easier to write correctly in SQL than in Java
  • Performance. The database will probably be faster than your algorithm. And more importantly, you don't have to transmit millions of records over the wire.


  • The Cure:

    Every time you implement a data-centric algorithm in Java, ask yourself: Is there a way to let the database perform that work for me?

    Source


    SqlBestPractices
     
      Bookmark Topic Watch Topic
    • New Topic