This week's book giveaway is in the Reactive Progamming forum.
We're giving away four copies of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams and have Adam Davis on-line!
See this thread for details.
Win a copy of Reactive Streams in Java: Concurrency with RxJava, Reactor, and Akka Streams this week in the Reactive Progamming forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Junilu Lacar
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Knute Snortum
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Ganesh Patekar

File operations with MS Excel

 
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,
Can anyone suggest a framework matched for the requirement,
Read data from MS Excel and insert into data base.
Read the data from MS Excel and with the data already stored in the DB and do some manipulations
Create out put in MS excel.
All the excel files will be approximately 1~2 GB size.
The process are scheduled jobs, where files will be read from a windows directory or FTP location.

Thinking of spring batch, any recommended frameworks which works well with MS excel reading.

Thanks in advance
Tharp.
 
Saloon Keeper
Posts: 5809
146
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Apache POI is the prime Java library for working with Excel sheets.

The DB part would be handled by using JDBC, possibly with an ORM layer (like JPA) on top of it.
 
Tharpu dhyan
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Tim.
Thanks for your reply.
Sorry to miss the serials in my query.
A tually it is for a scheduled batch program, which reads csv files with more than 1 million records,then need to perform aggregation and some calculations ( for data fore cast ) then write the file to Excel sheets.
Any inputs.

Regards
Feroz
 
Tim Moores
Saloon Keeper
Posts: 5809
146
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Nothing in that additional information changes the recommendation.
 
Tharpu dhyan
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,
Seems the subject line is misleading.
The support needed is to have any suitable framework or better APIs by which I can do some statistics operations.
For example, the program has to read CSV file of 1 million records and need to aggregate it with a similar file having another 1 million records.
Any suggestion like, need to choose between Java 8 streams, Jython, or any stream ORMs.

Two problems: 1) APIs for statistics operations to reduce the Java code.
2) Better performance while handling this much data in the memory.

Considering these factors , any recommendations please
 
Tim Moores
Saloon Keeper
Posts: 5809
146
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
For basic statistics, check out Apache Commons Math - it has a statistics module.
 
Wait for it ... wait .... wait .... NOW! Pafiffle! A perfect tiny ad!
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!