• Post Reply Bookmark Topic Watch Topic
  • New Topic

Reading a big file(3GB)  RSS feed

 
chandrap prakash
Greenhorn
Posts: 28
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

I have a csv file of size 3GB,i have to read this file and store the file into data base(SQL Server 2008) as stand alone application.

But the memory size of my machine is 2GB.

So,could anybody help me to how to read this file without exhausting the memory resources.

Thanks in advance.

Regards

Chandra Prakash
 
Ulf Dittmer
Rancher
Posts: 42970
73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Reading a file doesn't cause it to be held in memory all at once. With CSV files -which can be read sequentially, line by line- this shouldn't be a problem. Have you tried processing the file and encountered any problems?
 
Bauke Scholtz
Ranch Hand
Posts: 2458
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A self-respected DB ships with an import tool for CSV files. It is undoubtely more efficient than what you're going to try to implement in Java.

Just refer the DB specific documentation. I'll give only an example of MySQL to give you the idea: http://dev.mysql.com/doc/refman/5.1/en/load-data.html
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!