Win a copy of Cross-Platform Desktop Applications: Using Node, Electron, and NW.js this week in the JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

basic i/o question  RSS feed

 
Anurag Saxena
Greenhorn
Posts: 13
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I need an advice for the following situation:
I have a file (big file 74 mb) where each record has fixed length (277 bytes per record). I want to read specific place in each record (15th -25th byte of a record of 277 bytes). After reading those bytes I want to do some processing and replace those bytes with some other information.
Here are my concern: (1)File to too big, (2)I need to create another file whose records(length 277 each) are same as previous one except 15-25th byte in each record has now different value.
What is the best way of implementing it? I am using version 3.1.1
Thanks for any kind of help.
Anurag Saxena
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The large file size will take some time to process, but don't optimize just yet. Start by making the code understandable for humans. Here's an interesting article on how methods get so small that talks about code clarity, test first and refactoring. (It's in C# but easy to follow) The author, Ron Jeffries, takes an eXtreme position that leaves some people sputtering in frustration, but it should be an eye-opener and you might just love the style.
 
Stan James
(instanceof Sidekick)
Ranch Hand
Posts: 8791
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Oh, I started out last time to suggest a simple algorithm and then forgot to.
  • Read a line
  • Substring into left, right and middle
  • Do your transformation on the middle
  • Concatenate left + middle + right into a new line
  • Write the new line

  • Now, that might sound too simple to bother with. But start with code that makes that simple algorithm as obvious as possible. You might even try to make a method with exactly those five lines. Some or all will be calls to other methods.
    If you sit down now and think about reading bytes or byte arrays instead of lines, using string buffers instead of strings you will write code that hides the algorithm inside of many lines of optimization. This looks like a pretty tiny program; you can optimize it later after using a code profiler to identify the true hotspots.
     
    • Post Reply Bookmark Topic Watch Topic
    • New Topic
    Boost this thread!