Win a copy of Practical SVG this week in the HTML/CSS/JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

File writing synchronization

 
Gabriel Beres
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi guys,

I have service which accepts transfer objects (from an online stream) as parameter and write them into an ascii file.
The current implementation works like that the service writes incoming objects immedatelly into a temporary file, and there is a backround thread which fires every 5 minutes, and takes care of publishing the temporary file. Publishing means renaming the file to a unique name, and creation of a new temporary file.


Can synchronization issue happen in this soulution? (I guess it can) If yes, any suggestion how to overcome it?

Thanks

 
Lalit Mehra
Ranch Hand
Posts: 384
Eclipse IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
i am taking it as follows :

1. you are working with two threads
2. one for temporary data writing
3. and one for publishing the temporary data

i think you might be needing synchronization if in case both the threads try to access the same data at the same time.
 
Gabriel Beres
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yeah this is what i tought. Any idea how to synchronize this without loosing my requirements?
 
Lalit Mehra
Ranch Hand
Posts: 384
Eclipse IDE Firefox Browser Java
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
use thread communication mechanism ... i mean

wait()
notify()

these two methods will be very helpful in your scenario ...

 
Gabriel Beres
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Can you recommend me some exact steps for doing this? (applied to my scenario)

 
Buchi Reddy Busireddy
Greenhorn
Posts: 2
Eclipse IDE Java Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It depends on lot of other factors like how much time is taken in writing into the temporary file, how much time is taken in processing the file, etc.

You can use the below code to achieve your job.

Assume myFile is the temporary file, the thread which is writing into temp file will get a lock on it and write. If the other thread was trying to process that file, it won't get the lock. when the other thread gets the lock, it processes and releases the lock.



Please let me know if you need any further help.
 
Sergey Babkin
author
Ranch Hand
Posts: 50
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
On Unix/Linux the worst that can happen is the "published" file containing an incomplete record. The renaming is atomic, so no problem there. However the writer might open the temporary file, which would then be renamed, so the write will complete after the file has been renamed. Windows is a pain, and likely would not allow to rename the file while it's open.
 
Nitesh Kant
Bartender
Posts: 1638
IntelliJ IDE Java MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It is always better to indicate in the file name itself that the file is partly written. Something like: myfile.txt.part
When the writer finishes writing it, renames the file to myfile.txt.

The publisher should not touch files that contain ".part" in the name.

Synchronizing using threads assumes that both the threads are running in the same process which may limit your design.

Gabriel Beres wrote: and creation of a new temporary file.


Does it mean that the contents of the main file is copied to a new file? If yes, then this definitely means that you need to make sure nobody is writing on the file before you copy content.
 
Gabriel Beres
Ranch Hand
Posts: 62
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Nitesh Kant wrote:It is always better to indicate in the file name itself that the file is partly written. Something like: myfile.txt.part
When the writer finishes writing it, renames the file to myfile.txt.

The publisher should not touch files that contain ".part" in the name.

Synchronizing using threads assumes that both the threads are running in the same process which may limit your design.

Gabriel Beres wrote: and creation of a new temporary file.


Does it mean that the contents of the main file is copied to a new file? If yes, then this definitely means that you need to make sure nobody is writing on the file before you copy content.


This sounds like a good idea. I will implement this.
Thank you.
 
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!