How I choose buffer size
Costa Lamona
Greenhorn
Posts: 29
posted 9 years ago
Hi
Please correct me if I am wrong.
When I choose a buffer size, I am considering the following
1- How much physical memory I have. Ask "Is the machine is dedicated only for this operation?"
2- How much Sequential Physical memory I have, because buffer will need sequntial place. "I think buffers never swapped to hard-disk, what is the point of it if it is swapped, and also it always implemented as arrays, so I need a sequential place" , please correct that if needed.
3- What is the max amount of incoming data.
for some cases, I only need 4 KB, and I am sure it will be fine.
if you have a server of 4 GB of ram, there is uploading messages will reach 250 MB(s), what buffer size you choose ? Do you think that a buffer of 256 MB(s) is a good choice?
if we are talking about PC with 1 GB(s) of RAM, I always make the upper limit of the buffer size 2 MB(s), because there is a possibility that another programs are running beside my program. What do you think? Is this a good estimation.
Is there a known way to estimate your buffer size?
Thanks
Please correct me if I am wrong.
When I choose a buffer size, I am considering the following
1- How much physical memory I have. Ask "Is the machine is dedicated only for this operation?"
2- How much Sequential Physical memory I have, because buffer will need sequntial place. "I think buffers never swapped to hard-disk, what is the point of it if it is swapped, and also it always implemented as arrays, so I need a sequential place" , please correct that if needed.
3- What is the max amount of incoming data.
for some cases, I only need 4 KB, and I am sure it will be fine.
if you have a server of 4 GB of ram, there is uploading messages will reach 250 MB(s), what buffer size you choose ? Do you think that a buffer of 256 MB(s) is a good choice?
if we are talking about PC with 1 GB(s) of RAM, I always make the upper limit of the buffer size 2 MB(s), because there is a possibility that another programs are running beside my program. What do you think? Is this a good estimation.
Is there a known way to estimate your buffer size?
Thanks
Taariq San
Ranch Hand
Posts: 192
posted 9 years ago
You don't need the buffer to be the size of the message, else don't bother buffering. What if 20 or 200 clients upload messages at once, is that possible in your application?
You could have an 8kb buffer and read and write quite quickly, but this is for you to understand your application usage and fiddle with and test your results as close to production load as is reasonably possible.
Originally posted by Costa Lamona:
if you have a server of 4 GB of ram, there is uploading messages will reach 250 MB(s), what buffer size you choose ? Do you think that a buffer of 256 MB(s) is a good choice?
You don't need the buffer to be the size of the message, else don't bother buffering. What if 20 or 200 clients upload messages at once, is that possible in your application?
You could have an 8kb buffer and read and write quite quickly, but this is for you to understand your application usage and fiddle with and test your results as close to production load as is reasonably possible.
Ulf Dittmer
Rancher
Posts: 42972
73
posted 9 years ago
It depends on how many simultaneous clients you intend to service, if any, and on what else is going on that machine. 256MB seems far too large, while 8KB seems much too small. Not having the benefit of knowing the surrounding circumstances, I'd say you might try with a larger buffer up to 16MB.
Make sure it's appropriate for the message, though - if the message is 100KB then there's no point of using a much larger buffer.
Make sure it's appropriate for the message, though - if the message is 100KB then there's no point of using a much larger buffer.
|