Keep the Socket alive

[surlac surlacovich]

Hello Partners!
I have an Android client that connects to the Server. The client sends ping-message to the server periodically.
When the period about 1 hour, everything works fine. But if I set the period to 24 hours, the client throws exception:

java.net.SocketException: Software caused connection abort: socket write error

Why it works for 1 hour, but doesn't work for 24 hours?

[Maneesh Godbole]

Surely there is more to the stack trace than you posted?
Instead of polling, you might also want to consider Cloud Messaging

[surlac surlacovich]

Maneesh Godbole wrote:Surely there is more to the stack trace than you posted?

Thank you Maneesh. The exception occurs when client calls flush() on BufferedOutputStream instance:

Socket s = new Socket();
s.setSoTimeout(30000);
s.setKeepAlive(true);
s.connect(address);
BufferedOutputStream bos = new BufferedOutputStream(s.getOutputStream(), 1024 * 1024);
bos.write(byteArray);
bos.flush();

java.net.SocketException: Software caused connection abort: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.io.BufferedOutputStream.flush(Unknown Source)

Maneesh Godbole wrote:Instead of polling, you might also want to consider Cloud Messaging

Thank you for suggestion, but right now I don't have any possibility to switch to GCM.

Why the period of sending the message matters?

[Henry Wong]

surlac surlacovich wrote:
Why the period of sending the message matters?

It could just be your network. Some older network equipment just don't like open connections that don't send traffic for long periods of time. They just quietly shut them down. And unfortunately, in many cases, the endpoints can't figure it out until they actually try to send data.

Henry

[surlac surlacovich]

Thank you Henry!
But TCP is on Transport level, which should inherit "recoverability" (from Link layer maybe) and provide reliable connection. Is this right?

[Henry Wong]

surlac surlacovich wrote:Thank you Henry!
But TCP is on Transport level, which should inherit "recoverability" (from Link layer maybe) and provide reliable connection. Is this right?

If the connection is lost, there is nothing that TCP can do to recover from that... kinda make sense, do you really think that it will be able to recover, if I go and yank the network cable between two of the switches in-use?

Henry

[surlac surlacovich]

Thanks Henry!

Henry Wong wrote:... in many cases, the endpoints can't figure it out until they actually try to send data.

So the best solution would be to handle exception and reopen the socket connection?

[Henry Wong]

surlac surlacovich wrote:Thanks Henry!

Henry Wong wrote:... in many cases, the endpoints can't figure it out until they actually try to send data.

So the best solution would be to handle exception and reopen the socket connection?

Is reestablishing the socket even an option? If so, what are you even holding the socket open? Keeping a socket open, when you don't have to, so you can send a ping once in 24 hours is not very efficient, is it?

Henry

[surlac surlacovich]

Henry Wong wrote:
Is reestablishing the socket even an option? If so, what are you even holding the socket open? Keeping a socket open, when you don't have to, so you can send a ping once in 24 hours is not very efficient, is it?

Very good point. The socket connection keeps opened because:
1. server can push data to the client anytime
2. it's very hard to server to initiate connection (in my situation only client can initiate connection, thus server should notify client that it has data to send in case if socket connection is lost)
I believe in every 'data-push' technologies the same approach exists on a client.

Also after this exception is thrown and client creates new socket connection, somehow the new socket uses old output stream. Why this can happen?

[surlac surlacovich]

surlac surlacovich wrote:
Also after this exception is thrown and client creates new socket connection, somehow the new socket uses old output stream. Why this can happen?

has anybody stumbled upon this problem before?

[Winston Gutkowski]

surlac surlacovich wrote:Very good point. The socket connection keeps opened because:
1. server can push data to the client anytime
2. it's very hard to server to initiate connection (in my situation only client can initiate connection, thus server should notify client that it has data to send in case if socket connection is lost)

Those two statements seem contradictory to me. In most client-server situations, the server can't initiate a connection to a client (or at least it can't guarantee to be able to), for all the reasons already explained; but I like to think of it this way - the client might be a laptop, and I've simply turned it off for the night. So you need to come up with some way of sending the data asynchronously.

Basically you need some form of logic like this:

Client establishes connection with server.

Server checks if there is any data to be sent to client, and if so, sends it.

No polling, no open connection, simple async comms.

In fact, I'm pretty sure JMS can do this automatically.

Winston

[surlac surlacovich]

Winston Gutkowski wrote:

Client establishes connection with server.

Server checks if there is any data to be sent to client, and if so, sends it.

No polling, no open connection, simple async comms.

Thanks Winston!
OK, what if one turned off his laptop and go out where no Internet is available. At the same time server has urgent message to the client, so the server sends SMS/MMS/sends courier/etc. to let user know that he need to connect to the server via Internet and download the message. I mean this case involves a lot of overhead (think how long it will take to send SMS and initiate connection back to server, in comparison if we already connected to the server and can push data to the client), thus keeping the open Socket is justified solution.

[Winston Gutkowski]

surlac surlacovich wrote:OK, what if one turned off his laptop and go out where no Internet is available. At the same time server has urgent message to the client, so the server sends SMS/MMS/sends courier/etc. to let user know that he need to connect to the server via Internet and download the message. I mean this case involves a lot of overhead (think how long it will take to send SMS and initiate connection back to server, in comparison if we already connected to the server and can push data to the client), thus keeping the open Socket is justified solution.

No it isn't. If you don't have an Internet connection, you don't have one; if someone's laptop is turned off, it's turned off - and no amount of open sockets is going to get you the data any quicker.

What you might be able to do is have the client automatically initiate a connection whenever it detects a viable feed - and you could do that with a simple ping - but you still have the problem that someone might have their laptop on in a train going through tunnels, so the signal is intermittent. You could also have the server send an "urgent data" message every so often (in the same way that Windows sends you update notices) to remind the client to initiate a connection.

There are several possible alternatives, but an open socket is not one of them.

Winston

[surlac surlacovich]

Winston Gutkowski wrote:
No it isn't. If you don't have an Internet connection, you don't have one; if someone's laptop is turned off, it's turned off - and no amount of open sockets is going to get you the data any quicker.

In this case, server have to notify client that it needs to connect to the server when Internet will be available, but it's not good to notify client to connect to the server when the client has Internet but not connected to the server - it's much easier to keep socket opened and avoid overhead of sending notification to the client.
Winston, am I right that you consider sending notification to the client when both server and client has Internet but not connected to each other (and close the connection to release Socket right after the data received), is good solution?

Winston Gutkowski wrote:
What you might be able to do is have the client automatically initiate a connection whenever it detects a viable feed

That's right, when client got Internet it automatically connects to the server and keeps the connection opened.

[Winston Gutkowski]

surlac surlacovich wrote:That's right, when client got Internet it automatically connects to the server and keeps the connection opened.

Fine. You're plainly not in the mood to listen, so give it a go.

Remember though that you came here for advice, presumably because you had a problem, and you've been advised that what you want to do is not a good idea; and also that there are other alternatives.

But if you're determined to force this square peg into a round hole...hey, it's your time.

Good luck.

Winston

[Winston Gutkowski]

surlac surlacovich wrote:Winston, am I right that you consider sending notification to the client when both server and client has Internet but not connected to each other (and close the connection to release Socket right after the data received), is good solution?

No. If both "ends" have a viable Internet connection, then presumably you can open a channel. Your problem is what to do when one end of that connection (presumably the client) is down.

Think about how it works with e-mail: the sender doesn't need to care if you're running or not, s/he simply sends a message to your "post office", whose responsibility it is to forward it on to your Inbox when it receives a message from you saying "Hi, I'm awake".

Well, the very same is true of async messaging (which I know JMS - and I'm sure many other messaging protocols - supports).

I'll say it one more time - you don't need an open socket.

Winston

[surlac surlacovich]

Winston Gutkowski wrote:
No. If both "ends" have a viable Internet connection, then presumably you can open a channel. Your problem is what to do when one end of that connection (presumably the client) is down.

You're talking about publish/subscribe model, in which I'm sure you have deep insight, but I just want to remind that almost every implementation of this model uses long-lived TCP connections under the hood (example).
There are 2 ways to "subscribe":
1. polling (connect to the publisher periodically, check for updates, disconnect)
2. long-lived TCP connection: keep connection to the publisher opened as long as possible.
Each approach has it's pros and cons but most popular is 2'nd (consider Jabber, Google GCM, etc.).

So I'm agree this model is effective and I'm actually using it on the server. But I'm talking about implementation. So by Keeping the Socket alive I meant this 2'nd approach.
That's why it's hard for me to understand why you object against this approach. Maybe I don't understand you completely.

[surlac surlacovich]

OK, so currently I didn't find any significant arguments to forbid keeping the socket opened for a long time, despite all previous posts by Winston which I really appreciate and had pleasure to read.
Because all the conversation narrowed down to contradistinction between polling and long TCP connections.