does a nio socket buffer data internally too?

[Jonathan Doe]

hy people

i try to write something like a portmapper

class NIOProxy extends Thread{
    SocketChannel     srcChannel = null;  // src and dst nio sockets
    SocketChannel     dstChannel = null;
    InetSocketAddress source = null;     // src and dst address
    InetSocketAddress destination = null;
 
    public void run(){
        try{
            long datasize =  0;
 
            Selector selector = Selector.open();
            // Recording server to selector (type OP_READ)
            this.srcChannel.register(selector,SelectionKey.OP_READ);
            boolean end = false;
            
            // Infinite server loop
            for(;!end    :wink:     {
              // Waiting for events
              selector.select();
              // Get keys
              Set keys = selector.selectedKeys();
              Iterator i = keys.iterator();
 
              // For each keys...
              while(i.hasNext()) {
                SelectionKey key = (SelectionKey) i.next();
 
                // Remove the current key
                i.remove();
                if (key.isReadable()) {
                        SocketChannel client = (SocketChannel) key.channel();
                        int BUFFER_SIZE = 32;
                        ByteBuffer buffer = ByteBuffer.allocate(BUFFER_SIZE);
                        int count = 0;
                        try {
                            // read incoming data
                            while((count=client.read(buffer))>0){
                                datasize += count;
                                buffer.flip();
                                while (buffer.hasRemaining()) {
                                     this.dstChannel.write(buffer);
                                }
                                System.out.println("i got "+datasize+"bytes something from "
                                        +this.srcChannel.socket().getRemoteSocketAddress().toString()
                                        +" to "+this.dstChannel.socket().getRemoteSocketAddress().toString());
                                buffer.clear();
                            }
                            if (count < 0) {
                                    // close channel on EOF, invalidates the key
                                    this.dstChannel.close();
                                    this.srcChannel.close();
                            }
                        }
                        catch (Exception e) {
                           // client is no longer active
                           e.printStackTrace();
                           end = true;
                           break;
                        }
                }
                
              }
            }
            
            try{
                this.srcChannel.close();
            }
            catch(IOException ioe){}
            try{
                this.dstChannel.close();
            }
            catch(IOException ioe){}
        }
        catch(Exception e){
            e.printStackTrace();
            System.out.println(e.getMessage());
        }
    }
 
    public NIOProxy(InetAddress srcHost, int srcPort, InetAddress ideAddr, int idePort,String session)
        throws IOException
    { 
       // first we connect to source and dest
        SocketChannel src = SocketChannel.open();
        SocketChannel dst = SocketChannel.open();
 
        // nonblocking I/O
        src.configureBlocking(false);
        dst.configureBlocking(false);
 
        // Connection to host port 8000
        src.connect(new InetSocketAddress(srcHost,srcPort));
        dst.connect(new InetSocketAddress(ideAddr,idePort));
 
        // Create selector
        Selector srcselector = Selector.open();
        Selector dstselector = Selector.open();
        
        // Record to selector (OP_CONNECT type)
        SelectionKey srcclientKey = src.register(srcselector, SelectionKey.OP_CONNECT);
        SelectionKey dstclientKey = dst.register(dstselector, SelectionKey.OP_CONNECT);
 
        SocketChannel srcconchan = this.finishConnection(srcselector,srcclientKey);
        SocketChannel dstconchan =this.finishConnection(dstselector,dstclientKey);
 
        // now write some information that we can start port mapping
        ByteBuffer buffer = null;
        buffer = ByteBuffer.wrap(new String("conn: "+session+"\n").getBytes());
        buffer.flip();
        srcconchan.write(buffer);
        buffer.clear();
        
        // start the mapper for both Channels
        new NIOProxy(srcconchan,dstconchan);
        new NIOProxy(dstconchan,srcconchan);
    }
 
    private NIOProxy(SocketChannel src,SocketChannel ide) {
        this.srcChannel      = src;
        this.dstChannel      = ide;
        this.start();
    }
    
    private SocketChannel finishConnection(Selector selector,SelectionKey clientKey) throws IOException{
 
        // Waiting for the connection
        while (selector.select(500)> 0) {
 
          // Get keys
          Set keys = selector.selectedKeys();
          Iterator i = keys.iterator();
 
          // For each key...
          while (i.hasNext()) {
            SelectionKey key = (SelectionKey)i.next();
 
            // Remove the current key
            i.remove();
 
            // Get the socket channel held by the key
            SocketChannel channel = (SocketChannel)key.channel();
 
            // Attempt a connection
            if (key.isConnectable()) {
 
              // Connection OK
              System.out.println("Server Found");
 
              // Close pendent connections
              if (channel.isConnectionPending())
                channel.finishConnect();
              return channel;
            }
          }
        }
        return null;
    }
}

the code is mostly from example code
it works as following:
1. an NIOProxy object is created
2. the constructor connects to tcp addresses
3. it sends a connection string with a sessionid to one of the 2 hosts
4. it creates 2 threads both just reading data and forward it to the other socket

the problem i have:
the portmapper above is a workaround for some firewall rules on the remote host
everything works great, but after some bytes of data (i think it may be around 2kb) it get's stuck
the two apps which want to communicate through this port mapper, send some data to each other

what i think the problem is:
my question is, if nio sockets buffer data internally?
cause i think i read and write the data correctly, BUT the data isn't sent if it was too few data to send
so is it possible to flush() the sockets?
maybe the code is not right?
the interesting thing about it is, none of the connections get closed
it seems that both applications (local and remote) are waiting for some data
and i think the data hangs in an internal buffer in the socket api
but that's hard to verify
but maybe i got the threads wrong? can there be a problem in my code?

the output looks like this:
Server Found
Server Found

i got 264bytes something from remote.host/192.168.11.61:23453 to localhost/127.0.0.1:10001
i got 422bytes something from localhost/127.0.0.1:10001 to remote.host/192.168.11.61:23453
i got 1232bytes something from remote.host/192.168.11.61:23453 to localhost/127.0.0.1:10001
i got 486bytes something from localhost/127.0.0.1:10001 to remote.host/192.168.11.61:23453
i got 1264bytes something from remote.host/192.168.11.61:23453 to localhost/127.0.0.1:10001
i got 3624bytes something from remote.host/192.168.11.61:23453 to localhost/127.0.0.1:10001
i got 502bytes something from localhost/127.0.0.1:10001 to remote.host/192.168.11.61:23453
i got 4120bytes something from remote.host/192.168.11.61:23453 to localhost/127.0.0.1:10001

many thanks

[ October 23, 2006: Message edited by: Jonathan Doe ]
[EDIT: altered code and striped output]

[...and I inserted some line breaks into the print statement, so it doesn't force the whole window to be unnecessarily wide - Jim]
[ October 23, 2006: Message edited by: Jim Yingst ]

[Jim Yingst]

The output from your code doesn't match the print statement you show. Can you provide a version of code and output that go together? You can edit your post using the icon, so we don't see a second copy of all that stuff.

How do you know that something is stuck? Looks like there's something remote that has written about 4 kb to you, and something local that has written about 502 b to the remote machine. Do you know if the data has been received at the other end? Has either end attempted to write more than has been sent? Also you say you think something happens around 2 kb, but I don't see anything special in the output there. What's special about 2 kb?

[Jonathan Doe]

The output from your code doesn't match the print statement you show

changed code to fit your needs

How do you know that something is stuck?

i don't know 100%
that's why i'm asking ;-)
but the application's don't continue with their work, seems that both remote and win app are waiting for something

Looks like there's something remote that has written about 4 kb to you, and something local that has written about 502 b to the remote machine. Do you know if the data has been received at the other end?

i'm sure the data is received by the remote server and my host receives sent data
but this doesn't mean my port mapper receives/sends it what comes from the application on my windows box

Also you say you think something happens around 2 kb, but I don't see anything special in the output there. What's special about 2 kb?

afaik 2kb (i think it was 1500 incl. packet overhead) is standard ethernet MTU on unix systems
that's why i think it could have something to do with 2k (maybe it's the same on win32 box)

maybe my questions weren't clear
Q1: do nio sockets buffer internally?
Q2: is there a buggier bug in my code than missing correct System.out.println's?
i'm not asking for a code correction, but maybe someone is a better java hacker than me and spot's a problem within minutes (something like not flipping ByteBuffer)
would be great
Q3: any tips or hints (maybe there is an java open source port mapper, altough i didn't find one)

thanks sheriff
[ October 23, 2006: Message edited by: Jonathan Doe ]

[Jim Yingst]

[Jonathan]: Q1: do nio sockets buffer internally?

That, I don't know. Though even if there's some buffering, I would think that it shouldn't cause the communications to just grind to a halt. Maybe I'm just being naive. But I'm not seeing any flush() methods that you should've called, so I would have to think it's the channel's responsibility to

[Jonathan]: Q2: is there a buggier bug in my code than missing correct System.out.println's?
i'm not asking for a code correction, but maybe someone is a better java hacker than me and spot's a problem within minutes (something like not flipping ByteBuffer)
would be great

Not that I've found. Though it's hard to tell; it's been awhile since I used NIO, and it would be easy for a bug to slip by me in this class.

[Jonathan]: Q3: any tips or hints (maybe there is an java open source port mapper, altough i didn't find one)

One thing I noticed is that it looks like each source/destination pair has its own Thread to handle the transfer. Which means you aren't really getting any benefit from using a Selector here, I think. Those are more useful for managing a large number of connections with a single thread - or maybe with a single "main" thread, and a pool of worker threads that tasks are handed off to. If you're going to have one full thread per connection, I think you might as well use blocking IO and forget the Selector, since blocking IO is probly simpler. That doesn't directly solve our problems, except that a simpler API would be harder for bugs to hide in.

Sorry I don't have a more direct answer for the problem you're having.