Hi Dan,

on Tue 14-03-2000 18:33 Dan Ackerman wrote:
>
>You got it!

I presume that this exclamation was to confirm to Sean that he does
have posting rights here now.


>Ronald I was thinking about what Sean had said and wondered if
>this wasn't the problem.

As he didn't say much this time, I assume you refer to the recent
problems he had with your latest CAB.OVL and my latest TCP module.
(As discussed in our separate private mail discussions with him.)


>If a connection sends 4k to a connection what happens?
>
>This is just a random number I doubt what I send would ever get
>that high.  HOwever what happens with STiNG if you send a larger block of
>data than the buffer can handle?  Is it reporting the E_OBUFFULL message?

Yes, although as presently coded the behaviour is a little more flexible.
I'm afraid I have to go into 'verbose mode' here, but bear with me please,
as this subject is very important for TCP fluency.

The 'buffer' defined is not implemented as one contiguous buffer in STinG,
but is instead a limit on the total unacknowledged data allowed in a chain
of NDBs which are used instead of a fixed buffer.  One reason for this is
that we can't consider any data properly sent (thus removed from buffer),
until the other end has acknowledged it.  It is only then that we can know
that further retransmission of that data will not be needed.

Inside TCP_send a check is made to see if all of the new data to send
can be added (in a new NDB linked to the chain), without the total data
(not yet acknowledged by the TCP at the other end) exceeding the 'buffer'
size that was declared in TCP_open for this connection.

If that check shows that this size would be exceeded, then the call will
return E_OBUFFULL, and none of the new data will be added to the chain.
The client will have to call again to get any of that data sent.

Obviously, if TCP_send is called with a larger data size than the 'buffer'
size that was declared in TCP_open for this connection, then the call
will always return E_OBUFFUL.  The client must limit its TCP_send data
to chunks less than the 'buffer' size.  Else nothing is ever sent.

One more thing to consider in how STinG handles the 'buffer' is that even
if partial data of one NDB is ACKed, this will increase the size allowed
for next TCP_send call directly by the ACKed amount, even though the NDB
and its data block still remain in the chain.  This is essential to avoid
lockup of the sending when using TCP_send sizes (thus also NDBs) larger
than the outgoing MSS, as incoming ACKs will normally arrive for chunks
of data in multiples of the MSS (for bulk transfers that is).


If you have opinions/suggestions concerning this implementation, I am as
always eager to hear them.


NB: For STinG programmers.
--------------------------
This TCP_open 'buffer' size works almost like an extra limitation on the
TCP send window, as ACK must be received for data in the buffer to increase
the remaining size again.  Therefore it is NOT correct to use a very small
buffer, like many programs do (1 or 2 KB), as this will cause the same kind
of uneven data flow that a very small TCP window does, although applied
only to that connection and in the send direction.

eg:
If STinG machine 'A' is to send bulk data to unknown machine 'B' and the
server/client at machine 'A' uses a 1KB 'buffer' size in TCP_open, then this
will limit the transfers in almost the same way as if machine 'B' had been
configured with a TCP reception window of 1KB.

That will lead to very inefficient bulk transfer !

Instead you should think of the 'buffer' size as a client-imposed TCP_send
window, and as any TCP window it should be sized so as to allow good fluency,
which means that it should have room for at least 3 normal MSS chunks. Larger
size will not hurt any, as it is only the sizes used in TCP_send calls that
lead to KRmalloc calls (to make NDBs).  Therefore you should not calculate
the size on minimum MSS that can occur, but on some larger value, such as
the 1460 bytes MSS which is used (or at least allowed) by most modern ISPs.
Note that three times 1460 is 4380, which is a lot more than many programs
allow for the TCP 'buffer' sizes.

It is still a good idea to use fairly small chunks (a few KB) in TCP_send,
as this reduces fragmentization problems for the ALLOCMEM area, but normally
the 'buffer' size should allow for several such chunks.

Note that this problem will vary depending on the identity of machine 'B',
since TCP implementations perform ACKing differently. Modern implementations
suppress/delay alternate ACKing if they have no own data to send, and that
can double the kind of fluency problem caused by too small TCP_open 'buffer'
at machine 'A'.  (STinG itself does use such methods, as RFCs recommend...)


>I just seem to remember talking Oliver Booklage some time back on
>IRC and he was having a problem with something like this.

I'd be interested to have more details on his problem too, so if you
can remember in greater detail what he said, please tell me.
(Or if he has time and sees this, he can do it himself.)

-- 
-------------------------------------------------------------------------
Regards:  Ronald Andersson                  mailto:dlanor@ettnet.se
http://dlanor.atari.org/    ICQ:38857203    http://www.ettnet.se/~dlanor/
-------------------------------------------------------------------------
