To: stik@ON-Luebeck.DE
Subject: Re: [12] STIK: STIK/DIALER                                                      


On Mon, 26 Aug 1996 09:48:43 Peter Rottengatter <perot wrote: 
>
>On Sun, 25 Aug 1996, Ronald Andersson wrote:
>
>> The way you describe it this would be necessary for a truly non-blocking version
>> since it is evident from the above that regardless of how they are polled, the
>> current routines do block during the actual transfer operations, which may be
>> a majority of the time since the modem speeds are a significant bottleneck.
>
>The other day I calculated some numbers (Remember, I was talking about 
>half a second for an IP packet under certain circumstances). Comparing 
>these number with reality I find that with any host that is not directly 
>linked to my providers host transfer is far slower than modem speed. So 
>the more serious bottleneck is not the modem transmission, but all the 
>time the clients are waiting for data pouring in. With proper programming 
>of the clients we'd already get rid of these dead times, which is a 
>significant improvement.

Yes, of course!  All clients that block this way must be recoded ASAP.
If this is not done, all other improvements (interrupts etc) are wasted, since
the client is going to waste the freed CPU time in its own loop anyway.


>But in order to go all the way we need interrupt driven transfer.
>This requires far more effort, and wouldn't get noticed 
>that much as if we got rid of the problem of improperly used STiK calls.

More importantly:

If STiK is called by a routine which itself blocks other needs of the application,
any CPU time freed by other improvements is irrelevant.

Such clients will even then show no improvement whatsoever !!!


>Thus I would give that lower priority. Instead the client programmers 
>should feel urged to look for ways to improve their use of the STiK 
>client interface, now after I've given a recipe to do this. ;-) .

Yes indeed.


>Since we all talked about a release soon, and some people also mentioned 
>this to the world outside, we should concentrate now on getting a 
>reasonably proper working version ready for release. This means solving 
>actual problems like the CSLIP thing and the communications with the 
>Dialer, and delaying 'new' things like interrupt driven ports beyond this 
>release.

I definitely agree.
The future improvements we've discussed are for future releases.
No way are we going to reach a decision even, about those in time for the
current release we've been talking about (not to mention coding them).


>> For the actual reception of port data I agree, I was thinking more of having a
>> way for STiK to continuously move completed incoming packets into client data
>> structures, such as we have discussed previously.
>> It remains to be seen if and how such structures should be implemented of course.
>
>Ok, it seems reasonable to utilize the 200 Hz interrupt for this. Care 
>must be taken because if lots of packets come in it might take longer to 
>process all data then the time 'til the interrupt fires again. But I 
>guess you know how to work around this ? ;-)

I know some such ways, yes.
If it happens on a continuous basis, however, this means the interface speed is
too high for the available CPU time percentage (regardless of method used).
That can only be solved by freeing more CPU time (eg: Fewer Falcon screen planes). 


>The structures are needed, but should be kept completely STiK internal 
>and transparent for the clients.

I think we mean slightly different things by "internal", since we seem to agree
on buffer usage below.  To me "internal" in this context means the clients don't
even know the stuff exists, and certainly have no direct way to access it.


>If it comes to implementing bigger pieces of code, I always prefer doing 
>one thing after another, this avoids entangling of the bugs, which makes 
>them harder a) to find and b) to fix.

Of course.  What I meant was that both forms of improvement should be planned
and defined so as to simplify the other, since they then support one another.
True multi-client support requires interrupt support, but may also need some
data structure elements not required for single-client uses.


>After all, none of both have a particularly high priority,
>as nobody is actually waiting for it.
>But again, Dan has to decide, as he is gonna do a significant part of the work.

Yes, though we should be able to lend a hand too, if he wants some parts to be
in human-wrought assembler rather than compiled C.


>> >STiK could allocate a chunk of internal memory associated with that 
>> >particular connection to hold all non-global structures (local to that 
>> >connection, or, to that client). This should be done by the open call. 
>> >The close call could release the memory. This way no change for the 
>> >interface is needed, and still each process has it's own set of 
>> >structures. Some may have two or more, because they have two or more 
>> >connections open, but that won't hurt.
>> 
>> That should work yes, but a drawback is that the buffer will then always have to
>> be of a standard size, since STiK can't know what the client needs are.
>> For that to work, the standard size must be sufficient for the most demanding
>> clients, which may lead to a waste of memory.
>
>Well, that's why I proposed a set_buffer() call. I intended it to be used 
>straight after the open() call, freeing the internal buffer, and using 
>the client supplied one for this particular connection. So the connection 
>handle must be the first parameter for set_buffer, buffer pointer and 
>size the next ones. The client thus must not free this buffer before the 
>corresponding close() call. This way any connection has it's standard 
>buffer, unless it explicitly requires the use of a larger one.

I agree.
Once all is clarified, it appears we have very similar ideas on what is needed.

----- snip -----
>> These are the advantages, yes, but this way no client can ask for a buffer
>> adapted to his needs. (eg: CAB.OVL might need an extra-large buffer)
>
>See, above. I guess the set_buffer() solves this.

Yes.


----- snip -----
>> We could cater for both variants of course, letting the 'open' function allocate
>> a buffer of default size, which demanding clients could replace by one they have
>> allocated themselves, by calling 'set_buffer' before doing any I/O.
>
>Yeaaaah, we met finally !!!!!!!   ;-)

Right on!


>> Unfortunately our discussion on the interrupt support makes it clear that it is
>> only between packets that the routines do not block.  During actual reception
>> or transmission they are still apparently locked in the STiK code, spending most
>> of the time waiting for data on the serial interface.
>
>Yeah, but if this resembled all of the waiting the GUI would already 
>feel much nicer to handle.

I think I agree, though I can't quite make sense of that sentence... :-)

My real point (again) with this post is to emphasize that the client improvements
in using the current interface are absolutely required before any more advanced
improvements can have any significant effect.

A self-blocking client will still block, no matter how much CPU time we liberate.

-------------------------------------------------------------------------
Regards:  Ronald Andersson                     mailto:dlanor@oden.se
                                               http://www.oden.se/~dlanor
-------------------------------------------------------------------------
