Re: Message Pushing and TCP Transport

[Dave Crocker <dcrocker@brandenburg.com>]

At 11:21 AM 8/23/2001, Daniel Manley wrote:
>>>... somewhere along the line, a process or thread will have to poll 
>>>something to see if there's anything to do.
>>
>>
>>wrong.
>
>You keep saying that I'm wrong without describing your alternative. That's 
>not very fair, is it?

thought it was obvious.  instead of having the consumer constantly polling 
the queue, the producer just dispatches the consumer, after adding to the 
queue.


>>>Except for queuing when the message can't be delivered taken right away.
>>
>>queuing is an entirely separate bit of mechanism and overhead, with equal 
>>overhead on top of push or pull.
>
>If you're going to push queued messages, then I don't see how it's an 
>entirely separate mechanism.  Yes, with equal overhead, except that 
>pulling is not done by the server, so the load is distributed a little 
>more equally between client and server.

pulling isn't done by the server?  interesting idea.  please explain how 
the client obtains the data without any work beihng done by the server.

I think you meant that pulling is not INITIATED by the server and, of 
course, that is the definitional difference between push and pull.


>>the point that is being missed is polling permits only a subset of the 
>>behaviors and efficiencies that push permits.  Push demonstrates vastly 
>>better scaling properties.
>
>Can you enumerate these behaviours, efficiencies and scaling properties? I 
>see the same messages being passed either way so the argument is over the 
>delivery method.

it has now been explained repeatedly:  constantly polling means that a 
request and a response is sent and received.  Most of the time, the queue 
is empty, so the poll is wasted effort.

A server with a large number of clients, therefore, processes a large 
number of polling requests.  That makes for amount of wasted overhead.


>But that was in a really controlled environment.  Only a handfull of big 
>clients.  Provreg is defining a generic protocol which will have many 
>application groups.  I don't think it would be fair to the smaller players 
>to impose a heavy-duty protocol.

fine.  I agree.  (I hope it is not necessary to note that I have 
substantial appreciation for the issues of getting successful Internet 
protocols designed and adopted.)

Is it fair on the larger players to impose heavy-duty 
inefficiencies?  (Please skip the obvious path of citing some current 
larger players as saying that they do not want push.)


>>The server needs to get the data to the client, no matter what.  You seem 
>>to feel that sending it immediately, rather having to wait an arbitrary 
>>amount of time, somehow incurs onerous overhead.  It doesn't.
>
>We seem to be going in circles here.  I'm saying there are two 
>activities:  generating messages and delivering messages.  Pushing puts 
>these two activities in the server, while polling splits them between the 
>server and the client, respectively.

It does no such thing.  The server is still a full participant in getting 
the data to the client.  What is changed is ONLY the question of how the 
exchange is initiated.


>   With the client pinging anyway (yes, an integral part of my argument), 
> the total overhead is roughly the same, but balanced better between two 
> parties.
>
>>Frankly I can't figure out the processing model that you are describing, 
>>since you seem to view queuing as intimately involved in the i/o model, 
>>although it does not need to be.
>
>Then *please* present an alternative so I can understand your point of 
>view better.
>
>>>What happens if there is a persistent queue for a particular registrar? 
>>>If there are a bunch of old (but not yet expired) transfer notifications 
>>>waiting around and a notification of low balance shows up, which do you 
>>>try to send the transfers first?  Do you implement a prioritized 
>>>queue?  If you do, should we include the message priorities in the 
>>>protocol spec?  Then registrars have to additionally deal with message 
>>>priorities.
>>
>>None of this is relevant to the choice of mechanism that sends the "next" 
>>entry from the queue.
>
>So you're saying that the queuing of a backlog of message must obey the 
>order in which the messages were issued?

I am saying no such thing.  I am saying that the mechanism for sorting the 
queue is independent of the mechanism for initiating transfer of the top of 
the queue to the consumer.

d/

----------
Dave Crocker  <mailto:dcrocker@brandenburg.com>
Brandenburg InternetWorking  <http://www.brandenburg.com>
tel +1.408.246.8253;  fax +1.408.273.6464