This is an attempt and proposal for a solution to many of the points raised
in http://poe.dyndns.org/~troc/notes/poe-standards-summary.text.  It will
describe extensions to the POE kernel and calling semantics. These
extensions will probably be implemented as an overloaded kernel, though might
require some modifications to POE's kernel.

One of my goals is that this new system will interoperate as seamlessly as
possible with all legacy POE code.

My vision of a POE system is one of loosely coupled Components talking to
each other.  Doing an action, such as posting a state, calling a callback or
what have you, should be as polymorphic as possible.

This document is literate programming.  Sorta.

Also, bear in mind that this all came to me during a THC halucination.  And
that the author is insane.

While I'm trying to keep the API as close to what POE currently uses, there
are a few new concepts and a few clarifications of current concepts. 
Specifically, what were previously called states and events will be split in
2 : Messages is used to designate information about an event, say it's
destination, what to do when an error occurs, where answers should be sent
to, etc.  Messages end up at a destination, a bit of perl code somewhere
that handles them.  These are Session States.

So (this is important), a Messages is created, propagates through the
kernel's queue (possibly to other kernels via IKC) and then invokes a State
at it's destination.  

Currently, there are 5 ways of creating new messages: post(), call(),
yield(), signal(), and the alarm functions.  I'm going to ignore the alarm
functions for now.

Here is how the new system will defined post, call and signal.

sub Kernel::post
{
    my($self, $session, $state, @args)=@_;
    $self->colour_new();
    $self->propagate({session=>$session, 
                      state=>$state,
                      called=>'post',
                      args=>\@args});
    return;
}

sub Kernel::call
{
    my($self, $session, $state, @args)=@_;
    $self->colour_new();
    return $self->propagate({session=>$session, 
                             state=>$state,
                              called=>'call',
                              args=>\@args,
                              wantarray=>wantarray()});
}

sub Kernel::yield
{
    my($self, $state, @args)=@_;
    $self->colour_new();
    $self->propagate({state=>$state,
                      called=>'post',
                      args=>\@args});
    return;
}

sub Kernel::signal
{
    my($self, $signal, @args);
    $self->colour_new();
    $self->propagate({session=>$signal, 
                      state=>'invoke,
                      called=>'signal',
                      args=>\@args});
    return;
}

You will note 2 new functions: colour_new and propagate.  colour_new is part
of a new system that allows one to influence future messages.  These
functions will allow us to specify where errors go to, where a reply should
go to, and more.  This is implemented as Tags, which can be scoped.

So, if we do 
    $kernel->errors_to($honk);
    $kernel->post(....);
    $kernel->call(....);
only the post() is tagged by the errors_to.  However, if we do
    my $t=$kernel->errors_to($honk);
all future post()s, call()s and so on, from the current session, will be
tagged by the errors_to(), until $t->DESTROY is invoked.


Sum up: a Message is a Destination, with zero or more tags attached to it.

The concept of colour will be described later.


propagate is where some of the magic happens:

sub Kernel::propagate
{
    my($self, $message, @args)=@_;

    $message=$self->__message_fix($message, [@args]);
    $self->__message_tag($message);
    $self->__message_enqueue($message);
}

A lot of mess is hidden in __message_fix:

sub Kernel::__message_fix
{
    my($self, $msg, $args)=@_;

    my $r=ref $message;

                        # $kernel->propagate("/S=alias/M=state", ...);
    unless($r) {        
        ($kernel, $session, $state)=$self->__message_parse($message);
    }
Ha!  did you notice that I've introduced a new syntax?  What is
/S=alias/M=state, you ask?  It's a way of designating the State 'state' of
the Session 'alias'.  There are many other forms possible, __message_parse()
deals with them indirectly.

                        # $kernel->propagate(['alias', 'state'], ....);
                        # $kernel->propagate(['alias', 'state', ....]);
    if('ARRAY' eq $r) {
        ($session, $state, $margs) = @$message;
    }
Perceptive readers might want to know why I haven't included kernel in the
above.  This is because an Alias could point to an IKC proxy, which would
deal with getting the Message to another Kernel.

                        # $kernel->propagate(sub {....});
    if('CODE' eq $r) {
        $state=$message;
    }
Did you catch that last bit?  A State can be a coderef.  Why do we want
this?  So that the following would Just Work.

Example:
    POE::Component::Server::Honk->create(
        ......
        SomethingEvent=>sub{ ..... },
        OtherEvent=>'State',
    );

Ideally it will be invoked in the Session it is defined in.  We can't
control that here though.  In fact, I really don't know how we will be able
to guarantee this in the above case.  However, the following is easy 
Example:
    POE::Component::Server::Honk->create(
        ......
        SomethingEvent=>Message->new(sub{ ..... }),
        OtherEvent=>'State',
    );

in sub Kernel::__message_fix
    if('HASH' eq $r) {      # $kernel->propagate({...}, ...);

There has to be some magic inserted right here!  A message, if created via
->post() or ->call(), might have a Message object itself in
$message->{state}.  In which case we will do a recursive __message_fix or
something... In fact, most of __message_fix would probably be better in
Message->new()....
in sub Kernel::__message_fix

        $message=Message->new($message);
        $r=ref $message;
    }

    unless('Message' eq $r) {   
        $message=Message->new({kernel=>$kernel, session=>$session,
                               state=>$state});
    }
    else {          # $kernel->propagate(Message->new({...}));
                    # Do nothing for this
    }

    $message->{kernel}||=$self->ID;
    $message->{session}||=$self->current_session->ID;

Argument handling is fraught.  There are 3 possible places that arguments
can be defined:

1- In the hashref or Message object passed to us.  These appear are already
in $message->{args} at this point
2- In the string representation of a Destination.  These these appear in
$margs at this point.
3- As parameters to propagate()

Number 1 and 2 mean that the some code set up a Response or Callback or
something and wants to "parameterise" the call back.  As in 
example:
    POE::Component::Server::TCP->create(
        Address=>$host
        Acceptor=>{state=>'connected', args=>[$host]},
    );
    # Server::TCP of course invokes this as 

    $kernel->post($alias=>$self->{Acceptor}, @more_args);

Now, what order should these appear to the State (connected in our example)
that is finaly invoked?  Should they all be in @_ at all?  I'd say yes, and
in the following order.

In sub Kernel::__message_fix
    push @{$message->{args}}, @$margs;
    push @{$message->{args}}, @$args;

I choose this order because the arguments that are controlled by the creator
of a callback go first.  Then the ones that are controlled by the user of
the callback.  This means that in the Destination state, we don't have to
check the first N params, because we know we are the ones who set them.

example:
    inline_states=>{
        connected=>sub {
            my($heap $host, $handle, $remote_addr, $remote_port)=
                    @_[HEAP, ARG0..$#_];
        }
    }


Of course in cases like
example:
    $kernel->propagate({state=>'poe:session/state?honk=bonk', args=>[$foo]},
                        @something);

We are using all 3 methods.  Which is a bit silly, but still defined.

A case could also be made that Message-specific arguments for a callback
could be retrieved by querying the Kernel.  However, this simply adds to the
complexity and won't interporate with existing code.

Back to sub Kernel::__message_fix

    die "No state to invoke!" unless $message->{state};

    return $message;
}


One problem that I've often met is that there is no way, in one string, to
designate the kernel/session/state destination triplet.  I've proposed many
syntaxes, from URL-like (ikc://kernel/session/state), bang-paths
(kernel!session!state), e-mail-like (state@session.kernel) and LDAP-like
(/K=kernel/S=session/M=state).  I've decided to punt on this issue in the
worst way: make it extensible.  Each of the above variants would be
implemented as a parser.  These parsers would register themselves with the
kernel.  

package My::Parser;
use strict;
use POE::Kernel;
$poe_kernel->parser_register(__PACKAGE__);

sub Kernel::parser_register
{
    my($self, $obj_or_package)=@_;
    push @{$self->{parsers}}, $obj_or_package;
    return;
}

We then use these when we are given a string as a message destination.

sub Kernel::__message_parse
{
    my($self, $msg)=@_;

    my @ret;
    foreach my $parser (@{$self->{parsers}}) {
        next unless $parser->parse($msg, \@ret);
        return @ret;
    }
    # all the parsers failed!
    # maybe it's an alias?
    my $session=$self->alias_resolve($msg);
    if($session) {
        $ret[1]=$session->ID;
        return @ret;
    }
    # everything failed!  
}

Alias resolving could also be implemented as a Parser package.  However,
because there is little to no control over the order of parser invocation,
it is here as a special case.

Because we have no control over the order of parser invocation, it is
important that each Parser be as specific and strict as possible when
parsing a Destination string.  For instance, the following implementation
of an LDAP-like form might be insufficient.
 
sub My::Parser::parse
{
    my($package, $msg, $output)=@_;

    my @ret;
    if($msg=~ s(/K=([^/]+))()) {
        $ret[0]=$1;
    }
    if($msg=~ s(/S=([^/]+))()) {
        $ret[1]=$1;
    }
    if($msg=~ s(/M=([^/]+))()) {
        $ret[2]=$1;
    }

    # make sure we have at least a State, and there is nothing left.
    return unless $ret[2] and not $msg;

    @$output=@ret;
    return 1;
}

While it's a good begining, [^/]+ might be to broad a regex to match the
Kernel, Session or State's name.  This needs to be decided.


On to the subject of tags.

Tags are some extra information that influences how a Message is handled at
it's various stages of propagation.  Dealing with these tags will be
hellish.  Where are they defined?  How long do they last?  What do they
mean?  

Colour is one type of tag.  If we think of States as nodes, and of Messages
as edges in the nodes, we get a graph of the propagation of messages through
the system.  That is, one Message will probably create more Messages when it
invokes a State, which could then create even more Messages and so.  If each
Message has a Colour, we can trace it's influence throughout the system. 
And we can discover the edges of the graph that has same-coloured nodes.

Why is this useful?  

We could designate the return values of all the edge States as the return
value of the orignal Message and have it sent to some Destination.  This
would get rid of one of the uses of call().  It would also do away with
IKC's post_respond() and all other forms of "post() but give me an answer
sometime."

We now have an excelent place to hook in some exception handling.

We can deal with something that I call a Crossbar, which is sort of like
a semaphore or a way point.  That is, say a task requires 5 steps (A B C D
E), A has to come first, E last, the others can happen concurrently.  So A
would post B, C, D.  And E would wait until the coloured graphs from those 3
Messages would die.

We can "effortlessly" add timeouts to post-with-response and other
situations.

If we want to, we could make the death of one colour wait for a new colour
to die.  This way we could auto-vivify IKC connections.  All with the
aformentioned timeouts.

Let's look at a possible implementation.

sub Kernel::colour_new
{
    my($self)=@_;
    my $id=$self->colour_alloc;
    $self->colour_incr($id);
    return $self->tag_create(name=>colour,
                             value=>$id, 
                             after_invoke=>
                                sub {$poe_kernel->colour_decr($id, $_[0])}
                            );
}

sub Kernel::colour_alloc
{
    my($self)=@_;
    my $id=$self->{colourID}++;
}

sub Kernel::colour_incr
{
    my($self, $id)=@_;
    $self->{colours}{$id}++;
    return;
}

sub Kernel::colour_decr
{
    my($self, $id, $message)=@_;

    unless(exists $self->{colours}{$id} or DEBUG) {
        croak "Decrementing an empty colour!";
    }
    if($self->{colours}{$id} <=0) {
        croak "Decrementing a dead colour!";
    }
    $self->{colours}{$id}--;
    return unless 0==$self->{colours}{$id};

    delete $self->{colours}{$id};

    $self->colour_died($id, $message);

    return 1;
}


We'll leave colour_died be for now.  It would be the place where Crossbars,
Response and so on are dealt with.


But this tells us little about tags.  So far, we've seen that Tags have a
name, a value and something that looks like a callback called
'after_invoke'.  These callbacks are invoked at various points in the Tag's
life.  'after_invoke' is invoked after the Message this Tag is attached to
is invoked.  Make sense?

Anyway 

sub Kernel::tag_create
{
    my($self, $colour, %tag)=@_;

    croak "No tag name!  unless $tag{name};
    my $id=$self->{tagID}++;
    $tag{ID}=$id;
    $tag{references}=0;

    my $ret;
    if(undef==wantarray) {
        $tag{from}='TAGS_NEXT';
    }
    else {
        $tag{from}='TAGS-SESSION-'.$self->get_active_session->ID;
        $ret=bless \$tag{id}, 'Tag::Mortal';
    }

    $self->tag_set($tag{from}, \%tag);

    $self->{TAGS_ALL}{$tag{id}} = \%tag if $tag{references} > 0;

    return $ret;
}

$self->{TAGS_ALL} is a hashref, key is Tag ID, values are the tags.

$self->{TAGS_NEXT} is an hashref for all the tags that will be applied to
the next Message.  Keys are Tag names, values

$self->{TAGS-SESSION-id} is a hashref of all the tags that will be
applied to the Messages from the Session named 'id'.

$self->{TAGS-COLOUR-id} is a hashref of all the tags that are associated with
a colour.

sub Kernel::tag_set
{
    my($self, $where, $tag)=@_;

    if($self->{$where}{$tag{name}}) {
        $self->tag_merge($self->{TAGS_ALL}{$self->{$where}{$tag{name}}}, $tag);
        return;
    }
    $tag{references}++;
    $self->{$where}{$tag{name}}=$id;
    return 1;
}


Tag::Mortal is just a way of getting detecting when DESTROY gets invoked. It
should be noted that because of perl's optimisation in some cases DESTROY
doesn't happen when we think it should.  Judicius use of undef() is
recomented.

sub Tag::Mortal::DESTROY
{
    my($self);
    $poe_kernel->tag_destroy($$self);
}


This means, that when a Tag leaves scope, it is removed from whereever it
was created.  It could still be referenced though.

sub Kernel::tag_remove
{
    my($self, $id)=@_;
    my $tag=$self->{TAGS_ALL}{$id};
    return unless $tag;
    $self->tag_invoke_hook($name, 'remove');

    if($id eq delete $self->{$tag->{from}}{$tag->{name}}) {
        $tag->{references}--;
    }
    else {
        carp "Deleting tag $tag->{name} from $tag->{from} but it wasn't there!";
    }
    $self->tag_GC;

    return 1;
}

sub Kernel::tag_invoke_hook
{
    my($self, $id, $hook, $args)=@_;
    my $tag=$self->{TAGS_ALL}{$id};
    return unless $tag and $self->__tag_valid_hook($hook);

    return unless $tag and $tag->{$hook};
    $self->propagate($tag->{$hook}, @$args);;
}

Woah!  We're going around in circles!  Why would we use propagate() to do
whatever the tag wanted to have happen when it's removed from kernel? 
Because that way we can leverage all the possible invocation styles that
propagate() gives us!

Let's look a possible implementation of errors_to:

sub Kernel::errors_to
{
    my($self, $message)=@_;

    return $poe_kernel->tag_create(name=>'errors_to', value=>[$destination],
                            merge_value=>sub{
                                my($prev_value, $new_value)=@_;
                                return(@$prev_value, @$new_value);
                            });
}

(Fuck.  Tags can include coderefs, so can't be serialised.  How is IKC going
to deal with Messages that include such tags?  I'll punt on that for now.)

sub Kernel::tag_merge
{
    my($self, $old, $new)=@_;
    my $code=$old->{merge_value} || $tag->{merge_value};

    return unless($code);

    $old->{value}=$code->($old->{value}, $tag->{value});
}

Oh damn!  It seems that not all the hooks can be dealt with by a simple call
to propagate.  Because this one here needs to be done here and now.  Ah
well.


Remember that propagate() did a __message_tag()?  Well here we go:

sub Kernel::__message_tag
{
    my($self, $message)=@_;

First we look to see if we've changed colours...
in sub Kernel::__message_tag

    my $old_msg=$self->get_active_message;
    if($old_msg) {
        my $new_colour=$message->{'TAGS_NEXT'}{colour}{value};
        if(($new_colour||0) ne $new_colour) {
So, we've changed colours.  Forget about copying tags from the old Message
in sub Kernel::__message_tag
            undef($old_msg);
        }
    }
Right.  Tags get copied from the list for current Session.  
in sub Kernel::__message_tag
    my $session=$self->get_active_session;
    if($session) {
        $self->__message_add_tags($message, 'SESSION',
                        $self->{"TAGS-SESSION-".$session->ID});
    }

Then we copy the tags for the current Message (ie, Colour), which has the
effect of copying the colour over.
in sub Kernel::__message_tag
    if($old_msg) {
        $self->__message_add_tags($message, 'COLOUR', $olg_msg->{TAGS});
    }
    else {
We must make sure this Message is tagged with at least a colour!
in sub Kernel::__message_tag
        $self->colour_new();
    }

Then we copy the accumulated tags for the "next" Message.
in sub Kernel::__message_tag
    $self->__message_add_tags($message, 'CALLER', $self->{'TAGS_NEXT'});

    $self->{TAGS_NEXT}={};
    return;
}


sub Kernel::__message_add_tags
{
    my($self, $message, $whence, $where)=@_;

    while(my($name, $id)=each %{$self->{$where}}) {
        my $tag=$self->{TAGS_ALL}{$id};
        next unless $tag;
        if(exists $message->{$name}) {

We want to allow various tags to be able to merge their values, even between
provenences.  But for now i'm going to punt on that
in sub Kernel::__message_add_tags
            my $old_id=delete $message->{"TAGS"}{$name};
            if($old_id) {
                my $old_tag=$self->{TAGS}{$old_id};
                if($old_tag) {
                    $old_tag->{references}--;
                }
            }
        }
        $message->{"TAGS"}{$name}=$id;
        $tag->{references}++;
        $message->{$name}=$tag->{value};
    }
    $self->tag_GC;
}


Now the hoary functions.  

sub Kernel::__message_enqueue
{
    my($self, $message)=@_;

    unless($message->{called} eq 'call') {
        $self->SUPER::post($message->{session}, $message->{state}, $message);
        return;
    }
    return $self->SUPER::call($message->{session}, $message->{state}, $message);
}


sub Kernel::_dispatch_event
{
  my ($self)=shift;
  my ( $session, $source_session, $event, $type, $etc, $file, $line,
       $time, $seq 
     ) = @_;

There are many many ways for _dispatch_event to be called.  Make sure
That we only process ones that interest us.
in sub Kernel::_dispatch_event

    return $self->SUPER::_dispatch_event(@_) 
            unless ref($etc) eq 'Message';

    $self->__message_invoke_hook($message, 'before_invoke');
    my(@ret, $args);
    $args=$message->{args}||[];

We went to the trouble of keeping a eye on wantarray previously. This is so
that the code can be invoked as wanted.
in sub Kernel::_dispatch_event

    if($message->{wantarray}) {
        @ret=eval {
            $self->SUPER::call($session, $event, @$args);
        };
        $type=1;
    elsif(0==$message->{wantarray}) {
        @ret[0]=eval {
            $self->SUPER::call($session, $event, @$args);
        };
    }
    else {
        eval {
            $self->SUPER::call($session, $event, @$args);
        };
    }

The .... above is an eliding of all those paramaters that POE normaly adds
to @_;
in sub Kernel::_dispatch_event

    if($@) {
        $self->__message_invoke_hook($message, 'errors_to', [$@]);
    }
    elsif(defined $message->{wantarray}) {
        $self->__message_invoke_hook($message, 'respond_to', \@ret);
    }
    else {
        $self->__message_invoke_hook($message, 'respond_to');
    }
    
    $self->__message_invoke_hook($message, 'after_invoke', [$message]);
    return;
}


Nearly done!


sub Kernel::__message_invoke_hook
{
    my($self, $message, $hook, $args)=@_;

    foreach my $id (@{$message->{TAGS}}) {
        $self->tag_invoke_hook($id, $hook, $args);
    }
}


Right.  But we still haven't dealt with the following issue: what happens
when a colour dies.

sub Kernel::colour_died
{
    my($self, $colour, $message)=@_;


Get rid of all the tags associated with this colour.
in sub Kernel::colour_died

    while(my($name, $id)=each %{$message->{TAGS}}) {
        my $tag=$self->{TAGS}{$id};
        next unless $tag;
        $tag->{references}--;
    }
    $self->tag_GC;
}


TODO: 

- tag_GC
- Colours will have a "died" call-back associated with them.  This is how we
implmenent the Crossbar.
- Multicast Sessions
- Redefine Signals as a special case of Multicast!  :-)
- How are tags and colours going to work with IKC?

BTW, yes I am insane!