Queues should be used to set up long-term I/O links between ECLiPSe and Tcl. An example would be the main output from an application that is to be displayed by a Tcl window. Streams of bytes can be sent along the queue from one side to the other: on one side, data is written to the queue; and when the queue is flushed, the data is sent to the other side, which can now read the data. The data can either be sent as normal strings (where each byte represents a character) using the normal I/O calls, or they can be in EXDR format, in which case both sides need to read and write using EXDR.
On the Tcl side, the queue is seen as a Tcl I/O channel. On the ECLiPSe side, a queue is seen as an ECLiPSe I/O stream, which has a unique (numeric) ID, the stream number, and has a user supplied symbolic name. These all refer to the same queue. Queues are created using the symbolic names, and the Tcl side maintains tables of the correspondence between Tcl channel names, symbolic names and stream numbers. The built-in Tcl I/O commands accepts the Tcl channel name for operating on the queue, and for compatibility with the embedding interface, many of the Tcl remote interface commands refer to the queue using the stream number. The interface provides commands to inter-convert the various names so that the right name can be used for a particular command.
There are two types of queues:
These queues are designed to be compatible with the queues created via ec_queue_create of the embedded interface (see section 5.5). Their actual implementations are different, in that the queues in the embedded case are memory queues and the synchronous queue use socket connections. The interface tries to minimise the difference by buffering where possible at either ends of the socket connection. However, there is an overhead for doing this, and not all differences can be hidden. This is discussed in more detail in section 6.9.
They have no direct equivalent in the embedding Tcl interface, but some uses of the embedding Tcl interface queues, such as writing data from one side without a corresponding reader on the other side, are better approximated by the asynchronous queues than the synchronous queues. They can also be more efficient in that there is no buffering of the data is performed by the interface.
The processing of data on queues (synchronous and to some extent asynchronous) can be performed via handlers. A handler is a piece of code (a procedure in Tcl, a goal in ECLiPSe) whose execution is data-driven: it is invoked to deal with the transfer of data on a queue on their respective sides.
In ECLiPSe, the handler goal is invoked using the events mechanism. That is, an event is raised, and the event handler goal associated with the event (see set_event_handler/2) is then executed when ECLiPSe has control.
A handler can be called under two situations:
For each queue and for a particular direction of data flow, a handler can be defined on either the Tcl or the ECLiPSe side, but not both. The handler either consumes or provides data as described above. The reason that handlers cannot be defined on both sides is that this avoids possible infinite loop of alternately triggering the data provider and the data consumer.
These queues can be created on the Tcl side. This is done with the ec_queue_create command from within Tcl code:
{}).
It is strongly recommended that the queues should be used for long-term I/O connections between the two sides, and so the queues should not be created and closed on a short-term basis. For quick interchange of data, it is recommended that the user use the ec_rpc mechanism.
For a from-ECLiPSe queue, the Tcl handler command would be a data consumer. This handler is initiated when ECLiPSe side initially has control and flushes the queue (by calling flush/1). With a Tcl handler defined, control is transferred to the Tcl side, where command is invoked to consume the data. When the handler finishes, control is returned to the ECLiPSe side. The general sequence of actions are:
| ECLiPSe side | Tcl side |
| Writes to the from-ECLiPSe queue | |
| Flush the from-ECLiPSe queue | |
| Handler invoked to handle data on the from-ECLiPSe queue | |
| ECLiPSe returns from flush, and continue executing the following code |
The Tcl handler is specified by command in ec_queue_create. command includes the name of the Tcl procedure to invoke, and any user defined arguments. When the handler is invoked, two additional arguments are appended: the ECLiPSe stream number for the queue, and the number of bytes that has been sent on the queue. This command should read the data off the queue and process it. The following predefined Tcl data consumer handlers are provided:
To create the queue on the Tcl side with a Tcl handler:
Tcl code : ec_queue_create myqueue fromec {ec_stream_to_window_sync red textwin} {}
Note that the last {} specifies that there is no ECLiPSe
handler. This is the actual default for this argument, so it could be
missed out. After creating the queue, it can be used on the ECLiPSe
side. The programmer can write to the queue, and to send the data to the
Tcl side, the queue should be flushed:
ECLiPSe code : ... write(myqueue, hello), flush(myqueue), ...
When the queue is flushed as shown above, then control is handed over to
Tcl, and the Tcl handler, in this case ec_stream_to_window_sync,
would be invoked. This reads the data on the queue (hello, and anything
else that has been written since the last flush), and puts it into the text
widget textwin, with the tag red. The procedure is also called with the
ECLiPSe stream number for the queue and the number of bytes sent as extra
arguments. The textwin widget and the tag red must
be defined already in the Tcl program (presumably ‘red’ means printing the
text in red colour); if no tag is desired, {} can be used.
The procedure ec_stream_to_window_sync is predefined in the interface, but here is a slightly simplified version of it:
proc ec_stream_to_window_sync {Tag Window Stream Length} {
set channel [ec_streamnum_to_channel $Stream]
set data [read $channel $Length]
$Window insert end $data $Tag
$Window see end
}
Currently, the Tcl remote interface does not support ECLiPSe handlers (which will be a data provider) for from-ECLiPSe queues. Thus, the event argument for ec_queue_create is currently a dummy argument that is ignored. The available alternative is to use ec_rpc to obtain the required information: instead of reading from a from-ECLiPSequeue, an ec_rpc should be called with argument(s) left to be filled in by the ECLiPSe side with the required data.
For a to-ECLiPSe queue, the Tcl handler command defined in ec_queue_create would be a data producer. This handler is initiated when ECLiPSe side has control, and reads from the to-ECLiPSe queue, which is initially empty. With a Tcl handler defined, control is transferred to the Tcl side, where command is invoked to provide the data. The handler should write the data to the queue, and call the Tcl remote interface command ec_flush to send the data to ECLiPSe side. When the handler finishes, control is returned to the ECLiPSe side, and the read operation is performed to read the now available data. The general sequence of actions are:
| ECLiPSe side | Tcl side |
| Reads an empty to-ECLiPSe queue | |
| Handler invoked to supply data to the to-ECLiPSe queue. The data is written to the queue and flushed with ec_flush | |
| ECLiPSe returns from the initial read operation, reading the data supplied by the Tcl handler, and continue execution the following code |
The Tcl remote interface command ec_flush, instead of the standard Tcl flush command, should be used to flush a queue so that the data would be transferred and processed on the ECLiPSe side. ec_flush should be used both inside the Tcl data provider handler, and also to invoke an ECLiPSe data consumer handler (see the next section).
Normally, data is written to the queue using standard Tcl output commands, and the amount of data written is not known. However, the programmer may have kept track of the number of bytes written inside the handler, and thus know how many bytes will be sent. In this case, ec_flush can be called with the number of bytes supplied as a parameter. It is the programmer’s responsibility to ensure that this information is accurate. Without nbytes, the output is restricted to EXDR terms for synchronous queues. The reason for this is because the data is sent through a socket connection, and without knowing the amount of data, it is not possible in general to know when the data ends, unless the data sent has implicit boundaries, like an EXDR term.
For the use of ec_flush inside a Tcl data provider handler, the sequence of events that appears to the user is that the ec_flush flushes the data, and the Tcl side then continues executing Tcl code until the handler’s execution is finished. Control is then returned to ECLiPSe, where the original read operation can now read the available data. The actual sequence of event is slightly more complex for synchronous queues: when ec_flush is invoked, control is actually transferred to ECLiPSe, and the data flushed is then read into a buffer by ECLiPSe, which then returns control to Tcl to continue the execution of the handler. When the handler finally finishes, control returns to ECLiPSe, and the original read operation reads the data from the buffer and continues. This extra complexity should be transparent to the programmer except when the intermediate ECLiPSe read to buffer does not complete (e.g. because nbytes is greater than the actual amount of data sent).
The Tcl handler is specified by command in ec_queue_create. command includes the name of the Tcl procedure to invoke, and any user defined arguments. When the handler is invoked, an additional argument is appended: the ECLiPSe stream number for the queue. This command should get the data required, output it onto the queue, and call ec_flush to flush the data to ECLiPSe side. If the command does not flush data to ECLiPSe, ECLiPSe will print a warning and return control to Tcl side.
The following predefined Tcl data producer handler is provided:
To create the queue on the Tcl side with a Tcl-handler:
Tcl code :
ec_queue_create myqueue toec \
{ec_stream_input_popup "Input for myqueue:"} {}
This associates the pre-defined Tcl data producer handler ec_input_popup with myqueue. The last {} specifies that there is no
ECLiPSe handler and can be omitted as that is the default. This queue
can now be used on the ECLiPSe side in a demand driven way, i.e.
ECLiPSe side can read from the queue:
ECLiPSe code :
...
read(myqueue, Data),
...
When the ECLiPSe side reads from myqueue, and the queue contains no data on the ECLiPSe side, then control will be handed over to Tcl, and ec_input_popup invoked. This pops up a Tcl window, with the label “Input for myqueue:” with a text entry widget, asking the user to supply the requested data. The data is then sent back to the ECLiPSe side.
Here is a slightly simplified version (there are no buttons) of ec_stream_input_popup:
set ec_stream_input_string {}
proc ec_stream_input_popup {Msg Stream} {
global ec_stream_input_string
toplevel .ec_stream_input_box
label .ec_stream_input_box.prompt -width 40 -text $Msg
entry .ec_stream_input_box.input -bg white -width 40 \
-textvariable ec_stream_input_string
bind .ec_stream_input_box.input <Return> {destroy .ec_stream_input_box}
;# pack the popup window
pack .ec_stream_input_box.prompt -side top -fill x
pack .ec_stream_input_box.input -side top -fill x
tkwait window .ec_stream_input_box
puts -nonewline [ec_streamnum_to_channel $Stream] $ec_stream_input_string
;# flush the output to ECLiPSe with the length of the input
ec_flush $Stream [string length $ec_stream_input_string]
}
Data is flushed to the ECLiPSe side using ec_flush. The puts needs the Tcl channel name of the queue to write to, and this is provided via the Tcl remote interface command ec_streamnum_to_channel (see section 6.6.5). ec_flush is called with two arguments in this case, both the queue number (Stream), and the length of the data that is sent. Note that this makes the assumption that no other unflushed data has been written to the queue.
For a to-ECLiPSe queue, the ECLiPSe handler would be a data consumer. This handler is initiated when Tcl initially has control, and flushes data on a queue using ec_flush. Control is transferred to ECLiPSe, and if the ECLiPSe handler is defined, this is invoked to consume the data. When the handler returns, control is returned to Tcl, which continues executing the code after the flush. The general sequence of actions are:
| ECLiPSe side | Tcl side |
| Outputs data onto the to-ECLiPSe queue | |
| Calls ec_flush to send data to ECLiPSe side | |
| The ECLiPSe handler associated with the queue is called to consume and process the data | |
| Execution continues after the ec_flush |
The ECLiPSe handler is specified by the event argument of ec_queue_create. This specifies an event that will be raised on the ECLiPSe side when data is written to a previously empty queue. The ECLiPSe side does not see this data, and the event not raised, until the data is flushed by ec_flush and copied by ECLiPSe to its buffer and, if the buffer was initially empty, the event would then be raised.
The programmer should define the event handler associated with event.
To create the queue on the Tcl side with an ECLiPSe-handler:
Tcl code:
ec_queue_create myqueue toec {} remoteflush_myqueue
Note that the {} is needed to specify that there is no Tcl handler.
It defines remoteflush_myqueue as the event that will be raised when
the queue is flushed by ec_flush on the Tcl side.
The event handler needs to be defined on the ECLiPSe side:
ECLiPSe code:
:- set_event_handler(remoteflush_myqueue, read_myqueue/0).
...
read_myqueue :-
read_exdr(myqueue, Data),
process(Data).
This read handler assumes that the data is written using EXDR format. So on the Tcl side, the data should be written using EXDR format:
Tcl code:
...
ec_write_exdr [ec_streamname_to_channel myqueue] $data
ec_flush [ec_streamname_to_streamnum myqueue]
...
Asynchronous queues are created on the Tcl side using the Tcl command ec_async_queue_create:
Asynchronous queues are bi-directional queues which allows data transfer between ECLiPSe and Tcl sides without transfer of control. In the case where a Tcl data consumer handler is defined in fromec_command, which is invoked on the Tcl side when the queue is flushed on the ECLiPSe side, the ECLiPSe side will carry on execution while the handler is invoked on the Tcl side.
These queues are designed to allow for more efficient transfer of data between ECLiPSe and Tcl than the synchronous queues.
For data transfer from ECLiPSe to Tcl, the intended use is that a Tcl data consumer handler would be invoked as the data becomes available on the Tcl side, after being flushed from the ECLiPSe side. Note that control is not handed over to Tcl side in this case: the Tcl handler is invoked and executed on the Tcl side while ECLiPSe side still has control, with the restriction that the Tcl handler is unable to issue ec_rpc goals because ECLiPSe side still retains control. Another difference with the synchronous from-ECLiPSe queue is that the handler would read from the queue in non-blocking mode, i.e. it will read whatever data is available on the queue at the Tcl side and never wait for more data. If more data become available, the handler would be invoked again. The following Tcl handler is pre-defined for the asynchronous queue for handling from-ECLiPSe data:
For data transfer from Tcl to ECLiPSe, the queue can be used either asynchronously or synchronously. If the queue is used asynchronously, then the standard Tcl command flush should be used to flush the queue. There would not be any transfer of control, and so there would not be an immediate corresponding read on the ECLiPSe side. In fact, no handler would be invoked automatically on the ECLiPSe side, even when control is transferred. Output and flush operations do not block on the Tcl side, as the Tcl side of the queue is put into non-blocking mode, so that the data is buffered and the operations carried out when they will not block. It is the programmer’s responsibility to write and call the code to read the data from the queue on the ECLiPSe side when the ECLiPSe side is given control.
This asynchronous use to send data to ECLiPSe should be useful when the queue is used as an auxiliary data channel, where the main data is sent either via ec_rpc or another queue. The desired effect is that data can be sent on the auxiliary channel without triggering processing on the ECLiPSe side until it is told to do so on the main data channel, which would be handled synchronously.
To use the queue synchronously for to-ECLiPSe data, ec_flush should be used to flush the queue on the Tcl side. With ec_flush, control will be handed over to the ECLiPSe side to process the data: the goal associated with the event toec_event is executed, and this goal should read the data from the queue. Unlike the synchronous to-ECLiPSe queues, the data is not buffered, and the handler goal is called every time ec_flush is invoked, rather than only when the queue is empty. This should normally not make any difference, as the handler should empty all the contents of a queue each time it is invoked.
The goal is called with two optional arguments: the first argument is the
event name, the second argument is the ‘culprit’ of the form
rem_flushio(Queue,Len), indicating that this event is caused by a
remote flush, where Queue is the ECLiPSe stream number,
and Len is the number of bytes sent (this is supplied by ec_flush, if
ec_flush does not supply a length, then Len is the atom
unknown.
An example of using an asynchronous queue asynchronously to send data to ECLiPSe is in the tracer for TkECLiPSe development tools. Here the trace line is printed on a synchronous from-ECLiPSe queue, and handled by a Tcl data consumer handler which prints the trace line and waits for the user to type in a debugger command. This debugger command is sent to the ECLiPSe-side using an asynchronous queue, which is read by the ECLiPSe side when it returns. Here is a much simplified version of the code:
Tcl code:
...
ec_queue_create debug_traceline fromec handle_trace_line
ec_async_queue_create debug_input toec ;# no handlers
...
During the initialisation of the development tools, the Tcl code creates
the from-ECLiPSe queue where the trace-line information is sent
(debug_traceline), and the asynchronous queue (used only in a
to-ECLiPSe direction) for sending the debugger commands to ECLiPSe
(creep, leap, skip etc.). Note that as this queue is used asynchronously,
there are no handlers associated with it.
On the ECLiPSe side, when a goal with a spy-point is executed, this
raises an event that calls the predicate trace_line_handler/2 which
should output the trace-line, and wait for a debug command, process the
command, and carry on:
trace_line_handler(_, Current) :-
% Current contains information on the current execution state
% from this a trace line Traceline (a string) can be created
make_current_traceline(Current, Traceline),
% send the traceline to Tcl side
write_exdr(debug_traceline, Traceline),
flush(debug_traceline),
% flush will return when the Tcl handler has finished
read_exdr(debug_input, Cmd),
% read the command from debug_input and process it
interpret_command(Cmd, Current).
The trace-line handler is called with the second argument set to a
structure that contain information on the current execution state
(Current), from this, a trace-line (the debug port name, depth, goal
being traced etc.) can be constructed: Traceline is the string that
should be printed, e.g.
(1) 1 CALL append([1, 2, 3], [], L)
This is sent as an EXDR term to the Tcl side using the synchronous
queue debug_traceline. When flush/1 is
called, control is handed over to the Tcl to handle the data, and the
Tcl data consumer handler handle_trace_line is invoked:
proc handle_trace_line {stream length} {
global tkecl
$ec_tracer.trace.text insert end \
[ec_read_exdr [ec_streamnum_to_channel $stream]]
configure_tracer_buttons active
;# wait for a tracer command button to be pressed...
tkwait variable tkecl(tracercommand)
configure_tracer_buttons disabled
ec_write_exdr [ec_streamname_to_channel debug_input] \
$tkecl(tracercommand)
flush [ec_streamname_to_channel debug_input]
}
As this is invoked as a handler, the ECLiPSe stream number (stream) and number of bytes sent (length) are appended as
arguments. Note that as the trace-line is written as an EXDR term, the length information is actually not needed. What the handler does is
simply read the trace-line as an EXDR term, and placing the resulting
string onto the tracer text window $ec_tracer_trace.text. Next, configure_tracer_buttons active is called. This code is not shown, but
what it does is to enable the buttons for the debugger commands so that the
user can press them. There are buttons for the debugger commands such as
‘leap’, ‘creep’ etc. When one of this button is pressed, the global
variable tkecl(tracercommand) is set to the corresponding command,
and the handler continues its execution beyond the tkwait. The
buttons are disabled, the command sent to ECLiPSe side on the
debug_input queue using flush. This is the asynchronous
sending of data on the asynchronous queue:
control is not handed
over to ECLiPSe to process this command. Instead, the execution on the
Tcl side carries on (and happens to finish immediately after the flush. Control is then returned to the ECLiPSe side as the Tcl handler
has finished, and the ECLiPSe side continues execution after the
flush(debug_traceline) goal. Next, debug_input is read for
the tracer command, and this command is acted on.
If the Tcl remote interface command ec_stream_input_popup (see section 6.6.2) is used to send data to the ECLiPSe-side (in section 6.6.2, the command was initiated by a read operation on the ECLiPSe side; here the command is invoked directly when Tcl side has control), then the following is a possible ECLiPSe handler:
Tcl code:
;# create the asynchronous queue, with
;# from-ECLiPSe Tcl consumer handler: data_to_window
;# to-ECLiPSe ECLiPSe handler event: flush_myqueue
ec_async_queue_create myqueue {data_to_window textwin} flush_myqueue
...
;# get input for the queue and send to ECLiPSe side
ec_stream_input_popup "Data:" [ec_channel_to_streamnum myqueue]
...
ECLiPSe code:
:- set_event_handler(flush_myqueue, read_remote_data/2).
% Len is known when ec_stream_input_popup is used to send data
read_remote_data(_Event, rem_flushio(Queue,Len)) :-
read_string(Queue, "", Len, Data),
process(Data).
The ECLiPSe code defines read_remote_data/2 as the handler for
to-ECLiPSe data sent with ec_flush on the Tcl side. This handler
is called when control is handed over to ECLiPSe side to read the
data. Both the two optional arguments are used in this handler. The second
argument supplies the ECLiPSe stream number for the queue and the length
of data written. As the data is
sent by explicitly calling ec_stream_input_popup, the length of the
data sent is known, so read_string/4 can be used to read the exact amount of
data. In the asynchronous queue, it is generally the programmer’s
responsibility to ensure that the read will not block.
The example ec_async_queue_create also defines a Tcl data consumer
handler to handle data sent on the from-ECLiPSe direction, with a user
defined argument of the text window that the data will be sent to. Here is
a simple procedure which reads the data on the queue and places it on the
text window specified:
Tcl code:
proc data_to_window {Window Stream} {
set channel [ec_streamnum_to_channel $Stream]
$Window insert end [read $channel]
}
The Stream argument is appended by the interface when the handler is invoked, and is the ECLiPSe stream number of the queue. The procedure simply reads the data from the corresponding Tcl channel and display the data on Window, the text window specified by the programmer.
ECLiPSe stream names are global in scope, so using fixed queues names like ‘myqueue’ might cause name conflicts with other modules, if the programmer intend the remote Tcl code to be usable with other ECLiPSe code. One way to avoid name clashes is to dynamically composing queue names using the name of the control connection:
Tcl code:
append queue_name [ec_control_name] myqueue
ec_queue_create $queue_name fromec {ec_stream_output_popup red textwin}
The user specified name ‘myqueue’ is appended to the control name of the remote connection to give a unique queue name. On the ECLiPSe side, the code will also need to use the dynamic name:
:- local variable(remote_control).
...
% code fragment to remember the control name
remote_connect(Addr, Control, _),
setval(remote_control, Control),
...
...
% code fragment to use the queue
getval(remote_control, Control),
concat_atom([Control, myqueue], QName),
...
write(QName, hello), flush(QName),
...
The remote queues connecting ECLiPSe and Tcl are given different names on the two sides. The remote Tcl interface keeps track of the ECLiPSe names for the queues on the Tcl side. On the ECLiPSe side, the queue has a stream number, as well as possibly several symbolic aliases. The interface only keeps track of one symbolic name – the one that is supplied in ec_queue_connect and ec_async_queue_create. If the ECLiPSe stream number was supplied in these commands, then the stream number is also considered the symbolic name for the queue as well. The Tcl interface provides several commands to convert the names from one form to another: