Newton 2.0 Communications
1.x to 2.0 Communications Overview

Overview
Routing
Transports
Endpoints
DILs
CDIL
FDIL
References
Where to Go From Here...

This article describes the differences between communications on a 1.x Newton and a 2.0 Newton device. It covers new features as well as revisions to the old system. While 2.0 communications is backward compatible with 1.x communications (if the 1.x programmer followed the documentation), there are enough improvements both to the feature set and the performance of the 2.0 communications that it is likely that most programmers will want to upgrade to 2.0. If you have never done Newton Communications programming you may want to start with the article 2.0 Communications Overview.

If you have no prior experience in Newton communications programming you may want to start with the Communications Overview article found elsewhere in this module. If you have no Newton programming experience, you may want to take the Newton Programming: Essentials 2.0 course available from Apple's Developer University.

The goal of this article is to provide a quick summary of the changes to communications in 2.0 and to provide references to specific places within the 2.0 Communications course and elsewhere to get further information and code examples. After reading this article it is hoped that a programmer will have an idea of what changes he or she will want to make to the communications sections of their existing Newton programs and will know where to find the information necessary to make these changes.
Throughout this and other modules in this class the Newton Programmer's Guide is abbreviated as NPG. In 2.0 the Guide has been broken into two pieces: NPG: System and NPG: Communications.
Figure 1 shows the different pieces of the Newton communications system and the programming interfaces used to access these pieces. The rest of this article gives brief descriptions of these APIs and how they have changed from 1.x to 2.0 as well as references to more details about them.

Figure 1 is the "road map" for this class. Part or all of it will show up repeatedly to show what part of the comms system is being discussed. You may also want to see the QuickTime animation based on this figure which has a brief explanation of what happens when sending or receiving.

The following is a brief description of the items shown in Figure 1 and their APIs.
A NewtonScript App using the Routing API is the simplest ways for application programmers to provide communications support from the Newton. Any application written in NewtonScript can use communications modules which have been installed as Transports. On 2.0 Newtons this will include the built-in transports for beaming, faxing, mailing and printing. To use the available transports the Routing application programming interface (API) is used to specify what data is being "routed", what form it takes and how it should appear (e.g., print format). The Newton uses a store-and-forward model for this kind of communications and the In/Out boxes arewhere incoming or outgoing data is stored in the routing model.
In/Out Box Application and Transport API. Built into the system is the In/Out Box Application which manages the soups used to store incoming and outgoing data. It communicates with one of several Transports, which are code to move the data to or from the appropriate destination or source. While there are several built-in transports in the system, NewtonScript programmers may write their own transport to provide system-wide data management.
Endpoint API and Endpoint System. The Endpoint API is a NewtonScript interface for doing direct communications with the outside world. Applications programmers may add endpoint code to their programs to communicate directly with an external source or destination. An example of this might be point code which communicates directly with a GPS device on demand.Transports have endpoint code to move the data it receives out to an external destination or to receive data from an external source prior to passing it to the In/Out Box Application.
Low-level Communication Tools are implemented in C++ and actual communicate with the C++ interfaces to the hardware drivers. While these interfaces are not yet available, they will be published in the future.
Each of the APIs will be described in more detail.

Routing is described in detail in Chapter 2 of Newton Programmer's Guide: Communications. It is also covered in the course Newton Programming: Essentials 2.0.

Starting at the highest level of NewtonScript communications, routing has been changed to be more powerful and flexible for applications programmers. Some of the changes are transparent to the programmer, others are to 2.0 specific and programmers will have to make changes to their existing applications to take advantage of the enhancements.
Printing and non-communications routing are described in detail in the course Newton Programming: 2.0 Essentials.

As with other parts of the communications system, most of the changes to routing are backward compatible, that is, 1.x routing will not generally break though users will come to expect the 2.0 features to be available in all applications. Non-communications aspects of routing (such as printing formats) will not be covered in this article.

One of the biggest changes to routing is that the application no longer needs to specify the communications transports within the application. In 1.x, the application programmer had to define a routing frame with pointers to routing formats he or she wanted to display in the Action Button popup. This frame would be put in the base view of the application and might look something like this:

myRoutingFrame := {

// here is the printing stuff
print: {
title: "Print Thing",
routeForm: 'printSlip,
formats: [kPrintFormat, kOtherPrintFormat]},

// and here is the fax stuff
fax: {
title: "Fax",
routeForm: 'faxSlip,
formats: [kPrintFormat, kOtherPrintFormat]},
// now mail
mail: {
title: "Mail",
routeForm: 'mailSlip,
formats: [kPrintFormat, kOtherPrintFormat]},

// and finally IR
zap: {
title: "Beam",
routeForm: 'zapSlip},
}

Transports are separate code packages that move routed data. The next section is an overview of Transports.

The drawback of this approach is that the only transports (i.e., ways of moving data) allowed are the ones specified in this frame (printing, faxing, mailing, beaming). If the owner of a Newton adds a new transport (such are point-to-point) to the system, the application must be rewritten to reflect this change. In addition, the routing frame provided in the 1.x system is global to the entire application. Unless the application deregisters the old routing frame and registers a new one, the user is stuck with the same list of options regardless of how appropriate they are at any given time.
Registering View Definitions is described in Chapter 5 of Newton Programmer's Guide: System Software 2.0
In 2.0 this is not the case. Instead, the application programmer specifies one or more routing format frames and registers them with the View Definition Registry. These formats describe what kind of information the application can route and how it is displayed. Based on these factors appropriate transports are automatically shown in the Action Button pop-up when the user taps on it. Thus if a new transport is installed which can transport frames, any application which can route frames and has registered this will automatically display the new transport in the Action Button. The relationship of the data being routed, the routing formats and the installed transports is shown in Figure 2.

Note that the names of these transports are arbitrarily chosen. In particular, you should not rely on these names for the system installed transports such as printTransport, faxTransport, etc.

The application shown in Figure 2 has registered a routing frame using its app symbol: |forms:MYSIG|. When the target slot is set to the frame shown at the top of the figure, the routing system automatically checks for a routing frame with the same name as the class slot in the target frame. Finding the previously installed frame, it sees that the application can route frames, text and views and so adds the print, fax, beam and mail transports to the list which shows up when the user taps the Action Button. Since the application did not register as supporting the routing of binary data, the compress transport will not appear.

In addition to the dynamic selection of transports, 2.0 routing provides a mechanism context sensitive routing within a program. In 1.x everything revolved around the routing frame which was registered at install time and was global to the application. In particular this meant that the application could not easily change the availability of delete and dispose or other routing actions which are normally implemented as scripts.

In 2.0 the routing popup is built by sending a GetRouteScripts message to the protoActionButton view. This method looks for the slot RouteScripts which should be an array of routing scripts. Using the normal rules of inheritance if the proto of an Action Button has a RouteScripts slot, the list of actions in the slot is displayed. If none is found in the proto chain (it won't unless the programmer overrides the system proto), the system continues looking for routing scripts in the parent or any ancestor view including the base view of the app. Knowing this, a programmer may have "default" RouteScripts to do something (say, Duplicate and Delete) at the base level but may at various times replace these with other, view specific routing actions.
An additional improvement is that the same inheritance scheme is used for determining the target and targetView slots. As with the search for routeScripts, the system begins looking for these slots in the action button view and goes on up the inheritance chain.

Because of the need for target to be cloned, it is expected that GetTargetInfo will be implemented in most applications. Otherwise the application will need to update the target slot every time the target data changes. GetTargetInfo() is described in detail in NPG: Communications, on pp. 2-52.

In addition, instead of simply looking for these slots the system sends the GetTargetInfo method to the action button view. By default this method explicitly looks for the target and targetView slots but the application may override the method and dynamically provide these slots to the system. By the same logic that meant that routing scripts may be specific to a view, target and targetView may differ depending on the view containing the Action Button. Also, since the GetTargetInfo method provides a dynamic way to determine the current target and since it is necessary to clone the contents of these slots before they are routed, by using GetTargetInfo it is not necessary to update the slots as the content of the target changes.
Note that this is an entirely fictitious example which is meant to demonstrate how routing is done and is not typical or preferable as the structure for an application

Figure 4 shows a possible situation in 2.0 where depending on the Action Button the user taps the target and targetView slots used for routing and the routing scripts displayed in the Action Button will all be different. If the user taps on the Action Button in baseview or in view2, the routing scripts will be displayed from the routeScripts array in baseview. If the view1 action button is tapped, the scripts in the view1 routeScripts slot will appear. Similarly, if routing is done from view1 or view2, the target and targetView slots in these views will be used while if we route from the baseview it will determine the target and target view will be determined "on the fly" by calling it's GetTargetInfo() method.

Other routing enhancements include the ability to route multiple items by using a cursor or a mock cursor to specify the data being routed, bypassing the routing interface by using the Send() function to send data directly to a transport, sending and receiving control messages between an application and a transport, and the ability to send and receive binary data. All of these are beyond the scope of this article but are described in the 2.0 communications and system documentation.

Routing multiple items is described in the NPG: Communications on pp. 2-16. Mock cursors are described in NPG: System on pp. 11-48. The Send function is described in NPG: Communications starting on pp. 2-49.

Transports are described in Chapter 3 of NPG: Communications. The Archive Transport example code written by J. Christopher Bell is a good introduction to writing Transports.

Transports are 2.0 specific programs which move data to or from a destination or source. When an application sends data out by routing it is placed in the Out Box. When this occurs the selected transport is notified and, at the appropriate time, the transport sends the data to the destination. Conversely, when the user taps the Receive button in the In Box and selects a transport, the transport attempts to connect and receive data from a source. If this transfer of data is successful, the data is returned to the In Box where it is sent to the appropriate application at the appropriate time.
In 1.x there was an API which was seeded on a limited basis called Categories. This API is entirely replaced by Transports in 2.0. Categories are not compatible with 2.0.

In 1.x systems there was no way to write this kind of program. The built-in transports of beaming, faxing, mailing and printing were all that was normally available. In 2.0 there is now an API for adding custom designed transports to the built-in ones.
The simplest definition of a transport is something that can be routed to. But a more useful definition is that a transport is a globally available service offered to applications for sending or receiving data. Because of the global nature of transports, it is not necessary, or even likely, for an individual application to define a transport.

The built-in transports include printing, faxing, mailing, and beaming, but one might imagine additional transports such as messaging, scanning, compressing, archiving, or encrypting. Thus, while transports are usually associated with hardware (printers, mail servers, and scanners, for example) this is not necessarily the case (e.g., compressing, archiving, encrypting), since a service may be offered that alters the data being routed without sending it to any outside hardware.
In 1.x there was an API which was seeded on a limited basis called Categories. This API is entirely replaced by Transports in 2.0. Categories are not compatible with 2.0.

Transports are usually built as auto-load parts; they appear in the Extensions folder of the Extras Drawer. A transport's InstallScript registers it with the system by calling the global function RegTransport(). As described in the routing section above, if appropriate target data is routed, the transport's name will appear in the action list in an application when the user taps the Action Button.
As most transports have actual communication code which will be used to send or receive the target data, they will typically have endpoint code which communicates with the source/destination.
Transports usually work with an application through the In/Out Box application. Figure 5 shows the interactions between the NewtonScript application, the In/Out Box and a transport during a send request.

In this figure a transport receives a SendRequest whenever an item is routed to the Out Box. The request sent to the transport has a cause slot which describes why the transport was notified. In the case of target data which is held in the Out box the cause slot will have a value of 'submit which indicates that it is posted but should not be sent yet. When a user opens the Out Box and selects one or more items to be sent, the transport receives a SendRequest message with the cause of 'user. In this case the transport should actually send the data.

When sending items, the user may specify a preference that all items be sent as soon as they are received in the Out Box. In this case the cause slot will be set to a value of 'item before the SendRequest is sent to the transport and the transport should send the item immediately.
If the cause is either 'user or 'item in a SendRequest, the transport attempts to send the data to the destination. Once sent (or if sending fails) the transport notifies the Out Box that the item has been sent by calling ItemCompleted and then calls ItemRequest to see if there are any further items posted in the Out Box. This last may occur if the user selected multiple items in the Out Box and then sent them to the transport all at once.

In the case of a request to receive data things are a little different. Figure 6 shows the usual sequence of events when receiving data. When the user selects a transport from the list shown in the Receive button pop-up in the In Box, the selected transport is sent a ReceiveRequest message. The transport will then try to connect to the remote source and get any pending data. It will return data received (or status if there was no data or the attempt failed) to the In Box application by calling the method ItemCompleted(). The In Box will then add the item(s) to its soup and display the newly received item(s) in the In Box when it is next opened.

In a slightly more complex situation, the transport may simply get a description of what is available at the source (for example the title of an email message) and post it to the In Box. In this case the transport adds a remote slot to the request and set it to true. This flags the item so that the In Box knows that the body of the data has not been received.

At a later date the user may select the item from the In Box and ask to see it at which point the In Box will send the transport another ReceiveRequest message but with the cause slot set to 'remote. The transport will then be responsible for getting the body of the data so the In Box can display it.

As with the case of sending a message the transport should then check for other pending receive requests by calling ItemRequest.
Note that ItemRequest is defined in the protoTransport prototype and so there is the somewhat unusual situation where the transport is sending itself a message for the purpose of getting information from the system, in this case the next item to be transported.
As mentioned above the main proto used to create a transport is protoTransport. The powerful thing about protoTransport is that in many cases surprisingly little code other than the actual endpoint code must be written. This is because the defaults typically "do the right thing" to provide an interface and default behavior for the transport. Only those features specific to the transport (e.g., archive name for an archive transport) must be added to the standard interface.
The DTS sample Archive Transport shows a minimally implemented transport that provides a global, frame-based transport for archiving frame information in memory in an "archive soup." A modified version of the Archive Transport is used in the DIL module of this course to provide the Newton side of the desktop-Newton communications
Transports which can send data also have their own routing slips based on the prototype protoFullRouteSlip. This allows users to provide transport specific options such as adresses in a particular format, etc.

In particular, such things as displaying the status of a routing request, logging of routed items, error handling, power-off handling, and general user interfaces are handled well by the defaults if the transport simply sets or updates a few slots when appropriate. Only the actual service code (such as communications) will be different between transports.

Endpoints have changed extensively between Newton 1.x and 2.0, almost entirely for the better. Much better. To begin with they are much more robust. Known bugs in ADSP, IR, and general endpoints have been fixed and the addition of many new features have made much of the hack code that was created to program around endpoint limitations unnecessary. To do all of this a new prototype, protoBasicEndpoint has been introduced. This is now the proto of choice for all endpoints though for backward compatibility protoEndpoint is still supported.
Note: while code using protoEndpoint and compiled for the MessagePad platform (i.e., any pre-2.0 OS version) will continue to work, such code will not compile using the 2.0 platform file in NTK. In other words, the symbolic reference to protoEndpoint has been removed from the 2.0 platform definition.
A quick look at the list shown in Figure 7 shows how extensively protoBasicEndpoint has improved the endpoint interface. These items will be discussed one by one in the remainder of this article.

Asynchronous calls
Improved Input Specs
Introduction of OutputSpec
Binary data support
Data Templates
Multiple active endpoints

Figure 7: Changes to Endpoints

Asynchronous Calls

The first and most noticeable change is the addition of asynchronous calls for most endpoint methods. For example, to connect an endpoint, the asynchronous call shown in Figure 8 avoids the previous awkwardness of installing DeferredAction tasks to ensure connection without interrupting the main thread of the app.

:Connect( nil,
{async: true,
reqTimeout: 45000,// 45 seconds -- for DEMO purposes
completionScript: func(ep, options, result)
ep:MConnectCompProc(options, result), });

Callback spec frames are described in NPG: Communications, pp. 4-33 to 4-36.
In this code, because the async slot in the callback spec frame (the second argument to the Connect method) is set to true the call will be executed asynchronously. The CompletionScript slot defines a NewtonScript type closure which will be called when the connect has completed. The closure will be passed a reference to the endpoint, any options returned by the original call (i.e., any messages returned from the low-level tool such as the actual connection speed negotiated), and the result of the attempt to connect. If the endpoint frame has been created with a _parent slot which refers an appropriate application view (such as the base view of the application) then the completion script will be able to access any application slot or method accessible to the base view.
Note that while the calls are asynchronous, there is still only one thread for all NewtonScript code. However, it is truly asynchronous relative to the low-level communication tool code.
While there are a finite number of asynchronous calls which may be queued up, the queue is large enough for most normal operations. Obviously asynchronous calls lend themselves to call chaining where the completion of one call, say an asynchronous Output call, will cause another Output call to be made from the completion script.
The Abort method is now obsolete. Cancel completely replaces it.
Another improvement related to asynchronous calls is the ability to cancel a communication request. To begin with a request can be set up with a timer to cancel it after the passage of a certain amount of time (45000 msecs in the example above). In addition there is now a Cancel method which kills all calls currently executing and return to their completion scripts (if any) with a result code of -16005.

Queued but not currently executing calls will receive a -36003 error instead after a Cancel message has been sent. As an added bonus, the Disconnect method acts as a cancel and all outstanding requests will be terminated just as gracefully thus eliminating the need for DelayedAction calls to guarantee completion of outstanding requests.
Cancel will also work on synchronous calls which are pending and Cancel itself may be called asynchronously.

Input Specs are described in NPG: Communications, pp. 4-35 to 4-41. The DTS sample, Basic Modem shows the basic structure and use of input spec frames.

Improved InputSpecs

InputSpecs have also been substantially improved over the 1.x system. In particular, in 1.x the programmer had a choice of triggering an InputSpec based on the receipt of a single character, on the arrival of a certain number of bytes or after the passage of so many milliseconds. While all of these are still possible, they have been extended and improved substantially.
To begin with, instead of accepting only a single character as a trigger, in protoBasicEndpoint there is a slot called endSequence which allows the programmer to specify a string for the input trigger. In other words, instead of looking for a single character like carriage return and then checking the characters which proceeded it to see if an expected command was received, we can set an endSequence of something like the following string: "\nBYE!\n" and the input spec will be triggered only when the string is received.
To make things better, endSequence can be an array of possible strings so the programmer could list all the command strings he or she was looking for such as the following list:

endSequence:=["\nBYE!\n","\nHELLO\n","LOGIN:\n",...];

Figure 9 shows a simple example of an InputSpec frame having multiple endSequence strings. Note that the InputScript method is passed a termination frame which includes the cause of the trigger as well as the number of bytes received.

anInputSpec: {
form: 'string,
termination: { endSequence: [unicodeCR,"\nBYE!\n"], },
discardAfter: 512, // which is our default buffer size
inputScript: func(ep, data, terminator, options)
begin
if terminator.index = 1 then // 1=cmd "BYE!"
:MDisconnect();
else
ep:MInput(data,terminator.byteCount);
end,
}


Sorry, Endpoint module not available yet!
But beyond looking for strings on input, protoBasicEndpoint recognizes several different types of incoming data including single characters, 30-bit integers, byte arrays, binary data, NewtonScript frames and templates which map C-structure type data into NewtonScript data types. A more detailed look at the data forms which can be accepted for input follows later in this article and details may be found in the Endpoint module of this class.
Binary data input is described in NPG: Communications in the section on protoStreamingEndpoint (pp. 4-26 to 4-27) and the section Working with Binary Data (pp. 4-27 to 4-28). Binary input and output is briefly discussed below.
Perhaps the most intriguing addition in 2.0 is the binary data option where (large) chunks of may be imported directly into a soup store without using any heap space. This makes it possible to read and write large memory objects on a machine with a limited heap memory.
Note that the slot nextInputSpec is no longer used in 2.0.
The activation and deactivation of an input spec in 2.0 is different as well. Unlike the 1.x system where you must reset an input spec after input is received, an input spec remains active until it is changed. To terminate the active input spec, SetInputSpec is called with a nil input spec frame.

Output Specs

Also new in 2.0 is the ability to control more closely how data is output through the use of an output spec. An output spec is a special kind of callback frame. A callback frame is a frame which controls how a call is made, how long before it times out (if ever) and it contains a callback script for asynchronous calls.
An example of a callback frame from the Connect call shown in Figure 10 is shown below:

{ async: true,
reqTimeout: 45000,// 45 seconds -- for DEMO purposes
CompletionScript: func(ep, options, result)
ep:MConnectCompProc(options, result),
}

Figure 10: A Sample Callback Frame

In this case the callback frame is a sub-type called a completion frame. Because the async slot is true, the call will be made asynchronously and the CompletionScript will be called when the Connect call is completed. If async is nil the CompletionScript is ignored and the call is made synchronously. The reqTimeout slot says how long in milliseconds to wait before canceling the call. Unlike the CompletionScript, the reqTimeout has an effect even if the call is made synchronously.
Forms are detailed in NPG: Communications, pp. 4-6 to 4-7.
An output spec frame is a particular kind of completion frame used in Output calls. It has the same slots as a completion frame but has additional slots which control data transfer during the output. The most important of these slots is the form slot which is a symbol which describes the handling of data being sent out. The form slot may have the values shown in Figure 11:

'string // output as string with unicode->ascii conversion
'char // output single byte with unicode->ascii conversion
'bytes // output as an array of bytes, truncate if necessary
'binary // output binary data without conversion
'number // output a (30-bit) integer
'frame // output a flattened NewtonScript frame
'template // output using template description

Figure 11: Form Slot Values

Since the output spec can specify 'frame as an output form, all output is now done using the method Output. OutputFrame is obsolete and not implemented in protoBasicEndpoint.

Note that since NewtonScript objects have type information associated with them, if the form slot is omitted the endpoint will apply a default form for when outputting the object.
But beware specialized frames! If a frame has a class slot, its type will be the value of that slot, not 'frame as you might expect.
The other slots in an output spec include the target frame which is used for keeping track of how much of a binary object has been output and a flags slot which is used for high-level protocol (as opposed to media specific protocols such as AppleTalk) packetized data.
This completely replaces the obsolete rcvFlags and recvFlags slots.
An example of using the flags slot is to denote the start and end of a transaction which must be sent in several pieces. As the transaction is sent, the flags slot will be set to kPacket+kMore indicating that the data is part of a packet (or group) and that there is more to come. In the last Output call of the transaction, flags is set to kPacket+kEOP.
Figure 12 shows an example output spec.
{ async: true,
reqTimeout: kNoTimeout, // which is the default
CompletionScript: func(ep, options, result)
ep:MSendNext(options, result),
flags: kPacket+kMore, // more is coming
form: 'string, } // target not used - form not 'binary

Binary Data Support

Binary data is described in NPG: Communications in the section on protoStreamingEndpoint (pp. 4-26 to 4-27) and the section Working with Binary Data (pp. 4-27 to 4-28). Receiving binary data is described on pp. 4-19.
One of the most exciting additions to communications in the 2.0 system is support for binary data transfers. These may be any stream of bytes which the programmers wishes to import or export unchanged. Examples of the kinds of things which might be sent include pictures, sounds, and Newton packages.
To send a whole binary object you simply need to set the form slot of the output spec to 'binary and the endpoint interface will pass the data on directly to the low-level communication tool without any changes or filtering.
Additionally the programmer may want to send only part of a binary object at a time. In this case add the target slot to the output spec. Target is a frame with the following structure:

target:{ offset: offsetToCurrentPlaceInData,
length: numberOfBytesToSend, }

This sends length bytes from offset bytes into the data argument passed to the Output method. In other words, if length was 512 and offset was 4096, then the Output method would send 512 bytes from the start of the binary object passed into the Output method starting 4096 bytes into the object. This capability is useful if you want to break up transmission of large chunks of data.
Oops! At the time of writing, the target frames for binary object transfers does not work. Sorry!
Binary data may be received by setting the form slot to be 'binary and the target slot of an inputSpec to be a frame with a data slot which refers to a place where the incoming data will be put and optional offset slot which says where within the binary object you wish to put the latest chunk of data. This allows the inputSpec to take large objects a piece at a time.
An inputSpec target frame might look like this:

target:{ data: myBObject,
offset: offsetToCurrentPlaceInData,}

Binary objects are described in The NewtonScript Programming Language. VBOs are described in Newton Programmer's Guide: Communications, pp. 11-87 to 11-91.
Note that the object referred to by the data slot must be pre-allocated as the endpoint interface will not resize the object based on the data received. It may be a string object, a binary object (both of which live in the Newton frames heap) or a virtual binary object (VBO) which means that the object will actually be written out to a store.
Translate is described in NPG: Communications, pp.4-55 to 4-56
2.0 also makes it possible to pre-flatten a frame into a binary object before sending it (which improves the speed of sending) and then to unflatten it on receipt. This is done using the utility function Translate() which flattens or unflattens a frame before it is sent or after it is received. In the future there may be more options for this function but currently it is only used for flattening and unflattening frames.

Data Templates

In addition to the ability to transfer data in several pre-determined forms, 2.0 allows mapping incoming and outgoing data into a template. This is convenient for sending NewtonScript data to C structures or receiving C structure data and putting it into NewtonScript objects.
The pre-defined configOptions used in the 1.x system are completely replaced by the data template system of option specification. This gives Apple greater freedom to change option specification should it become necessary.
Basically a template is a frame with two slots: typelist and arglist. The typelist is an array which describes the kind of data in each slot in the arglist while the arglist is the actual data itself. Templates are used now to specify the data in endpoint options. An example of an endpoint option template is shown below in Figure 13.
local option := {
type: 'option,
label: kCMOSerialIOParms,
opCode: opSetRequired,
form: 'template, // not needed
result: nil, // not needed; returned
data : {
arglist: [
k1StopBits, // 1 stop bit
kNoParity, // no parity bit
k8DataBits, // 8 data bits
k57600bps, // date rate 57600 bps
],
typelist: [
'struct,
'long, // stop bits
'long, // parity
'long, // data bits
'long, // bps
]
}
};

Here the list of IOParms options are given in the data slot template. The typelist uses pre-defined data symbols to describe the data kept in the arglist. The first element in the typelist is a 'struct symbol which tells the system that the data in the arglist is put into a structure. The other arguments in the typelist describe the format of elements in the array. In this case four long values are the types of the data options.
Template data forms are described in NPG: Communications, pp. 4-8 to 4-10.
Templates may also be used to describe data being sent or received. In the case of receiving data, a target slot is added to the inputSpec. The target slot is a template with the arglist being a series of references to NewtonScript objects that the incoming data is mapped into. Note that in this case InputScript is called when enough data has been received to fill all the arglist slots in the template.
For outgoing data, if the form slot in the output spec is set to 'template, then the data slot is expected to be a template. Figure 14 shows a template which might be used to send data to a C structure. Note the definition of the char arrays in the typelist. This format essentially says "send until you reach the end of the string." If a non-zero value had been used (e.g., ['array,'char,4]) the endpoint interface would have sent a character array of 4 characters.

Beware of long-word alignment! There is null padding to fill out values to even long word boundaries. This means the second string will have 3 trailing null bytes to fill out the word.
outspec.data := {
arglist: [
"Silverwing",
"Honda",
500,
1982, ],
typelist: [ 'struct,
['array, 'char, 0],
['array, 'char, 0],
'ulong,
'ulong, ]
};

Multiple Endpoints

Another great improvement in 2.0 is the ability to open more than one endpoint at a time. For example, a serial endpoint could be open to the inspector at the same time an infrared endpoint is open to beam information between Newtons.
The port location option is described in NPG: Communications pp. 5-4 to 5-5.
Of course it is still not possible to have two serial endpoints open through the same hardware port but there is now a serial endpoint option available for specifying what port will be used for a serial or modem endpoint. This would make it possible to open an endpoint to a serial PCMCIA card so that data could be sent out through the card while the inspector was connected through the built-in port - which is a very useful for debugging.
Other
This article is by no means a complete list of all of the differences between 1.x and 2.0 endpoints, it merely highlights some of the most important differences. Other improvements include a richer set of endpoint options, better communications with low-level communication tools, filtering options both for incoming and outgoing data and new endpoint types for task-specific endpoints such as stream based communications and telecommunications uses.

DILs (Desktop Integration Libraries)

At the time of this article, DILs, or Desktop Integration Libraries, are being used in Beta form on 1.x systems and so are backward compatible. However in this article DILs will be discussed as if they are part of the 2.0 release.
The primary documentation for DILs is found in the document: Newton Desktop Integration Library which accompanies the libraries. There is also a DILs module in this course. J. Christopher Bell has also written an excellent article: Welcome to the Desktop Integration Libraries.
An important addition to Newton communications programming is a set of libraries called the Desktop Integration Libraries or DILs. The first and most important thing to know about DILs is that they have nothing to do with programming communications on the Newton. Instead they are used to create a communications link with Newtons from a desktop machine. In other words, they are an aid for writing code on a desktop machine which will communicate with a Newton.
Figure 15 shows this relationship. Essentially, endpoint code on the Newton, embedded in an application or resident in a transport, sends data from the Newton to a desktop machine. On the desktop machine an application built using one or more DILs sends and receives data which is then processed on the desktop machine. Currently DILs are only available for the Macintosh OS and for Windows machines.

The main advantage of DILs is that they make it easier to write desktop machine code to communicate with the Newton. In particular, they abstract the connection to the Newton to a virtual pipe for bytes and they provide control over such things as ASCII-to-Unicode conversions and Newton data structures and types such as frames and 30-bit integers (the default integer size on the Newton).
As shown in Figure 16 there are three DILs which build on top of one another: the CDIL, FDIL and PDIL. The CDIL provides basic connectivity to a Newton. To use the FDIL and PDIL you must first have established a connection using the CDIL. The FDIL provide a relatively simple way to map the contents of NewtonScript frames to desktop memory locations. The PDIL provides an easy mechanism for synchronizing data between a Newton application and a desktop application. At the time of writing, the PDIL is not yet available.

Figure 16: DIL Layers

The DTS sample SoupDrink has code showing how to establish a connection and exchange the contents of Newton soups with a desktop machine using DILs. It includes both Macintosh and Windows code for the desktop part of the program as well as Newton code. The DIL module in this course has a detailed walkthrough of the SoupDrink code.

All of the DILs are libraries written in C. On the Macintosh OS side, there are MPW and Metrowerks statically linked libraries. On the Windows side DILs are implemented as DLLs (Dynamically Linked Libraries) and so should be independent of particular C language implementations.

The CDIL or Communication Desktop Integration Library is used to create and open a virtual pipe to the Newton and then communicate by sending and receiving streams of bytes through the pipe. The CDIL creates and uses these pipeline in the following steps: initialization, connection, reading or writing, disconnecting (sound familiar?). Figure 17 shows the normal order of CDIL calls.

CDInitCDIL()
CDSetAppplication() // Windows only
CDCreateCDILObject()
CDPipeInit()
CDPipeListen()
CDPipeRead()/CDPipeWrite()
CDPipeDisconnet()
CDDisposeDILObject()
CDDisposeCDIL()

Figure 17: CDIL Calls

The CDInitCDIL() routine must be called before anything can be done with the CDIL. On Windows machines the routine CDSetAppplication() must be called next. There is no equivalent call in the Macintosh OS version. Next the routine CDCreateCDILObject() is called to create a CDIL pipe. CDCreateCDILObject() returns a pointer to a pipe which must be used for all subsequent calls which involve that pipe.
CDPipeInit() initializes a pipe object so that it is "open for business." In particular it defines the communications options including the media details such as media type (e.g., serial, AppleTalk, etc.), and relevant media options (e.g., speed of connection, data bits, modem type, etc.).
Next the pipe waits for the connection from the Newton using the call CDPipeListen(). When the Newton contacts the desktop machine, the application using the CDIL may accept the connection by calling CDPipeAccept(). At any time in this process the desktop application can cancel an attempted connection by calling CDPipeAbort().
Macintosh OS programmers note: callbacks are not executed at interrupt time so they are not subject to the restrictions placed on code running at interrupt time.
Once a connection is established and working, data can be sent and received using the routines CDPipeRead() and CDPipeWrite(). As with most CDIL routines, these calls may either be made synchronously or asynchronously with a callback routine.
From this point on, the desktop application and the Newton application will probably engage in a application specific protocol where there will be a predictable exchange of messages and data via the CDIL's virtual pipeline.
When the decision is made to terminate the connection, the routine CDPipeDisconnet() is called. Once this routine has completed connection has been broken and both sides must reestablish the connection before data can be sent or received.
Finally, when the desktop application is completely finished with the pipe, it must call the routines CDDisposeDILObject() to tear down the pseudo-object and CDDisposeCDIL() to clear the CDIL environment.

FDIL, or Frame Desktop Integration Library, (also called HLFDIL for High Level FDIL) supports transferring the contents of NewtonScript objects to a desktop machine. A CDIL connection must be established and a pipeline initialized before FDIL routines can be called.
The FDIL routine FDInitFDIL() must be called to initialize the library before anything else can be done using FDIL routines.
The simplest use of an FDIL is to map NewtonScript frames into C variables. If the frame shown in Figure 18 is going to be uploaded to a desktop machine, the desktop application can use FDIL calls to map this frame into the C structure shown in the figure below the Newton frame.
aNewtFrame:={ slot1:'b,
slot2: {slot3:24,
slot4:{ slot5:16,
slot6:$c}
}
slot7: "TROUT"};

struct fromNewt {
char slot1[5]; // symbols to str
struct slot2 {
long subslot3;
struct subslot4 {
long subsubslot5;
char subsubslot6;
}; // slot4
}; // slot2
char slot7[32]; // whatever max strlen
}

To build the mapping between the NewtonScript frame and the C structure repeated calls to FDbindSlot() are made. These calls match a Newton slot name (or an array object) to a C variable or buffer.
So in the case shown in Figure 18, there would be repeated calls to FDbindSlot(), one for each slot in the frame. Each of these would match a slot with the address of the C variable or (in this case) structure member. Once this mapping is done, the data can be transferred by a single call to FDget().When the Newton sends the frame data to the desktop, the FDIL will move the data into the appropriate locations on the desktop machine.
If data is being sent to the Newton, the desktop application would call FDput() to send the data from the addresses specified by the FDbindSlot() calls to the Newton. The FDIL will create a flattened frame format that the Newton. On the Newton an inputSpec would have been established which expected the data to arrive as a flattened frame (i.e., with a form of 'frame).
While this is the easiest and most efficient way to move data to and from the Newton, since the desktop application may not know the exact structure of the incoming data, there needs to be a way to get information about the structure of the data after it is transferred. This is done by moving data to the desktop machine into an "unbound" form. In this case memory is dynamically allocated on the desktop machine and the incoming data is put into a tree structure that can be parsed by the desktop machine.
As before, a CDIL connection must be established and then a call to FDget() will transfer any data. If any of the data transferred is bound it will be put in the appropriate location. Any and all data which represents slots which have not been bound goes into dynamically allocated memory in a tree structure.
To get the unbound data, the routine FDGetUnboundList() gets a pointer to the tree of data. This tree is organized with a branch for each element of the structure the data had on the Newton.
Figure 19 shows an abbreviated version of how the unbound data is structured in memory. Each slot of unbound data has a separate structure with a pointer to where the actual data is kept (var), a pointer to the next slot in the frame (next) and a pointer to the first child (i.e., sub-frame, in children). Note that in the figure, only one of the var members is shown pointing to a location in memory and that only some of the next and children members are shown pointing onward.

As an example of the use of unbound data, if the desktop application did not know the structure of the NewtonScript frame shown in Figure 18, it could be brought into the desktop machine in an unbound form. In this case the pointer returned by GetUnboundData() would point to a structure which describes slot1. The next member of this structure would point to a structure describing slot2 whose next member pointed to a structure describing slot7.
The children member of the structure describing slot2 would have two child structures, one of which (corresponding to slot4) would have two children.
In addition to the structure members described, each structure has a description of the data including the name, type, size and location of the data in the desktop machine.
Using this structure code may be written to parse and use the unbound data.
Examples of CDIL and FDIL usage (including parsing unbound data) can be seen in the DTS SoupDrink sample.
Finally, after the desktop application is done with the unbound data it should be disposed of by calling FDGetUnboundList().

1. Newton Programmer's Guide (NPG): Communications is the reference book describing communications on the Newton.

2. DILBOOK.PDF is the latest documentation for DILs. Current as of this writing is documentation for the Beta 8 (B8) version.

3. Newton Programmer's Guide (NPG):System Software is the basic programming guide for all non-communications programming.

4. DTS Sample code - get the latest available. CDs are sent out periodically and updates are posted on various online services such as eWorld.

The obvious places to go from here are code examples such as provided by DTS and the class solutions and to the endpoint module of this class. In particular the Basic Modem DTS sample shows many of the basic features of 2.0 endpoint scripting including option templates, and asynchronous calls. The Archive Transport shows basic transport functionality and the SoupDrink example gives a thorough example of transferring data using DILs.
Sorry, the DIL module has not been released yet so the DIL Archive code is not yet available.
The Newton DIL code and the Archive Transport used in the DIL Archive lab show endpoints being used for specific tasks which are likely to recur (uploading soups and sending and receiving frames respectively).
Of course the Q&A section of this module is probably worth looking at too.

Developer Relations Developer University DU Online Topics

View Classroom Courses & Schedule Register for Classroom Courses Order DU Products Send Feedback to DU

Developer Services | Technical Information | Tools | Sample Code

Main | Page One | What's New | Apple Computer, Inc. | Find It | Contact Us | Help

---

Copyright Apple Computer, Inc.
Maintained online by commscourse@newton.apple.com
Updated 26-Sep-96 by pmr