This text, originally at http://www.zendo.com/vsta/mail/1/0146.html was written by me Basile STARYNKEVITCH in october 1993 on a VSTa mailing list. I added a few notes in september 2003.
NeWS was a great thing from Sun (it is a dead Sun product; in SunOS5.3, coming out FCS in december 1993, there is no more NeWS, just an X11R5 server with Display Postscript) . NeWS means Network extensible Window System (or Server). Actually, Sun's window server was a mixture of NeWS and X11R4, but these can be almost viewed as 2 different servers sharing common devices (mouse, keyboard, screen). More precisely, the X11 server was a special NeWS lightweight process (see below). NeWS and X11 had different ports and different protocols. I am talking of the latest NeWS server (in Openwin3). Integrating NeWS and X11 was a difficult problem to Sun, and they did not succeed perfectly on it.
The NeWS3 reference manual has part number 800-6736-10 from SunSoft (september 1991). The NeWS Toolkit 3 reference manual has part number 800-6319-10.
The key idea of NeWS (and you already know it) was that the window server was programmable. So a typical NeWS client:
Actually, a client did not talk naked NeWS but used a big library (written in NeWS, more than 30000 lines of NeWS source code, and loaded into the server at server initialization time) called The News Toolkit (or TNT). TNT code handled connection initiation and termination, global selection processing, keyboard maps, and provided an object extension to NeWS. The initialization of the NeWS server took more than 10 seconds. TNT provided an OpenLook compliant widget toolkit (mostly running in the server).
NeWS is a superset of PostScript. I assume you are familiar of most PostScript ideas (it is a stack language, and uses several almost independent stacks, including a call stack, an operator stack, a graphic state stack, a message sent stack -for object oriented programming support- and a dictionnary stack. A dictionnary hold variable bindings; it is a finite set of name (or key) to value bindings). I also assume you know some basics of window systems such as X11.
NeWS data are atoms (numbers, booleans, strings, and names, quite like in lisp, where names are called symbols), arrays, dictionnaries, events, canvases, lightweight processes (or lwps) and others (fonts, etc). Composite objects (arrays, dictionnaries, etc) contain (reference to) any kind of objects (perhaps themselves). Executable arrays are NeWS program fragments. All variable bindings (ie values) are thru dictionnaries (in contrast to Scheme or Lisp), so a variable is just a name, and a name has no internal content (except its immutable string of characters designation), and the value of a variable is obtained by scanning the dictionnary stack until a dictionnary containing the variable as a key is found. NeWS is a reference counting garbage collected system.
Window system specific objects such as canvases, events, lwps are magic dictionnaries. They are magic in the sense that changing some particular key value in such objects could have some important side effect. Also, magic dictionnaries should have some fixed names (or keys) and are created with them. All dictionnaries (magic or not) can have any additional keys added into. Magic keys have some specific predefined names.
Canvases are very similar to X11 windows. There is a hierarchy (tree) of canvases, analog to the X11 window hierarchy. A canvas is a magic dictionnary. Hence mapping a canvas is done by changing the /Mapped key to true. Canvases are part of graphic contexts, so all graphic operators are executed in a given canvas (much as on a PostScript printer all graphics are executed on the paper). The top of the canvas hierarchy is a virtual root canvas, containing several physical screen canvases. Canvases, (like postscript printer devices) contains a coordinate transformation matrix and an enclosing (clipping) path. In effect, canvas can have arbitrary shapes (it is possible to have a round canvas to implement a round clock). Canvases have cursors. Each canvas has an interest list, which gives the event the canvas is interested in. X11 windows are special kind of canvases.
Events are somelike analog to X11 events. The main difference is that an event is not a protocol message; it is generated (by NeWS code or by physical devices such as mouse and keyboard) and handled inside the NeWS server thru appropriate client loaded code. Events are also magic dictionnaries, containing /Name, /Action, /Canvas, /TimeStamp, /Coordinates and several other important keys. The system generate events. For instance, a mouse left button press generate an event whose /Name is /LeftMouseButton, whose /Action is /DownTransition, whose /Canvas is the canvas inside which the mouse cursor was, and whose /TimeStamp and /Coordinates indicate unique time&space measures. There is a global event queue (containing unhandled events) and events are distributed (ie copied) to each lwp expressing an interest into it. Interests are templates or patterns for events. Syntactically, an interest is an event with the /Interest key set to true. NeWS operators exist for event (or interest) creation, event sending (ie adding an event to the global event queue, where system initiated events are also automagically added), and interest expression.
All events are put in the global event queue (by /TimeStamp order), either because they are automagically generated (eg a keypress) or because they are explicitly sent in NeWS code (thru the sendevent NeWS primitive). So, events are also used as interclient communication (such as selection processing). In practice, hardware generated events are handled by existing NeWS code (ie the TNT library, see below) which sometimes translate them into higher level events. Hence, event can be used for agent style programming. Global selection processing (eg cut & paste) was partly done in NeWS global code (inside TNT) using such program generated (and not device generated) events.
Beside obvious hardware related events (mouse & keyboard press, release, motion, etc) the NeWS server generates 2 important kinds of events: /Damage & /Obsolete events. Damage events (whose /Name is /Damage) are similar to X11 expose events and are generated when a canvas (or part of it) is damaged (because it just became visible). Damage events gives a damagepath, which enclose the canvas region needed to be repainted. Obsolete events (whose /Name is /Obsolete) are generated by the garbage collector for finalization purposes. Each object reference in NeWS can be hard (the usual default case) or soft (an object reference may be made soft thru the soften NeWS primitive, and can be made hard by the harden NeWS primitive). When all references to an object are soft, an Obsolete event is generated (whose /Action jey is the obsoleted object). This permits some cleanup. In particular, every cycle of references (and these are very common in windowing systems) should contain at least one soft reference.
A NeWS lightweight process (or lwp) is a virtual NeWS executing machine inside the NeWS server. So it is a NeWS interpreter. Each lwp is a magic dictionnary containing among others an /OperandStack, a /DictionnaryStack, an /ExecutionStack, a /SendStack, an /EventQueue (containing events distributed from the global event queue into this lwp but not yet processed), an /Interests array which give the template of events interesting for this lwp. Lwps are grouped in process groups. Of course, all lwps are internal to the NeWS server; as seen from the Unix kernel, there is only one Unix process which is the NeWS server itself! Most lwps are of course asleep, usually awaiting for input (on a socket from the client Unix process) or for an incoming event thru the awaitevent NeWS primitive.
Synchronization primitives (in particular monitors) are provided for serialization between lwps.
In practice, NeWS was used with TNT. TNT contained code for NeWS server initialization (including keyboard mapping) and for widget programming. TNT is mostly made of a (multiinheritance) class hierarchy. NeWS classes and instances are dictionnaries (magic or normal). The send (object oriented message primitive) operator is a NeWS primitive, but the class hierarchy was in TNT (and TNT was delivered as 32000 lines of source NeWS code). The NeWS server initialization was another 21000 lines of source NeWS code. In practice most NeWS object creation (eg creation of interests, lwps, canvases) was done by instantiation of (client specific subclasses of) TNT classes. OpenLook widgets such as text fields, menus, buttons, etc are provided by TNT classes (and subclasses of a ClassCanvas provided in TNT). Window managment (in the sense of X11 wms) could be done partly or completely within NeWS.
The NeWS initialization code started several NeWS system lwps, including a NeWS listener (just for listen(2) and accept(2) socket calls), an X11 listener, a GlobalEventMgr and a GlobalSystemEventMgr (handling system wide NeWS events, including selection processing).
When a new Unix process make a successful connection to the NeWS server, this connection (handled by the NeWS listener) made a new NeWS lwp specific to this Unix NeWS client. This lwp just read NeWS stuff from the wire (ie connection to Unix client) and interpreted it.
Data exchanges between NeWS and client is either in plain (printable) ascii or in a more compressed (somehow binary encoded) format. Each NeWS input token was in ascii (if it started with a printable ascii character >=0x20 and <0x7f) or in binary encoded form (if starting with a byte >=0x80, specifing the format and length of following bytes). A NeWS object can be designed in ascii by its name or in binary by its number in a connection specific array.
Conventionnally, a well behaved NeWS client has 2 NeWS lwps: the client NeWS interpreter, created by the NeWS system listener lwp (itself created at NeWS startup time), and a local event manager (created by appropriate class instantiation at client initialization). The local event manager expresses interests in all events pertinent for the client (it was attached to activated canvases).
A typical client send specific definitions (class definition for new widgets or new widgets variants; instanciation of classes for new object creations). The loaded code either handled events locally (eg keystrokes into a text widget are interpreted by TNT textwidget code to fill or edit the text widget's value) and at some times send a message back to the client. Hence, the expected wire bandwith was low; for instance, a client recieved NeWS messages only when a text widget was filled (by pressing a return key) and not at each keypress. Similarily, most repainting (eg Damage events) is done inside TNT and does not require client intervention.
Some simple applications (eg a calculator, or a window manager) could even be written entirely in NeWS. A very simple client program, called psh, was used as a NeWS shell. psh just opened a NeWS connection and copied all its stdin into this connection.
A simple programming utility, called cps, generated C code for sending fixed form NeWS messages in binary form. Essentially, it generated printf-like formats and statements.
NeWS3 also contains a package system.
NeWS was in practice quite slow[1] . Waiting half a minute before seeing the first window is very annoying. I think that NeWS lacked of easy extension capabilities (ie adding new NeWS primitives programmed in compiled C code, in a dynamically loaded shared library).
Most of NeWS failure was marketing, not technology. I think (as several others) that Sun should have put NeWS in the public domain [2].
In practice, i think that a fixed widget set could have been compiled into a programmable window server. In other words, big parts of the TNT widget code should have been rewritten in C and provided as NeWS primitives. Perhaps a NeWS to C compiler could have been nice. Ideally, an online, incremental, compiler would have been great: the client could upload some code which could be compiled on the fly, hence providing the best of 2 words: extensibility by code uploading and speed by compilation.
Choosing PostScript as a programming langage was in my opinion a bad idea. Debugging NeWS scripts and codes was really difficult (first because of a lack of a good debugger, and mostly because stack oriented languages such as PostScript or Forth are not enough structured: passing an extra argument, or forgeting one, is a common error, which is very difficult to detect, since there is no argument list, but just an operator stack; the extra argument was kept on the stack and incorrectly used later). I suggest using a small Scheme-like dialect for programmable servers, because Scheme is a structured and powerful, but yet small, language (of course, bignums should be excluded since they are useless in window systems); and Scheme to C compilers seems to exist somehow.
Having interpreters (lwps in NeWS) independent from client connections is too general (and de facto unused in NeWS). I suggest attaching one interpreter to each client connection. Also, I think that all messages should be syntaxically alike (in NeWS terminology, even input from the client connection should have been handled as an event).
I think that the imaging model (a complete transformation matrix with a general clipping path) of PostScript (or NeWS) is too general (and too expensive) for ordinary window systems. Plain rectangular window with fixed pixel-based coordinates (ie the plain X11 imaging model) should be enough for windowing system (and high multimedia technology needs perhaps even more than PostScript like sophisticated coordinate systems).
In my opinion, a programmable window server should already provide some fixed low level Graphical User Interface (eg elementary widgets like buttons, menus, multifont text fields and labels); having it wired into the server langage would make more consistent look & feel.
Also, I strongly suggest writing (for prototyping and pedagogical purposes) an X11 client emulating most of the proposed window server programmable protocol. Perhaps looking into Tk would be a good idea, although i think that tcl is not really appropriate as a server language.
Above text was written by me in october 1993. Almost ten
years later, I made Guis - a cool
hack which provides a small GTK2 widget server
(scriptable in Python or Ruby) to a single client
application.
Note 1 If I recall correctly, this was on a Sun4/110 with 16Mb RAM, at that time a quite fast machine (fasters than PCs).
Note 2 I actually wished that NeWS become an open-source or free software (public domain was a wrong phrasing)
Extra Note. I think that the NeWS team became (partly) the Java team, and I never understood why Java was not embedded into X11 (like NeWS suggested it).
($Id: NeWS_descr_oct_1993.html 17 2004-12-30 02:52:30Z basile $)
Basile STARYNKEVITCH but remove _NOSPAM and .invalid from email.I am interested by your feedback (on this newsdescr page)