Data eXchange Infrastructure

By Pype - 18.04.02
status: request for comments. your comments




Purpose of DXI infrastructure

As softwares are becoming more complex everyday, we need a way to produce new programs quickly from legacy existing programs. The DXI (pronounces "dixie") infrastructure is an attemp to address the program/program and program/user problem in an efficient and flexible way. The idea is to define a common "language" for exchanging informations between two program that provides enough informations about the needed informations so that a generic program can create a user interface for it if needed.

If used, this infrastructure could completely remodel the way I/O of a program are usually done and virtually express them independently from any viewer (text, html, gui ...) technology. Of course, we could have used the existing xml infrastructure, but it has inherent syntax that make it require a complex parser at every component, and which is not well-suited for readers that browse lot of informations to filter them.

Overview

primitive types:
compound types:

ex: the "ls" output:
"%U(
file=%S(name=%255s: size=%32ui: owner=%#UID#: access=%#TIME#):
dir=%S(name=%255s: size=%32ui: owner=%#UID#: access=%#TIME#):
end=%V
)"

Because the structure has been transmitted out-of-band, we can reduce the amount of needed bytes compared to a ASN-1 scheme. #type# is just some syntaxic sugar: the time will be encoded as a number of seconds since epoch, and we expect receiver to agree with this convention.

[1]-[[[8]-[kernel.c]]-[0x12345678]-[0x0050]-[0x13041206]]
which is equivalent to the byte stream
01 08 k  e  r  n  e  l  .  c  78 56 34 12 50 00 06 12 04 13
on a x86 machine.

However, as some fields can have a variable size, it could be a good idea for fast browsing to start a structure with its real length:

[typeid = 1]-[item-size = 19] - [
                [ string size = 8]-[ kernel.c ]
                [ 0x12345678 ]
                [ 0x0050 ]
                [ 13:04:12.06 ]
]

By doing so, the application that only want to find out directories can quickly skip every files. If a structure is being used as a sub-field of another structure, it can make sense to precede it by its size (as we do for strings) so that an application that is only interrested by things that lay behind that structure can get it. quickly. Note that this is only mandatory for variable-sized structures.


You'll notice that the dxi format also includes the symbolic name of each field. This is for generic processing purpose. An application that knows absolutely nothing of the stream it gets still can produce a comprehensive output. It also can be used to convert DXI representation into a XML one:

<file name="kernel.c" size="12345678" user="pype" access="13:04:12">


We can easily create a displayer for the generic DXI pattern, that will get the dxi definition of a stream and display it as a table of row. every "system resource" class could have a registered "printer" in kds://system.dxi.classes.*, so that simply calling system.dxi.classes.UID.print(0x500) would return the string "pype".

using dxi 

the usage of dxi formats is not restricted to output of informations: a program that expects some setup from an external source (gui or some other calling program) can create an input stream with a dxi format of what kind of infos are needed. The shell can then construct a prompt or a form where the user will fill arguments, collect informations and then sending them as a dxi stream to the application. The use of a union between a type and the VOID type can be used to express that filling the given type is optionnal.

Let's imagine we have a program find that can browse directories looking for some specific pattern in filenames. That program can be configured to recurse subdirectories or not and to match files that are newer/older/dating from a given time.

the find "init" stream would then accept dxi input having the following format:

"%S(
    pattern=%s:
    %U(recurse=%b:%V):
    timing=%U(newer=%#TIME#:older=%#TIME#:dating=%#TIME#:%V)
)"

A text-based shell will now be able to produce a filter for that application and to respond on "help find" with "find <pattern> [--recurse] [--newer=<time> | --older=<time> | --dating=<time> ]". When "find clicker --recurse" is entered to the shell, it can be transformed into a bytestream that can be sent through the init stream of find and the application can read its configuration and copy it into a global configuration object before it starts working.

Now, if your shell is a graphic-based one, it can produce a form from the dxi format and let the user fill it in:


find parameters

recurse



well, i admit it: the above overview is absolutely awfull, but you have to admit that the dxi format string is strictly enough to generate it. Now, the caller of the application might want to have more informations about the items themselves. We could imagine that it can get them with another stream that would be bidirectionnal (or two unidirectionnal streams under the same name) : "help" where the shell would put the name of the item it'd like to get infos from and read an item (string) as a result. So sending "pattern" on the "help" channel would reply "Regular expression of the pattern searched accross the directories" and sending "recurse" would reply "dive into sub-directories recursively".

Now, according to the shell you plan to make, you can use this information to generate tooltips for the input forms, or just do a unix-like output with
--recursive (-r)    dive into sub-directories

By giving a name to "timing", we give ourselves the ability to give a generic information about the timing options.