How to create and populate a MDSplus tree

In this first lesson we shall build a very simple MDSplus tree. A MDSPlus tree is a database which contains several types of data. A data item may be a number, a string, a signal or, more generally an expression i.e. a combination of data (possibly stored in the same tree) and operators.
Besides data, the nodes of a MDSplus tree may contain other kind of information, but we shall discuss about it in the following tutorials.

The MDSplus tree represents the core of MDSplus, and there are many ways for interacting with it. Here, we shall use two tools:

1) The Tree Command Language (TCL, not to be confused with TCL/TK) for creating and editing a tree;

2) The jTraverser tool for providing a graphical interface to the tree.

A Quick Check for some important environment variables

Before starting working with MDSplus, it is importat to make sure that a few environment variables are properly set. When installed via an install shield, the variables should be ok, but in the case MDSplus is build from the source distribution, the following variables must be defined for Linux:

• LD_LIBRARY_PATH this variable is used by the Operating System to locate dynamic libraries, which are listed separated by a colon. <MDSPlusRoot>/lib or <MDSplusRoot>/lib64 (for 64 bit architectures)must be included.
• PATH: must include directory <MDSplusRoot>/bin
• MDS_PATH must be defined as <MDSplusRoot>/tdi
• CLASSPATH must include (separated by colon) <MDSplusRoot>/java/classes/jScope.jar, <MDSplusRoot>/java/classes/jTraverser.jar, <MDSplusRoot>/java/classes/jDevices.jar, <MDSplusRoot>/java/classes/MDSobjects.jar, <MDSplusRoot>/java/classes/jDispatcher.jar

If the system has been built from the source distribution on Windows using Visual C++ 2008, the envirnonemnt variables must be set as follows:

• PATH: must include directory <MDSplusRoot>\Win32\Debug, separated by a semicolon from the other items
• MDS_PATH must be defined as <MDSplusRoot>\tdi
• CLASSPATH must include (separated by semicolon) <MDSplusRoot>\java\classes\jScope.jar, <MDSplusRoot>\java\classes\jTraverser.jar, <MDSplusRoot>\java\classes\jDevices.jar, <MDSplusRoot>\java\classes\MDSobjects.jar, <MDSplusRoot>\java\classes\jDispatcher.jar

The creation of a sample tree

Before starting we need to define an environment variable which indicates to MDSplus the location of the tree. Let's call the new tree my_tree, so we need to define the environment variable my_tree_path (NOTE: you must use lowercase letters for this variable) to the directory which will contain the database (the general rule for the variable name is <tree name>_path).
On Linux (using bash) we define the environment variable as follow:

export my_tree_path = <directory>

On windows, things are a bit more complicated. These are the required steps:

1) Open the Control Panel
2) Select System
3) Select Advanced Tab
4) Select "Environment variables"
5) If adding a new variable select New
6) In "Variable name" and "Variable Value" write the the name (<tree_name>_path) and the value (the directory which will contain pulse files)

Note that on Windows, the variable is defined forever, while on Linux you need to define it for each session, therefore it is convenient to put the definition in a shell script, such as .bashrc.

Now, we shall create a tree containing three nodes:

NUM1 containing a number
NUM2 containing an array
NUM3 containing an expression
TXT containing a string

Let's do it using TCL (a complete reference to the language is available here):

1) Start TCL
On Linux system TCL is started by command mdstcl. On Windows system TCL can be started by selecting the TCL menu item in the MDSplus menu. If MDSplus has been built from source files and not been installed using the install shield, a TCL session is started via command: mdsdcl -PREP "set command tcl_commands/prompt='TCL>'/def_file='*.tcl'"
2) Create a new tree: TCL>edit my_tree/new
3) Add node NUM1: TCL>add node NUM1/usage=numeric
This command creates a new node named NUM. It will contain numeric data, but currently is empty.
4) Fill node NUM1: TCL>put NUM1 "2"
This command fills node NUM1 with the number 2.
3) Add node NUM2: TCL>add node NUM2/usage=numeric
4) Fill node NUM2: TCL>put NUM2 "[1,2,3,4,5,6,7^]"
5) Add node NUM3: TCL>add node NUM3/usage=numeric
6) Fill node NUM3: TCL>put NUM3 "NUM1 + 3 * NUM2"
Node NUM3 now contains an expression involving the contents of nodes NUM1 and NUM2
7) Add node TXT: TCL>add node TXT/usage=text
This command creates a new node named TXT. It will contain text, but currently is empty.
8) Fill node TXT: TCL>put TXT " 'This is a text string' "
9) Write the current tree:TCL>write
10)Close the tree TCL>close

It is worth noting that the values we have just inserted into the nodes of my_tree represent different things, such as the number 2, an array of integer values, the string 'This is a text string'. Nevertheless, within MDSplus, they all represent expressions. The expression is a central concept in MDSplus: every datum is an expression. An expression can be something as simple as a simple number, or a node reference, but may represent also a very long combination of numbers, references and operators. Expressions are defined in a human-readable form using an appropriate matlab-like syntax, called TDI (for a more complete introduction to the TDI language, see here). For example, the expression NUM1 + 3 * NUM2 defining the content of node NUM3 evaluates to 2 + 3* [1,2,3,4,5,6,7]=[5,8,11,14,17,20,23]. Note that what is stored in the tree is the expression definition, not its evaluated value. Evaluation is done on the flight every time it will be required.

Looking at what we have created

Now we are ready to look at what we have just created. However, before presenting the graphical interface to the tree, it is worth introducing the concept of shot number. The tree we have just created is a database (by the way, look at the files created in the directory specified by my_tree_path, and you will discover that the database is implemented by means of three files) and therefore it is possible to insert, modify and retrieve the contents of its nodes. However, in nuclear fusion research, as well as in every other shot-based experiment or application, we need to create a description of each experimental shot, which is naturally described by a shot number. This means that we shall create a separate instance of the tree for each shot. Moreover, every experiment will define a set of pre-defined set-up parameters, and will produce some data.
The MDSplus approach is therefore the creation of a template database (called experiment model), containing all the required set-up values, as well as defining the places (represented by empty nodes in the tree) where acquired data will be stored by the data acquisition routines.

The usual approach is therefore the creation of the template experiment model (by convention, represented by the shot number -1) whose actual content (i.e. the set-up parameters) are likely to change from shot to shot. Just before the experiment sequence, the experiment model is copied into a pulse file (i.e. a tree with an associated shot number). Data acquisition routines will read the stored parameters for the proper set-up before the experiment, and will write acquired data into the tree just after the experiment.

Note we are ready to look at our newly created tree (experiment model). Let's do it using the jTraverser tool:

1) start jTraverser;
2) Give the command File->Open
3) Write my_tree in the Tree: field and -1 in the Shot: field of the popup window.

Now we have a graphical view of our tree:

You can see the nodes just created, whose associated icon indicates the usage for that node. When data is inserted in a node, the data access layer of MDSplus checks whether the type of data being inserted matches with the usage for that node.
Using jTraverser, by pressing MB3 button over a node, you can perform the following operations on the node content:

The commands we are interested in, for now, are:

- Display Data: displays the content (if any of the selected node);
- Display NCI: displays accessory node information such as size of contained data, and insertion date;
- Modify Data: modify the content of the node;

When modifying data for node NODE3, jTraverser displays the following dialog:

You can type any expression (remember the MDSplus mantra: everything is an expression) which replaces the content of the node.
To experience the check performed on the node usage when inserting data, try to change the content of the node NUM3 to 'This another string' and to save it. You will receive the following error message:

clearly indicating that the usage of that node is not the correct one (it must be an expression returning a numeric value).

Defining a tree structure

Up to now, we have created a flat collection of nodes, possibly containing data. As their name suggests, MDSplus trees allow data to be organized in a hierarchical (tree)structure. Let's do it using TCL, by adding a subtree called SUB1 containing a numeric node SUB_NODE1 and another subtree called SUB2, containing a text node SUB_NODE2.

TCL>edit my_tree
     Open tree my_tree for editing (i.e. adding/removing nodes). The default shot number is -1 (the experiment model)
TCL>add node .SUB1
     Creates a new subtree. Note that the name is preceded by a dot.
TCL>set def .SUB1
     moves into SUB1 subtree. Much like the UNIX cd command.
TCL>add node SUB_NODE1/usage=numeric
     add empty node SUB_NODE1 to subtree SUB1.
TCL>add node.SUB2
     Creates subtree SUB2
TCL>set def .SUB2
     Move into SUB2 subtree
TCL>add node SUB_NODE2/usage=text
TCL>write
     Write the newly created tree
TCL>close   
     Close the tree.

Now, when we open my_tree with jTraverser, and explode subtrees, we get the following image(you can explode/implode subtrees by clocking on the associated handles, or double clicking the subtree):

In the case you are allergic to graphical interfaces, you can nevertheless navigate into the tree structure of the tree using the TCL commands set def <node name> (for moving into a subtree), set def ".-" (for moving one level up) and dir (for showing the content of the current subtree).

Understanding node names

While in a flat list of nodes, the node name is enough to uniquely identify the single data item, in a tree structure, it is necessary to define the whole path. Let's take an example: select the popup item show data in jTraverser over node SUB_NODES2. After enlarging the displayed window we get:

The dialog tells us that the node is undefined (does not contain data yet), and a couple of other information we shall see later. We are now interested in the title of the dialog which shows the path name of node SUB_NODE2. The first part \MY_TREE::TOP indicates the root of tree MY_TREE and the rest of the name is the path from the root to node SUB_NODE2. Observe the dots and the colons: the hierarchy organization in MDSplus trees defines two kinds of nodes for every subtree: members, whose name is preceded by a colon, and children, whose name is preceded by a dot. In this example SUB_NODE2 is a member of node SUB2 which is in turn a child of node SUB1. The MDSplus data organization defines also the concept of default position. Node pathnames can in fact be absolute (i.e. starting from the root) or relative (i.e. starting from the default node), and the default node position is indicated in red in jTraverser and can be changed selecting the popup menu item Set Default (don't worry about it, you will use it very seldom).

Even though MDSplus allows an arbitrary organization of members and children (a member node may have members and/or children), the usual approach is to define members for containing data and children for defining the structure (data cannot never inserted into a child node).

Even in this simple example, it is clear that node pathnames can be very lengthy, increasing also the probability of typing errors. For this reason, MDSplus allows one or mode unique names to be associated with a given node. These identifiers are called tags and are very useful for providing a short and easy name to nodes (usually containing acquired data) which are often referred for data display.
Even though it is possible to define tags in TCL, it is easier to do it with jTraverser. To give tag name MY_SPECIAL_NODE to node \MY_TREE::TOP.SUB1.SUB2:SUB_NODE2 using jTraverser, you need to open experiment my_tree selecting also the edit checkbox in the open dialog. Then you position the mouse over the node, press MB3 button and select the Modify Tags item in the popup menu. You then get the following window:

in which you can add/remove the tag name(s) which will be associated with that node. Add MY_SPECIAL_NODE in the list (remember to press Add Tag button) and press Ok. From now, the tag is associated with that node, and is shown in the data dialog when showing or modifying data.

(Note that in this example, the first time you select the modify data popup item for this node, the node will be shown as undefined, as no data has been added to it in TCL. You can insert new data into it by changing the undefined option into expression and then typing an expression)

Tag names can be used everywhere a node reference is required, e.g. in an expression referring to that node, and the general syntax is:

\<tree name>::<tag name> (e.g. \MY_TREE::MY_SPECIAL_NODE in our example)

When only one tree is open (we shall see in another tutorial that it is possible to open at the same time several trees), the first part (<tree_name>::) may be omitted.
Tag names can have an arbitrary number of characters. Node names are instead limited to 12 characters.

The TCL and jTraverser tools

TCL and jTraverser are two equivalent tools, in the sense that both allow the creation and modification of MDSplus trees. It is in fact possible in TCL to navigate into the tree structure using the commands set def and dir. It is even possible to look at data in TCL, but what we see is not exactly what you would expect. Let's try it with the following commands:

TCL>set tree my_tree

TCL>show data NUM3

\MY_TREE::TOP:NUM3
DTYPE_FUNCTION : OPC$ADD
DTYPE_NID : \MY_TREE::TOP:NUM1
DTYPE_L : 2
DTYPE_FUNCTION : OPC$MULTIPLY
DTYPE_L : 3
DTYPE_NID : \MY_TREE::TOP:NUM2
DTYPE_L : Array [ 7 ]

Not very clear, isn't it? In fact the show data command displays the internal organization of the information required to specify the expression NUM1 + 3 * NUM2, an information very useful when debugging the system, but surely not easily readable to end users.

On the other side, jTraverser can be used also for changing node names, adding/removing tag names and adding/removing nodes. For these operations you need to select the edit checkbox in the Open dialog. You will note that the popup list in this case is more complete:

The new commands in which we are interested (the others will be discussed in other tutorials) now are;

- Add Node: add a member node to the selected node (that over which MDB3 has been pressed), whose usage is specified by the associated sublist (good occasion for looking at the usages supported by MDSplus - for now we know only Text and Numeric)
- Delete Node: remove the currently selected node;
- Modify Tags: add/remove tag names for this node;
- Rename node: rename the pathname, possibly moving the node to another subtree.(It is also possible to change the name of a node "a la Windows" by slowly clicking twice over the node name, editing the new name, and then pressing Ok).

Concluding tip: use TCL for creating and populating a tree the first time. Use jTraverser for looking at it, and for changes in its structure.

Understanding MDSplus signals

In our current exercise we have dealt with strings, numbers, arrays and expressions. A data type which is very useful in data acquisition is the signal, i.e. an array of data (y axis values) with associated x axis information. Most times, signals refer to some physical signal acquired at a certain frequency for a certain time. In this case the X axis represents the time value of the acquired samples.
MDSplus defines explicitly a data type for signals, and a signal usage for tree nodes. Though signals are usually produced by some data acquisition routine, it is possible to define signals manually. How?? By defining the corresponding expression, of course!

Let's add to my_tree a node called SIGNAL1 containing a sine wave made of 1000 sampled points taken over a period of 1 second.

TCL>edit my_tree
TCL>add node SIGNAL1/usage=signal
TCL>put SIGNAL1 "build_signal(sin(6.28 * [0..999]/1000.),, [0..999]/1000.)"
TCL>write
TCL>close

The TDI expression syntax for defining signals is the following (full reference to TDI expressions is here):

BUILD_SIGNAL(DATA, RAW, [DIMENSION...])

where:

- DATA is an expression defining the Y values. (sin(6.28 * [0..999]/1000.) in our example)
- RAW is an expression indicating raw data. Often an acquired signal is made of raw data converted then by taking into account parameters such as gain and offset. Now we are not interested in it, so we omit its definition (the two commas in the TCL example are not a typing error).
- DIMENSION is an expression returning the array of X axis, usually (but not always) time values. (In our example the expression is [0..999]/1000.)

More on MDSplus expressions

You will have understood that expressions are ubiquitous in MDSplus. Expressions are defined by means of a very rich syntax, called TDI, with tens of possible operators supporting many different data types. Just a few examples of expression:

• 2.3 A floating point number
• 2 + 3.14 A summation evaluated to a floating point number
• 'This is a string' A text string
• 'A string '//'Another string' The concatenation of two strings
• [1,2,5,7] An integer array
• 2. * ([1,2,5,7] + [2,2,2,7]) An (IDL or Matlab like) expression involving scalars and arrays.
• \MY_TREE:NODE1 A reference to a MDSplus tree node
• 2 * (\MY_TREE:NODE1 + 3) An expression involving number and tree nodes

You can do more: it is in fact possible to define variable in TDI. Once defined, they can be later used. For example:

_a = 2 * (\MY_TREE:NODE1 + 3)
                             Assign an expression to variable _a
_b = sqrt(_a)
                             Assign to _b the square root of _a

Remember that TDI variable always begin with an underscore!

You may wonder what are variables useful for. In fact, it makes no sense to use variable in insulated expressions, but expressions can be concatenated to form a TDI function, much like you would do in IDL, Matlab, or a programming language. Here is an example of a TDI function which returns the sum of all the members in a given array:

fun public vector_sum(in _a)
 {
     _sum = 0;
     for(_i = 0; _i < size(_a); _i++)
         _sum = _sum + _a[_i];
     return(_sum);
 }

Looks like a real program, isn't it? Note that variables (remember the underscore!!) are not declared, and can contain any type of data.

You save this function in a text file called vector_sum.fun and you put it in a subdirectory of MDSPLUS_DIR/tdi, where MDSPLUS_DIR is the root of the MDSplus distribution of your system.

After copying it in the right place, you can use without any compilation. Here is an example of usage:

 _arr = [1,2,4,5];
 _sum = vector_sum(_arr);
 write(*, 'The sum of the elements of ', _arr, ' is ', _sum);
 _sum2 = 2 * vector_sum(_arr);

As everybody who wrote a program knows, it is possible to make syntax errors when writing TDI expressions and programs. As there is no compilation, the errors arise when that expression is evaluated. The error message is often not very clear (just to be polite ), so correcting errors when writing TDI functions my be very frustrating. In this case the TDITEST tool can be very useful, and allows to spare quite a lot of debugging time.

When starting TDITEST you get an empty screen in which you can write TDI expression and see the result of their evaluation. You can define variables and use them in later expressions. You can also exercise the cornucopia of TDI predefined functions listed in the TDI reference in order to understand their meaning. When developing TDI functions, you can test each line individually, thus achieving an acceptable debugging.

What next?

We have now learnt how to build and populate a MDSplus tree. We know also how references into the tree are specified and we know that it is possible to use expressions for a flexible definition of data.

We are now ready to go further, and to access data contained in the tree. The next tutorial will show you how to export MDSplus trees over the network, and how to access data from programs and from jScope, the MDSplus tool for waveform display.