Documentation:Reference:MDSIP - MdsWiki
Navigation
Personal tools

From MdsWiki

Jump to: navigation, search

Contents

MDSip (MDSplus remote data access protocol)

MDSplus provides a very simple but powerful protocol for connecting to remote servers to perform tasks such as retrieving or writing data and performing actions. In an MDSip session, the client program opens a connection to a remote server and the communicates with that server by sending an MDSplus TDI expression text string along with optional parameters which are scalars or arrays of text or numbers. The server process evaluates the expression and sends the result as a scalar or array of text or numbers. Except for a few other low level functions used by MDSplus internals, that is the main functionality of MDSip. Although the concept is simple, combining this functionality with the capabilities of TDI enables MDSip to become a gateway for data exchange even if the data is not stored in native MDSplus format. The original implementation of MDSip was done in 1994 to give Unix based system access to MDSplus data which at the time only existed on OpenVMS servers. It has become a very important capability of MDSplus ever since, providing access to data and providing an infrastructure for distributed computing for data acquisition and analysis.

Simple API

A simple programming interface exists for MDSip in many languages. Most interfaces consist of few basic routines:

  • MdsConnect - Connect to a remote server by specifying a connection string.
  • MdsValue - Send a TDI expression along with optional arguments and get the answer.
  • MdsClose - Shutdown the connection to the remote server

MDSplus tree specific functions:

  • MdsOpen - Open a MDSplus tree on the remote server
  • MdsPut - Put data in a node of the MDSplus tree on the remote server
  • MdsSetDefault - Equivalent to a cd command to move to a node location in the MDSplus tree

The actual format and names of these basics routines may vary depending on the language so you will find more details in the various language specific reference pages on this web site.

Authentication

The standard MDSip included in the regular MDSplus distributions utilizes a simple, not very secure, authentication mechanism. When the client connects to a remote server, the MDSip connection code sends the username of the user to the server. The administrator of the server system can configure the MDSip servers to accept or deny connections based on the username of the client and the ip address of the client system making the connection. This information can also be used to map the connection to an account to use on the server system for file access permissions. Many years after the first MDSip implementation a version of the client and server software fitted with Globus certificate based authentication was developed to provide a very secure interconnect.

mdsip mode and user mapping

The user mapping for mdsip is controlled depending on the instance mode:

-s, --server	# server mode (with shared environment)
-m, --multi	# server mode (with private environment per connection)
			# session mode running on stdin/stdout, spawn of xinetd service

The mapping of remote user information to local accounts is managed by the hostfile:

-h <hostfile>		# hostfile is the path of a hostfile
-h TDI<hostfile_fun>	# hostfile_fun is a tdi hostfile function that manages the mapping
				# defaults /etc/mdsip.hosts (linux) or C:\mdsip.hosts(windows)

A tdi hostfile function may look similar to this

PUBLIC FUN hostfile(in _remote_user, in optional _hostname, in optional _althostname, ...) {
  /* e.g. map to local */
  _local_user = _remote_user;
  return(_local_user);
}

If hostfile maps multi to a user, the server process will be owned by that user (wont work on Windows).

hostfile

An ordered list of rules. It defines access permission and user mapping based on first matched rule.

# <comment>			: will be ignored
multi | <mapping>		: sets owner to <mapping> in server mode
<pattern> | <mapping>	: sets owner to <mapping> in session mode; grants access if successful, ignore otherwise
! <pattern>			: denies access if matched
<pattern>				: grants access if matched; ignored in session mode

Unmatched clients will be denied by default!

<pattern>

Defines a MDSplus wildcard string ('%': any char, '*': any number of any char);

Should match either <user>@<ip> or <user>@<hostname>. e.g.:

  • *@127.0.0.1
  • user@localhost
  • user@*

<mapping>

Defines a mechanism or a specific username.

SANDBOX		: maps to nobody and enables sandbox mode
SELF			: no mapping; uses current owner of process
MAP_TO_LOCAL	: maps to local user of same name; ignores rule if no such user
<user>		: maps to explicit user named <user>; ignores rule if no such user

example for service: session mode

JAVA_USER@*	| nobody
# deny access from specific machine
!*@192.168.1.200
# map known users from trusted subnet
*@192.168.1.*	| MAP_TO_LOCAL
# map unknown users to user with restricted access
*@192.168.1.*	| unknown
# map known users from extranet server
*@extranet	| MAP_TO_LOCAL
# rest will only have read access
*			| nobody

example: server mode

# run as system user 'mdsplus'
multi	| mdsplus
# only allow from control server
mdsplus@mds-control
# others will be denied by default

Transport "Plugins"

Attempts to provide additional authentication options and to add features such as parallel streams for increased throughput of MDSip connections over the years caused the low level MDSip code to be cluttered with conditional compilations to enable different versions of the code to be built with a wider variety of transport options. In 2010, a redesign of the MDSip code was performed to separate the MDSip message processing from the underlying transport layer. In 2011, the MDSip client/server software was rewritten based on this new design. With this new implementation, new low level transports can be deployed without any modifications to the underlying MDSip code. The choice of transport can now be specified in the connection string passed to the MdsConnect routine which now has a format similar to a http url: protocol:://protocol-specific-connection-string. If old style connection strings are used (i.e. hostname[:port] or _hostname[:port]) they default to tcp and gsi transports respectively. In the initial implementation the following transports were supported:

  • tcp://hostname[:port] - equivalent to the original hostname[:port] connection
  • gsi://hostname[:port] - Globus Security Infrastructure, equivalent to _hostname[:port]
  • ssh://[username@]hostname - Uses ssh authentication to connect to remote server w/o mdsip service.
  • sshp://[username@]hostname[:port] - Uses ssh authentication to connect to remote service running on hostname:port
  • http://hostname[:port]/MDSplusWsgi/mdsip - Uses http to connect to remote server

as other transports are added they will be listed here.

MDSip plugins: Detailed description of MDSip plugins

Custom Plugins

You can easily write you own custom plugins for any thinkable purpose. E.g. you can use plugins to simplify tunneling thru gateways.

create an executable script called 'mdsip-client-gate' in a folder included in PATH:

 #!/bin/bash
 # $1 is [user@]server[:port]
 # $2 is mdsip-server-gate
 [[ $1 =~ ^(([^@]+)@|)([^:]+)(:([0-9]+)|) ]]
 if [ -z ${BASH_REMATCH[2]} ]
 then user=$USER
 else user=${BASH_REMATCH[2]}
 fi
 server=${BASH_REMATCH[3]}
 if [ -z ${BASH_REMATCH[5]} ]
 then port=8000
 else port=${BASH_REMATCH[5]}
 fi
 exec ssh $user@mygate.mydomain.com nc $server $port

This will allow you to connect to 'mytarget' thru 'mygate.mydomain.com' using 'me' as login for gate.

 mdsconnect("gate://me@mytarget")

It is recommended to setup the connection to 'gate.mydomain.com' to be passwordless as 'me', e.g. by using key authentication.

 ssh-copy-id me@mygate.mydomain.com