Documentation:Tutorial:MDS ServerOSX - MdsWiki
Navigation
Personal tools

From MdsWiki

Jump to: navigation, search

Contents

Preamble

As I (intrator@lanl.gov) was setting up an MDSplus server on OSX, I ran into differences between what I needed for OSX and the mostly well known recipes for LINUX. This document is primarily about the differences (mostly paths and different implementation of xinetd). This document includes extra information, with some redundant explanations where I thought some redundancy would reduce the confusion on first reading, assembled in one place, about OSX server peccadillos. All this took some rooting around to figure out.

Most other generic MDS information that is not specific to OSX (tree structure, setting up trees, subtrees, networked daemons for processing data, etc) is omitted. Maybe the next iteration.

UNIX script text is a different font (italics) than the narrative text, or in the yellow boxes. We post everything with the caveat that you use it at your own risk.

Get a compiled MDSplus server for OSX

OSX 10.5.8 and earlier

The following is predicated on using Darren Garnier’s mdsplus-2.0.1-macosx-universal build, which you can get from him with a query <garnier@MIT.EDU>. There are several newer distributions from Darren, but they all posed problems with Traverser, due I surmise to Java libraries, Motif libraries, 64 bit compatibility. We think these problems disappear with OS10.6. I have tested this stuff on my machine using OS10.5.7, Intel MacBookPro, CoreDuo 32bit.

OSX 10.6.3 and earlier

I have tested mdsplus-2.2-4-universal-10.5 on SnowLeopard 10.6.3, and it seems to work OK. 64bit libraries are included, so that for example IDL calls in TCL and 64 bit mode dont choke.

OSX 10.6.7 and later

There is a new distro for OSX at, which seems to work for 10.6.7, 10.6.8, 10.7.4 so far. I installed the version named (Oct 14, 2012 MDSplus-4-0-14-osx.pkg). Thank you to whoever you are that is developing this. It can be found at

http://www.mdsplus.org/index.php/Latest_Macintosh_Distributions

as of 2012Oct17, I found one error in the

/Library/LaunchDaemons/org.mdsplus.mdsipd.plist

where it looks for

/usr/local/mdsplus/local/mdsip.hosts

but instead should look for

/usr/local/mdsplus/etc/mdsip.hosts

This error prevents the server from being contacted remotely. You will have to edit (as sudo) one argument in the plist to fix this.

You may have to 1) unload and reload launchctl (see below) 2) force the 64 bit Java to load first, using the Java Preferences application utility.

OSX 10.8.3 and later

The more recent distros of MDSPlus install seem to install everything you will need except the file /etc/mdsip.hosts

Last installation I tried is MDSplus-5-1-4-osx.pkg on OSX 10.8.5 which seems to work OK.

This MDS wiki webpage has a lot of useful information
http://www.mdsplus.org/index.php?url=mdsplus/download.php&page=Software%2FDownloads

It tells you where on the web to find the following which must be installed:

- OpenMotif
- XCode Command Line Tools
- FreeTDS
- Xquartz - OS-X 10.8
- Java Run Time (JRT) once from Oracle and once from Apple.

General information about MDSplus

Information about MDSplus is available at [1], although documentation is not always complete. A very useful TechX website with MDSplus server install information exists for generic (I think mostly LINUX, but this link seems to have vanished 2010jan5) servers at [2]

update 2013-01-11

bug in the latest mdsplus release for osx. This prevents one from connecting locally to the local server. Connection works ok from local server looking at other remote servers, but not for localhost.

Created Package Info Jan 10, 2013 MDSplus-5-0-2-osx.pkg

here is the error log message

Fri Jan 11 13:44:04 2013 (0) (pid 6725) Connection received from rsx_analysis@yorick.lanl.gov [128.165.120.104] Unable to open hostfile one: /usr/local/mdsplus/local/mdsip.hosts

PROBLEM: mds is searching and failing to find the file - /usr/local/mdsplus/local/mdsip.hosts

FIX: I copied the mdsip.hosts filefrom - /usr/local/mdsplus/etc to this local subdirectory

Set up the server

Set up the DAEMON

add a log directory:

cd /usr/local/mdsplus
mkdir logs

Set permissions - Owner, group and world are the 3 categories of user for which you set permissions. {read = 22 , write=21, execute=20} correspond to the three flavors of permissions. For example: owner=rwx (7), group=rx (5), world=rx (5) permissions should be set for the log directory

chmod 755 logs

You can add the following line to /usr/local/mdsplus/etc/mdsip.hosts

*@*.excessive_technology.com | nobody

where you replace exessive_technology.com with the domain that you want to serve. Or leave the following 3 lines untouched

/O=Grid/O=National Fusion Collaboratory/OU=MIT/CN=Thomas W. Fredian/Email=twf@psfc.mit.edu | twf
* | MAP_TO_LOCAL
* | nobody

There is a daemon script (/usr/local/mdsplus/bin/mdsipd), that runs OK as is.

NOTE: the setup.sh script requires that it runs from the top mdsplus directory, so one should cd to that directory first.

Add data trees

Copy or put data tree files into (for example) /usr/local/mdsplus/expt_data_files, making certain that the individual shot files have 755 (-rwxr-xr-x) permissions. For example, if you start with shot 00001 for the tree named this_expt, then the full file paths would read:

/usr/local/mdsplus/expt_data_files/this_expt_00001.tree
/usr/local/mdsplus/expt_data_files/this_expt_00001.datafile
/usr/local/mdsplus/expt_data_files/this_expt_00001.characteristics

The syntax for files inside the tree is tree_name_shotnumber.*. The file /usr/local/mdsplus/etc/envsyms tells the server which directory to look inside of for the data in the tree. The "tree_name" for this example data set will then be this_expt. You can have several trees corresponding to several experiments if you wish.

Define path to your trees

You can put the tree(s) anywhere you want, but you must tell MDSplus where to look for them. Edit this file

/usr/local/mdsplus/etc/envsyms

And for this example path, /usr/local/mdsplus/expt_data_files, append the following line(s) to the end

this_expt_path /usr/local/mdsplus/expt_data_files >;

Paths that are no longer needed or temporary can be commented out

#this_expt_path /usr/local/mdsplus/expt_data_files >;

Set up ownership and permissions for tree files

You may wish to declare a new group, for example mds_admin that has read and write permission, and everyone else with only read permissions. This can be done with chown commands. Lately, groups are declared in the System Preferences, by adding a 'user' that you can declare to be a group, then add members to it.

fix up your .bash_profile file.

I use these commands for bash:

echo "Adding MDSplus path and variables"
source /usr/local/mdsplus/setup.sh

Configure network services (this would be XINETD in LINUX)

However in OSX, xinetd is launched by launchd, with a utility called launchctl, which has an XML plist with parameters. The manual pages for launchd.plist and launchctl have a lot of information (more than you want to know) about this. Files that control launchd are

FILES

The mdsip.hosts file is site specific. It lets you control who can connect and what account a remote connect uses to access the data. The /etc/mdsip.hosts location was used on linux based platforms for many years, and also on OSX. Note that this is an absolute path, NOT relative to $MDSPLUS_DIR/, i.e. you must copy from $MDSPLUS_DIR/etc/mdsip.hosts or /usr/local/mdsplus/etc/mdsip.hosts to /etc/mdsip.hosts.

The 2.2.4 install package will automatically write the file /Library/LaunchDaemons/org.mdsplus.mdsipd.plist . The functionality of the various launchd files depends on where you put them, as outlined below. Otherwise you will have to write this file yourself. Later builds of MDSPlus installs for OSX take care of this for you.

/Library/LaunchAgents            Per-user agents provided by the administrator.
~/Library/LaunchAgents          Per-user agents provided by the user.
/Library/LaunchDaemons         System-wide daemons provided by the administrator.
/System/Library/LaunchAgents Per-user agents provided by Mac OS X.
/System/Library/LaunchDaemons  System-wide daemons provided by Mac OS X.

The MDSPlus download page has a lot of useful information that is more current ... http://www.mdsplus.org/index.php/Latest_Macintosh_Distributions

launchd requires a plist

information from launchd.plist(5) BSD File Formats Manual, for OS10.6

NAME
    launchd.plist -- System wide and per-user daemon/agent configuration files
DESCRIPTION
    This document details the parameters that can be given to an XML property list that can be loaded into
    launchd with launchctl.

XML PROPERTY LIST KEYS

Keys can be used to describe the configuration details of your daemon or agent. Property lists are Apple's standard configuration file format. Please see plist(5) for more information. Please note: property list files are expected to have their name end in ".plist". Also please note that it is the expected convention for launchd property list files to be named <Label>.plist. Thus, if your job label is "com.apple.sshd", your plist file should be named "com.apple.sshd.plist".

XML PLIST

The following file is a plist (/Library/LaunchDaemons/org.mdsplus.mdsipd.plist) that seems to work. The plist provides the arguments $1, $2, and could provide optional arguments $3, $4 that are inputs to the shell script

/usr/local/mdsplus/bin/mdsipd

which invokes the binary

/usr/local/mdsplus/bin/mdsip

Thanks to Darren Garnier garnier@MIT.EDU for the xml plist. NOTE -- this is different from LINUX, where internet daemons and xinetd are controlled from the file /etc/xinetd.d/mdsip

######### XML PLIST BELOW ###################
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
  <string>org.mdsplus.mdsipd</string>
<key>ProgramArguments</key>
  <array>
    <string>/usr/local/mdsplus/bin/mdsipd</string>
    <string>8000</string>
    <string>/var/log/mdsplus</string>
  </array>
 <key>Sockets</key>
     <dict>
 <key>Listeners</key>
     <dict>
       <key>Bonjour</key>
         <string>mdsipd</string>
       <key>SockServiceName</key>
         <string>8000</string>
     </dict>
     </dict>
 <key>inetdCompatibility</key>
   <dict>
     <key>Wait</key>
       <false/>
   </dict>
   </dict>
</plist>
############################

Do not forget to delete the hashmarks above, since these are evidently valid comment symbols for xml.

This is for a "one server per connection", and seems to work with "Wait = false" not "true". This XML list advertises the server on Bonjour. Not that anyone has written Bonjour (ZeroConf, Avahi) browser for MDSplus servers...

Load and unload launchd, launchctl

You will need to run:

sudo launchctl load -w /Library/LaunchDaemons/org.mdsplus.mdsipd.plist

Or, to unload (and later reload after you change something)

sudo launchctl unload -w /Library/LaunchDaemons/org.mdsplus.mdsipd.plist

The sudo command will request a sudo or root password.

File permissions and extended attributes

Apple file system sometimes uses “extended attributes”. If you have downloaded the plist from the web, and/or written it with for instance BBEdit, there will be additional file attributes, signified by an appended "@" sign. Check using ls -l on your directory. If so, the launchctl command thinks that the ownership of the plist is ambiguous.

%%%% example %%%%%%%%%%%%%%%%%%%%%%%

 intrator$ ls -l org.mdsplus.mdsipd.plist
-rw-r--r--@ 1 intrator  admin  660 Sep 26 09:17 org.mdsplus.mdsipd.plist

use the xattr command to list the other "extended attributes".

intrator$ xattr -l org.mdsplus.mdsipd.plist
com.apple.FinderInfo:
0000   54 45 58 54 00 00 00 00 00 00 00 00 00 00 00 00    TEXT............
0010   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................

%%%%%%%%%%%%%%%%%%%%%%%%%%%

Remove the extended attributes with command xattr,

intrator$ xattr -d com.apple.FinderInfo org.mdsplus.mdsipd.plist

and check with ls -l to see if the extended attributes have disappeared. %%%%%%

Network name of the local server

The name of your local machine is localhost 8000. The default port for MDSplus is 8000. You can test the server in IDL, or using ssh, or telnet.

Test the server

Through OS 10.5.x

Retrieve data sets: example in IDL, using the proper syntax (note the single quote locations).

IDL> mdsconnect, 'localhost' 
% Compiled module: MDSCONNECT.
% Compiled module: MDSDISCONNECT.
IDL> mdsopen, 'frxl', 6257
% Compiled module: MDSOPEN.
% Compiled module: MDSVALUE.
% Compiled module: MDSCHECKARG.
% Compiled module: MDSISCLIENT.
IDL> data = mdsvalue('dim_of(\THETA_ROGOWS)') 
IDL> time = mdsvalue('dim_of(\THETA_ROGOWS)')

caveat for SnowLeopard users 10.6.x and build 2.2.4

Note also that the mdsconnect, 'localhost' is not supposed to work, according the mavens, because the localhost does not actually establish an IP connection. It seems to work for builds 2.0 installed on Leopard 10.5.x, but on the other hand does not work for build 2.2.4 installed on SnowLeopard. The fix is to append a line in your .bash_profile file in the home directory, that defines the path to whichever tree you wish to look at. For example, for the tree mtf_data, I might use something like

export mtf_data_path="/AFRL_frchx_data_trees/MDS_FRCHX_data"

Then skip the mdsconnect command, and inside IDL, use directly the mdsopen command

latest build for OSX 10.8.3 tested and works OK

Created Package Info Jul 15, 2013 MDSplus-5-0-21-osx.pkg

jTraverser

jTraverser or better yet Traverser allows one to look at tree structure:

jTraverser seems to work on Garnier mdsplus-2.0.1-macosx-universal build example input for FRXL data (FRXL is one LANL experiment)

menu -- file
input --- tree 
   -------------- frxl (note, no need for quotes on the tree name)
input --- shot 
   -------------- 3903


I have not yet figured out how to get the server to serve jTraverser to a remote machine.

Traverser has problems with OSX

Traverser complains about missing dyld. For example the error messages I get are:

zenith:~ intrator$ traverser
dyld: Library not loaded: /opt/local/lib/libMrm.4.dylib
  Referenced from: /usr/local/mdsplus/bin/traverser
  Reason: image not found
Trace/BPT trap

%%%%%%

Supposedly installation of Motif will allow traverser to work. I am sure it is simple, but have not yet figured out the right build and compile sequence. There are several newer distributions from Darren, but they all posed problems with Traverser (Java libraries, Motif libraries, 64 bit compatibility, problems which I think disappear with OS10.6). Might need 64 bit hardware (later Apple Intel laptops after CoreDuo, most Intel G5).

jScope

Use jScope to look at the data.

java memory

Sometimes large jScope displays consume lots of memory. I dont think this is just an OSX issue. One obscure improvement for the Java entails increasing the memory allocation to 512MB or more. The file is

/usr/local/mdsplus/bin/jScope 

Here is an example file

#!/bin/sh
#!
#! display MDSplus javascope
#
java=`which java`
if ( uname -p | grep 64 > /dev/null )
then
  if (file -L $java } grep 64i-bit > /dev/null )
  then 
    lib=lib64
  else
    lib=lib32
  fi
else
  lib=lib
fi
env \
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$MDSPLUS_DIR/${lib} \
SHLIB_PATH=$SHLIB_PATH:$MDSPLUS_DIR/${lib} \
DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$MDSPLUS_DIR/${lib} \
CLASSPATH=$HOME:$MDSPLUS_DIR/java/classes/jScope.jar:$MDSPLUS_DIR/java/classes \
$java -Xmx512M jScope $1 $2 $3 $4 $5 $6 $7

Increase memory to 512MB by editing the last line in this file to read:

$java -Xmx512M jScope $1 $2 $3 $4 $5 $6 $7

I dont know if even larger memory helps much. You may need to fiddle with the file permissions prior to editing.

editing the signals

To edit signals

  • use the expand expression button
  • right click to 'set up data source'
  • look in the signals list
  • highlight 'Select this item to add new expression' line
  • write in what you wish in Y or X, or other fillable field in the form
  • hit return, and the entry should show up in the signals list
  • multiple signals
    • shot sequence can be specified in jScope shot window, eg
      • 2009082709:2009082712 should yield 4 shots, with sequenced colors
      • [2009082709,2009082711] should yield 2 shots, with sequenced colors

To delete a signal,

  • highlight 'Select this item to add new expression' line
  • use control-delete, not delete as per jScope documentation

IDL and TCL issues

TCL commands can be accessed via IDL. For example

idl> mdstcl, 'set tree test_tree/shot=4'

will open test_tree for viewing but not for edit, shot 4.

Some of the MDS plus builds are for 32 bit architecture, and IDL will launch by default into 64 bit mode, for DualCore and later Apple machines (ie these are 64bit cpu's). If you get error messages, then try launching idl in 32bit mode:

intrator$ idl -32 
You can find out the architecture of MDSPlus with following command in the terminal:

file /usr/local/mdsplus/lib/libMdsLibIdl.so

If you have the universal build, you will see something like: /usr/local/mdsplus/lib/libMdsLibIdl.so: Mach-O universal binary with 2 architectures /usr/local/mdsplus/lib/libMdsLibIdl.so (for architecture i386): Mach-O bundle i386 /usr/local/mdsplus/lib/libMdsLibIdl.so (for architecture ppc): Mach-O bundle ppc

Troubleshooting

If problems arise open up Console application (inside utilities folder) and look at the mdsplus logs. There should be files:

/private/var/log/mdsplus/access

/private/var/log/mdsplus/errors