Emulab Tutorial: XMLRPC Interface to Emulab
Emulab Tutorial - XMLRPC Interface to Emulab
This page describes the XMLRPC interface to Emulab. Currently, the interface mainly supports experiment creation, modification, swapping, and termination. We also provide interfaces to several other common operations on nodes end experiments such as rebooting, reloading, link delay configuration, etc. This interface is a work in progress; it will improve and grow over time. If there is something missing you need, please send us email.
The Emulab XMLRPC server can be accessed via two different transport layers.
- SSH: Yes, this is a little different then other RPC servers, but since all registered Emulab users already have accounts on Emulab and are required to use SSH to log in (and thus have provided us with their public keys), we decided this was any easy way to handle authentication.
- SSL: A more traditional SSL based server is also available. You need to request a certificate from the Emulab website in order to the SSL based server. Go to the "My Emulab" portion of the Emulab website, and click on the "Create SSL Certificate" link. You will be prompted for a passphrase to use to encrypt the private key. Once the key has been created, you will be given a link to download a text version (PEM format). Simply provide this certificate as an input to your SSL client.
The API is described in detail below. A demonstration client written in
Python is also available that you can use on your desktop to invoke
commands from the shell. For example:
sshxmlrpc_client.py startexp batch=false wait=true proj="myproj" exp="myexp" nsfilestr="`cat ~/nsfile.ns`"
sshxmlrpc_client.py -s boss.emulab.net -l rellots startexp ...
The sshxmlrpc_client python program is a simple demonstration of how to use Emulab's RPC server. If you do not provide a method and arguments on the command line, it will enter a command loop where you can type in commands (method and arguments) and wait for responses from the server. It converts your command lines into RPCs to the server, and prints out the results that the server sends back (exiting with whatever status code the server returned). You can use this client program as is, or you can write your own client program in whatever language you like, as long as you speak to the server over an SSH connection.
You may also use the SSL version of the client program, sslxmlrpc_client as long as you request an SSL certificate as described above. Once you have a certificate, use the --cert= option to tell the demonstration client where to find your certificate. You will be prompted for the private key passphrase when it attempts to contact the server.
A more sophisticated client called
script_wrapper.py provides a
traditional command line interface to Emulab, using the SSH transport
described above. The client should run on any machine that has Python
and SSH (or Putty) installed. For example, to swap the myexp
experiment in the testbed project in:
script_wrapper.py --server=boss.emulab.net swapexp -e testbed,myexp in
script_wrapper.py swapexp --help
swapexp -e pid,eid in|out
swapexp pid eid in|out
where:
-w - Wait for experiment to finish swapping
-e - Project and Experiment ID
in - Swap experiment in (must currently be swapped out)
out - Swap experiment out (must currently be swapped in)
script_wrapper.py --help
Usage: wrapper [wrapper options] command [command args and opts]
Commands:
readycount Get readycounts for nodes in experiment (deprecated).
startexp Start an Emulab experiment.
savelogs Save console tip logs to experiment directory.
endexp Terminate an experiment.
eventsys_control Start/Stop/Restart the event system.
batchexp Synonym for startexp.
node_list Print physical mapping of nodes in an experiment.
expinfo Get information about an experiment.
node_admin Boot selected nodes into FreeBSD MFS.
create_image Create a disk image from a node.
delay_config Change the link shaping characteristics for a link or lan.
modexp Modify experiment.
nscheck Check and NS file for parser errors.
swapexp Swap experiment in or out.
os_load Reload disks on selected nodes or all nodes.
portstats Get portstats from the switches.
link_config Change interface parameters for a wireless link.
node_reboot Reboot selected nodes or all nodes in an experiment.
ln -s script_wrapper.py swapexp
swapexp --server=boss.emulab.net -e testbed,myexp in
Server API
The API for the server is broken into several different modules that
export a number of methods, each of which is described below. Each
method is of the form (in Python speak):
def startexp(version, arguments):
return EmulabResponse(RESPONSE_SUCCESS, value=0, output="Congratulations")
- version: a numeric argument that the server uses to determine if the client is really capable of speaking to the server.
- arguments: a hash table of argument/value pairs,
which in Python is a Dictionary. In Perl or PHP this would be a
hashed array. Any client that supports such a datatype will be able to use
this interface directly. For example, to swap out an experiment a client
might:
args = {}; args["proj"] = "myproj" args["exp"] = "myexp" args["direction"] = "out" response = server.swapexp(CURRENTVERSION, args)
Unless specifically stated, the return value of most commands is a simple integer reflecting an exit code from the server, and some output to help you determine what went wrong. Otherwise, the return value is documented in each method description.
- code: An integer code as defined in emulabclient.py.
- value: A return value. May be any valid data type that can be transfered in XML.
- output: A string (with embedded newlines) to print out. This is useful for debugging and for guiding users through the perils of XMLRPC programming.
Finally, a quick note about the types accepted and returned by methods. Most
methods will accept a real XML-RPC type and try to coerce a string into that
type. For example, in python, passing True is equivalent to
passing the string, "true". When returning data, the methods will prefer to
return typed values, rather than formatted strings.
- /XMLRPC/emulab
The emulab module provides general information about this Emulab installation.
- news: Get news item headers. The optional
arguments are:
Name Type Default Description starting date -Inf. The date to start searching for news items (e.g. "2003-07-23 10:13:00"). ending date +Inf. The date to stop searching for news items.
The return value is a list of hash tables with the following elements:
Name Type Description subject string The item's subject. author string The item's author. date date The date the item was posted. msgid integer The item's unique identifier. This value should be used as the anchor when redirecting to the web site (e.g. http://www.emulab.net/news.php3#32).
- news: Get news item headers. The optional
arguments are:
- /XMLRPC/user
The user module provides access to user-specific information.
- nodecount: Get the number of nodes you have allocated.
There are no arguments and the method returns an integer.
- membership: Get the list of projects and subgroups of
which you are a member. The return value is a hash table where each entry is
the name of a project and a list of subgroups. The optional arguments
are:
Name Type Default Description permission string readinfo The name of an action that the user would like to take. The result will then be narrowed to those groups where this action is possible. Supported values: - readinfo - Read information about the project/group.
- createexpt - Create an experiment.
- makegroup - Create a sub-group.
- makeosid - Create an OS identifier.
- makeimageid - Create an image identifier.
- nodecount: Get the number of nodes you have allocated.
There are no arguments and the method returns an integer.
- /XMLRPC/fs
The fs module lets you examine the parts of the Emulab file system that can be exported to your experimental nodes.
- access: Check the accessibility of a path. The path
must lie on an exported file system or the call will fail. The return
value is true or false. The required arguments are:
Name Type Description path string The path to check permission string The access permission to check. Value should be one of: "read", "write", "execute", "exists".
- listdir: Get a directory list for a given path. The path
must lie on an exported file system of the call will fail. The return
value is a list of tuples for each entry, described in detail below. The
required arguments are:
Name Type Description path string The path to check
The return value is a list of tuples where each entry is organized as follows:
Index Type Description 0 string The name of the entry. 1 character The entry type, where: - 'd' - directory
- 'c' - character device
- 'b' - block device
- 'f' - regular file
- 'l' - link
- 's' - socket
- 'u' - unknown
2 integer Unix permission mask. 3 string Name of the user owner. 4 string Name of the group owner. 5 integer The size of the file. 6 integer The last access time as the number of seconds since the epoch. 7 integer The last modified time as the number of seconds since the epoch. 8 integer The creation time as the number of seconds since the epoch.
- exports: Get the root list of file system exports
available to you. This method takes no arguments and returns a list of
strings that represent the set of paths that are potentially accessible by
your experimental nodes.
Note 1: The list does not include other user directories even though they may be accessible.
Note 2: The list does include all project and group directories that you are a member of, even though the nodes will only be able to access the project/group directories the experiment was created under.
- access: Check the accessibility of a path. The path
must lie on an exported file system or the call will fail. The return
value is true or false. The required arguments are:
- /XMLRPC/experiment
The experiment module lets you start, control, and terminate experiments.
- constraints: Get the physical/policy constraints for
experiment parameters. There are no arguments and the return value is a hash
table with the following elements:
Name Type Description idle/threshold integer The maximum number of hours allowed for the idleswap parameter.
- getlist: Get the list of experiments created by a
user. There are no required arguments and the optional arguments
are:
Name Type Default Description format string "brief" The format of the data to return. The "brief" format contains a nested mapping of project names to group names and group names to a list of experiment names. The "full" format expands the experiment listing to include structures containing extra information about each experiment, instead of just the name.
- startexp: Create an experiment. By default, the experiment
is started as a batch
experiment, but you can use the batchmode option described below to
alter that behavior. You can pass an NS file inline, or you can give the
path of a file already on the Emulab fileserver.
The required arguments are:
Name Type Description proj string The Emulab project ID in which to create the experiment exp string The unique ID to call the experiment nsfilestr string A string representing the NS file to use, with embedded newlines, or, nsfilepath string The pathname of a NS file on the Emulab file server, within the project directory
(example: /proj/myproj/foo.ns)
The optional arguments are:
Name Type Default Description group string proj The Emulab subgroup ID in which to create the experiment
(defaults to project id)batch boolean true Create a batch experiment. Value is either "true" or "false" description string A pithy sentence describing your experiment swappable boolean true Experiment may be swapped at any time. If false, you must provide a reason in noswap_reason noswap_reason string A sentence describing why your experiment cannot be swapped idleswap integer variable How long (in minutes) before your idle experiment can be idle swapped. Defaults to a value between two and four hours. A value of zero means never idleswap (you must provide a reason in noidleswap_reason) noidleswap_reason string A sentence describing why your experiment cannot be idle swapped max_duration integer 0 How long (in minutes) before your experiment should be unconditionally swapped. A value of zero means never unconditionally swap this experiment. noswapin boolean false If true, do not swap the experiment in immediately; just "preload" the NS file. The experiment can be swapped in later with swapexp. wait boolean false If true, wait synchronously for the experiment to finish swapping. By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded
- metadata: Get any metadata associated with the given
experiment. The return value is a hashtable containing fields such as the
experiment's description and creation date.
The required arguments are:
Name Type Description proj string The Emulab project ID of the experiment exp string The Emulab experiment ID
- nsfile: Get the NS file associated with the given
experiment.
The required arguments are:
Name Type Description proj string The Emulab project ID of the experiment exp string The Emulab experiment ID
- swapexp: Swap an experiment in or out. The experiment
must, of course, be in the proper state for requested operation.
The required arguments are:
Name Type Description proj string The Emulab project ID of the experiment exp string The Emulab experiment ID direction string The direction in which to swap; one of "in" or "out"
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for the experiment to finish swapping in (or preloading if noswapin is true). By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded
- modify: Modify an experiment, either while it is
swapped in or out. You must provide an NS file to direct the
modification.
The required arguments are:
Name Type Description proj string The Emulab project ID of the experiment exp string The Emulab experiment ID nsfilestr string A string representing the NS file to use, with embedded newlines, or, nsfilepath string The pathname of a NS file on the Emulab file server, within the project directory
(example: /proj/myproj/foo.ns)
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for the experiment to finish swapping in (or preloading if noswapin is true). By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded reboot boolean false If true and the experiment is swapped in, reboot all nodes in the experiment restart_eventsys boolean false If true and the experiment is swapped in, restart the event system (all events are rerun from time zero)
- endexp: Terminate an experiment.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment to terminate
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for the experiment to finish terminating. By default, control returns immediately, and you must wait for email notification to determine if the operation succeeded
- state: Get the current state of the experiment.
The return value is a string; one of active, swapped, activating, etc.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment
- waitforactive: Wait for an experiment to reach
the active state. Typically, you would use this if you
launched a swapin asynchronously, and then wanted to wait later for
the swapin to complete.
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment
The optional arguments are:
Name Type Default Description timeout integer forever Timeout after this many seconds. The return code is is RESPONSE_SUCCESS if experiment reaches the active state or RESPONSE_TIMEDOUT if the timer expires.
- statewait: Wait for an experiment to reach a particular
state. State is one of swapped, active, swapping, activating, etc. If the
experiment is already in desired state, returns immediately, otherwise
blocks indefinitely until the experiment reaches the state. Use the timeout
option below to terminate the wait early. The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment state string The experiment state(s) to wait for. If a single string is specified the call will block until the experiment has reached that state. If a list of states is given, the call will block until the experiment has reached any of the states in the list. Finally, if an empty list is given, the call will block until the experiment undergoes a state change.
The optional arguments are:
Name Type Default Description timeout integer forever Timeout after this many seconds. The return code is is RESPONSE_SUCCESS if the state is reached or RESPONSE_TIMEDOUT if the timer expires.
The return value is the current state of the experiment.
- info: Get information about an experiment. The
return value is a hash table (Dictionary) of hash tables. For example,
the mapping request will return a hash indexed by node name, where
each entry is another hash table of name=value pairs, such as type=pc850
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment aspect string Request information about specific aspect of the experiment.
The aspect is one of:Name Description mapping Request the mapping of nodes in your NS file, to physical testbed nodes. This request is ignored if the experiment is not swapped in links Request information about all of the links in your experiment, including delay characteristics, IP address and mask
- nscheck: Check an NS file for obvious parser errors.
The return code is RESPONSE_SUCCESS or RESPONSE_ERROR.
The required arguments are:
Name Type Description nsfilestr string A string representing the NS file to use, with embedded newlines, or, nsfilepath string The pathname of a NS file on the Emulab file server, within the project directory
(example: /proj/myproj/foo.ns)
- delay_config: Change the link characteristics for a
delayed link or lan. Note that the link/lan must already be
delayed; you cannot convert a non-delayed link into a delayed link. When
operating on a delayed lan, all nodes (links to the nodes) in the lan
will be changed.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment link string The name of the link or lan to change; see your NS file params Dictionary A hashed array (Dictionary) of parameters to change; see below
The optional arguments are:
Name Type Default Description persist boolean false If true, the base experiment is changed in the Emulab Database; changes will persist across swapin and swapout. By default, just the physical experiment is changed, and changes are lost at swapout src string If specified, change a duplex link asymmetrically; just the link from the node specified will be changed. This option is ignored on lans; the entire lan must be changed
In addition to the required arguments, you must also supply at least one parameter to change in the params argument. The reader is encouraged to read the ipfw and dummynet man pages on users.emulab.net. It is important to note that Emulab supports a smaller set of tunable parameters then NS does; please read the aforementioned manual pages:
Name Type Range Description bandwidth integer 10-100000 Bandwidth in Kbits/second plr number 0 <= plr < 1 Packet Loss Rate as a number between 0 and 1 delay integer > 0 Delay in milliseconds limit integer Queue size in bytes or packets. Default is 50 ethernet sized packets queue-in-bytes integer 0,1 Limit is expressed in bytes or packets (slots); default is packets These are valid for RED/GRED links only maxthresh integer Maximum threshold for the average queue size thresh integer Minimum threshold for the average queue size linterm integer > 0 Packet dropping probability expressed as an integer (1/linterm) q_weight number 0 <= plr < 1 For calculating average queue size
- eventsys_control: Control the experiment event scheduler.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment op string One of start, stop, or replay (stop and then start))
- link_config: Change the link characteristics for a
wireless lan. Note that the lan must already be a wireless link; you
cannot convert wired link to a wireless link!
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment link string The name of the lan to change; see your NS file params Dictionary A hashed array (Dictionary) of parameters to change; see below
The optional arguments are:
Name Type Default Description persist boolean false If true, the base experiment is changed in the Emulab Database; changes will persist across swapin and swapout. By default, just the physical experiment is changed, and changes are lost at swapout src string If specified, change a duplex link asymmetrically; just the link from the node specified will be changed. This option is ignored on lans; the entire lan must be changed
In addition to the required arguments, you must also supply at least one parameter to change in the params argument. The reader is encouraged to read the wireless tutorial to see what parameters can be changed.
- reboot: Reboot all nodes in an experiment.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for all nodes to complete their reboot reconfig boolean false If true, do a soft reconfiguration instead of rebooting power boolean false If true, power cycle the node; do not try to reboot cleanly. Be very careful with this option!
- reload: Reload the disks on all nodes in an
experiment. You may specify an imageid to use for all nodes, or you can
allow the system to load the default imageid for each node.
The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment
The optional arguments are:
Name Type Default Description wait boolean true If true, wait synchronously for all nodes to complete their reload. The default is to wait; you must turn this off if you want the reload to proceed in the background (not a good idea) imagename string Specify the imagename to load on all of the nodes. See next option for more info imageproj string Specify the Emulab project ID of the imageid. By default the system will look in the project of the experiment, and then in the system project for globally shared images. imageid string Specify image to load using Emulab internal ID for the image reboot boolean true If false, the nodes will not be rebooted; you will need to do that yourseld. The default is true (reboot nodes)
- savelogs: Save off the console tip logs for all of the
physical nodes in your experiment. The logs are place in a subdir of your
experiment directory. The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment action string One of start, stop, or replay (stop and then start))
- thumbnail: Get the thumbnail image of the experiment
topology. The required arguments are:
Name Type Description proj string The Emulab project ID in which the experiment was created exp string The Emulab ID of the experiment
The return value is a byte array containing a PNG image.
- constraints: Get the physical/policy constraints for
experiment parameters. There are no arguments and the return value is a hash
table with the following elements:
- /XMLRPC/node
The node module lets you control nodes in your experiments.
- reboot: Reboot nodes. The caller must have
permission to reboot all of the nodes in the list, or the entire request
fails. The required arguments are:
Name Type Description nodes string A comma separated list of nodes to reboot
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for all nodes to complete their reboot reconfig boolean false If true, do a soft reconfiguration instead of rebooting power boolean false If true, power cycle the node; do not try to reboot cleanly. Be very careful with this option!
- create_image: Create an image from a node using
a previously created
imageid. The
The required arguments are:
Name Type Description node string The node to create the image from imagename string The image name (descriptor)
The optional arguments are:
Name Type Default Description wait boolean false If true, wait synchronously for all nodes to complete their reboot imageproj string emulab-ops The project ID in which the imageid was created; defaults to the system project
- reload: Reload the disks on all nodes specified.
You may specify an imageid to use for all nodes, or you can
allow the system to load the default imageid for each node.
The required arguments are:
Name Type Description nodes string A comma separated list of nodes to reload
The optional arguments are:
Name Type Default Description wait boolean true If true, wait synchronously for all nodes to complete their reload. The default is to wait; you must turn this off if you want the reload to proceed in the background (not a good idea) imagename string Specify the imageid to load on all of the nodes imageproj string Specify the Emulab project ID of the imageid. By default the system will look in the system project for globally shared images. imageid string Specify image to load using Emulab internal ID for the image reboot boolean true If false, the nodes will not be rebooted; you will need to do that yourseld. The default is true (reboot nodes)
- adminmode: Boot node into administration
mode. The node is rebooted, and its OSID is set to boot FreeBSD
from a memory resident filesystem. This allows you to operate on
the disk while the disk is offline, say to install a new OS, take a
snapshot,etc. The required arguments are:
Name Type Description mode string One of "on" or "off". Off returns the node to its previous OSID node string Node name (pcXXX)
The optional arguments are:
Name Type Default Description reboot boolean true If true, reboot the node. If false, reset the OSID, but do not reboot the node; you must do that yourself. wait boolean false If true, wait synchronously for node to complete its reboot
- reboot: Reboot nodes. The caller must have
permission to reboot all of the nodes in the list, or the entire request
fails. The required arguments are:
- /XMLRPC/osid
The osid module lets you operate on OS descriptors.
- getlist: Get the list of OS identifiers you can access. There are no arguments and the return value is a hash table containing the OS IDs and their descriptions.
- /XMLRPC/imageid
The imageid module lets you operate on Image descriptors.
- getlist: Get the list of Image identifiers you can access. There are no arguments and the return value is a hash table containing the Image IDs and their descriptions.