Systems¶

Once you are authorized to make calls to the various services, one of first things you may want to do is view storage and execution resources available to you or create your own. In Tapis a storage or execution resource is referred to as a system.

Overview¶

A Tapis system represents a server or collection of servers exposed through a single host name or IP address. Each system is associated with a specific tenant. A system can be used for the following purposes:

Running a job, including:
- Staging files to a system in preparation for running a job.
- Executing a job on a system.
- Archiving files and data on a remote system after job execution.
Storing and retrieving files and data.

Each system is of a specific type (such as LINUX or S3) and owned by a specific user who has special privileges for the system. The system definition also includes the user that is used to access the system, referred to as effectiveUserId. This access user can be a specific user (such as a service account) or dynamically specified as ${apiUserId}. For the case of ${apiUserId}, the username is extracted from the identity associated with the request to the service.

At a high level a system represents the following information:

id: A short descriptive name for the system that is unique within the tenant.
description: An optional more verbose description for the system.
systemType - Type of system: LINUX, S3 or IRODS
owner: A specific user set at system creation. By default this is the resolved value for ${apiUserId}, the user making the request to create the system.
host - Host name or IP address: FQDN or IP address
enabled - Enabled flag: Indicates if system is currently considered active and available for use. By default this is true.
effectiveUserId - Effective User: The username to use when accessing the system. Referred to as effectiveUserId. A specific user (such as a service account) or the dynamic user ${apiUserId}. By default this is ${apiUserId}.
defaultAuthnMethod - Default authentication method: How access authentication is handled by default. Authentication method can also be specified as part of a request. Supported methods: PASSWORD, PKI_KEYS, ACCESS_KEY.
bucketName - Bucket name: For an S3 system this is the name of the bucket.
rootDir - Effective root directory: Directory to be used when listing files or moving files to and from the system. For LINUX and IRODS this is required and must begin with /. For S3 this is optional and typically will not begin with /. May not be updated. Contact support to request a change.
dtnSystemId - DTN system Id: An alternate system to use as a Data Transfer Node (DTN).
dtnMountPoint - DTN mount point: Mount point (aka target) used when running the mount command on this system.
dtnMountSourcePath - DTN mount source path: The path exported by dtnSystemId that matches the dtnMountPoint on this system. This will be relative to rootDir on dtnSystemId.
isDtn: Indicates if system will be used as a data transfer node (DTN). By default this is false.
canExec: Indicates if system can be used to execute jobs.
canRunBatch: Indicates if system supports running jobs using a batch scheduler. By default this is false.
Job related attributes: Various attributes related to job execution such as jobRuntimes, jobWorkingDir, batchScheduler, batchLogicalQueues

When creating a system the required attributes are: id, systemType, host, defaultAuthnMethod and canExec. Depending on the type of system and specific values for certain attributes there are other requirements.

Note that a system may be created as a storage-only resource (canExec=false) or as a system that can be used for both execution and storage (canExec=true).

Getting Started¶

Before going into further details about Systems, here we give some examples of how to create and view systems. In the examples below we assume you are using the TACC tenant with a base URL of tacc.tapis.io and that you have authenticated using PySDK or obtained an authorization token and stored it in the environment variable JWT, or perhaps both.

Creating a System¶

Create a local file named system_example.json with json similar to the following:

{
  "id":"tacc-sample-<userid>",
  "description":"My storage system",
  "host":"tapis-vm.tacc.utexas.edu",
  "systemType":"LINUX",
  "defaultAuthnMethod":"PKI_KEYS",
  "effectiveUserId":"${apiUserId}",
  "rootDir":"/",
  "canExec": false
}

where <userid> is replaced with your username, and your host name is updated appropriately. Note that although credentials may be included in the definition we have not done so here. For security reasons, it is recommended that login credentials be updated using a separate API call as discussed below.

Using PySDK:

import json
from tapipy.tapis import Tapis
t = Tapis(base_url='https://tacc.tapis.io', username='<userid>', password='************')
with open('system_example.json', 'r') as openfile:
    my_storage_system = json.load(openfile)
t.systems.createSystem(**my_storage_system)

Using CURL:

$ curl -X POST -H "content-type: application/json" -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems -d @system_example.json

Registering Credentials for a System¶

Now that you have registered a system you will need to register credentials so you can use Tapis to access the host. Various authentication methods can be used to access a system, such as PASSWORD and PKI_KEYS. Here we will cover registering ssh keys.

Create a local file named cred_tmp.json with json similar to the following:

{
  "publicKey": "<ssh_public_key>",
  "privateKey": "<ssh_private_key>"
}

where <ssh_public_key> and <ssh_private_key> are replaced with your keys. The keys must be encoded on a single line with embedded newline characters. You may find the following linux command useful in converting a multi-line private key into a single line:

cat $privateKeyFile | awk -v ORS='\\n' '1'

Using PySDK:

t.systems.createUserCredential(systemId='tacc-sample-<userid>', userName='<userid>', publicKey='<ssh_public_key>', privateKey='<ssh_private_key>'))

Using CURL:

$ curl -X POST -H "content-type: application/json" -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems/credential/tacc-sample-<userid>/user/<userid> -d @cred_tmp.json

An optional attribute loginUser may be included in the request body in order to map the Tapis user to a username to be used when accessing the system. If the login user is not provided then there is no mapping and the Tapis user is always used when accessing the system. When a loginUser is provided the json would be similar to the following:

{
  "publicKey": "<ssh_public_key>",
  "privateKey": "<ssh_private_key>",
  "loginUser": "<linux_host_username>"
}

Note that credentials are stored in the Security Kernel. Only specific Tapis services are authorized to retrieve credentials.

Use of PKI_KEYS as credentials¶

When using an ssh keypair as credentials there are several important points to keep in mind. As discussed above, the public key and private key must be encoded on a single line. This can sometimes be challenging. For example, copying and pasting may convert newline characters in a way that is not compatible with processing in Tapis. You may find the following linux command useful in converting a multi-line private key into a single line:

cat $privateKeyFile | awk -v ORS='\\n' '1'

Also, Tapis does not currently support OPENSSH type keys. After generating the keypair, please inspect the first few lines of the private key file and confirm that it is not of type OPENSSH. Typically, a valid private key file will start with the line -----BEGIN RSA PRIVATE KEY-----. If your private key is of type OPENSSH please use a command similar to the following to generate your keypair:

ssh-keygen -t rsa -b 4096 -m PEM

When generating the keypair, do not use a passphrase. This can interfere with non-interactive use of the keypair.

Finally, please be aware that if the host has multi-factor authentication (MFA) enabled this may prevent Tapis from communicating with the host. Tapis does not currently support MFA.

When encountering problems here are some suggestions on what to check:

Public and private keys are each on one line in the json file. Newline characters in private key are properly encoded.
Keypair is not of type OPENSSH
Keypair does not have a passphrase
Public key has been added to the authorized_keys file for the target user. File ~/.ssh/authorized_keys
File ~/.ssh/authorized_keys has proper permissions.
MFA is not enabled for the target host.

If problems persist you can also attempt to manually validate the keypair using a command similar to this:

ssh -i /tmp/my_private_key testuser@myhost.com

where /tmp/my_private_key contains the original multi-line private key. If everything is set up correctly and the keypair is valid you should be logged into the host without being prompted for a password.

Viewing Systems¶

Retrieving details for a system¶

To retrieve details for a specific system, such as the one above:

Note

See the section below on Selecting to find out how to control the amount of information returned.

Using PySDK:

t.systems.getSystem(systemId='tacc-sample-<userid>')

Using CURL:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems/tacc-sample-<userid>

The response should look similar to the following:

{
   "result": {
       "tenant": "dev",
       "id": "tacc-sample-<userid>",
       "description": "My storage system",
       "systemType": "LINUX",
       "owner": "<userid>",
       "host": "tapis-vm.tacc.utexas.edu",
       "enabled": true,
       "effectiveUserId": "<userid>",
       "defaultAuthnMethod": "PKI_KEYS",
       "authnCredential": null,
       "rootDir": "/",
       "port": 22,
       "useProxy": false,
       "proxyHost": "",
       "proxyPort": -1,
       "dtnSystemId": null,
       "dtnMountPoint": null,
       "dtnMountSourcePath": null,
       "isDtn": false,
       "canExec": false,
       "canRunBatch": false,
       "jobRuntimes": [],
       "jobWorkingDir": null,
       "jobEnvVariables": [],
       "jobMaxJobs": 2147483647,
       "jobMaxJobsPerUser": 2147483647,
       "batchScheduler": null,
       "batchLogicalQueues": [],
       "batchDefaultLogicalQueue": null,
       "jobCapabilities": [],
       "tags": [],
       "notes": {},
       "uuid": "f83606bf-7a1a-4ff0-9953-dd732cc07ac0",
       "deleted": false,
       "created": "2021-04-26T18:45:40.771Z",
       "updated": "2021-04-26T18:45:40.771Z"
   },
   "status": "success",
   "message": "TAPIS_FOUND System found: tacc-sample-<userid>",
   "version": "0.0.1",
   "metadata": null
}

Note that authnCredential is null. Only specific Tapis services are authorized to retrieve credentials.

Retrieving details for all systems¶

To see the list of systems that you own:

Using PySDK:

t.systems.getSystems()

Using CURL:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems?select=allAttributes

The response should contain a list of items similar to the single listing shown above.

Note

See the sections below on Searching, Selecting, Sorting and Limiting to find out how to control the amount of information returned.

Minimal Definition and Restrictions¶

When creating a system the required attributes are: id, systemType, host, defaultAuthnMethod and canExec. Depending on the type of system and specific values for certain attributes there are other requirements. The restrictions are:

If systemType is S3 then bucketName is required, canExec and isDtn must be false.
If systemType is LINUX or IRODS then rootDir is required and must begin with /.
If effectiveUserId is ${apiUserId} (i.e. it is not static) then authnCredential may not be specified.
If isDtn is true then rootDir is required, canExec must be false and following may not be specified: dtnSystemId, dtnMountSourcePath, dtnMountPoint, all job execution related attributes.
If canExec is true then jobWorkingDir is required and jobRuntimes must have at least one entry.
If canRunBatch is true then batchScheduler must be specified.
If canRunBatch is true then batchLogicalQueues must have at least one item.
- If batchLogicalQueues has more than one item then batchLogicalDefaultQueue must be specified.
- If batchLogicalQueues has exactly one item then batchLogicalDefaultQueue is set to that item.

Permissions¶

The permissions model allows for fine grained access control of Tapis systems.

At system creation time the owner is given full access to the system. Permissions for other users may be granted and revoked through the systems API. Please note that grants and revokes through this service only impact the default role for the user. A user may still have access through permissions in another role. So even after revoking permissions through this service, when permissions are retrieved the access may still be listed. This indicates access has been granted via another role.

Permissions are specified as either * for all permissions or some combination of the following specific permissions: ("READ","MODIFY","EXECUTE"). Specifying permissions in all lower case is also allowed. Having MODIFY implies READ.

Authentication Credentials¶

At system creation time the authentication credentials may be specified if the effective access user effectiveUserId is a specific user (such as a service account) and not a dynamic user (i.e. not equal to ${apiUserId}).

If the effective access user is dynamic (i.e. equal to ${apiUserId}) then authentication credentials for any user allowed to access the system must be registered in separate API calls. In this case the payload provided may contain the optional attribute loginUser which will be used to map the Tapis user to a username to be used when accessing the system. If the login user is not provided then there is no mapping and the Tapis user is always used when accessing the system.

Note that the Systems service does not store credentials. Credentials are persisted by the Security Kernel service and only specific Tapis services are authorized to retrieve credentials.

By default any credentials provided for LINUX type systems are verified. The query parameter skipCredentialCheck=true may be used to bypass the initial verification of credentials.

Runtime¶

Runtime environment supported by the system that may be used to run applications, such as docker or singularity. Consists of the runtime type and version.

Logical Batch Queue¶

A queue that maps to a single HPC queue. Logical batch queues provide a uniform front end abstraction for an HPC queue. They also provide more features and flexibility than is typically provided by an HPC scheduler. Multiple logical queues may be defined for each HPC queue. If an HPC queue does not have a corresponding logical queue defined then a user will not be able use the Tapis system to directly submit a job via Tapis to that HPC queue.

Deletion¶

A system may be deleted and undeleted. Deletion means the system is marked as deleted and is no longer available for use. By default deleted systems will not be included in searches and operations on deleted systems will not be allowed. When listing systems the query parameter showDeleted may be used in order to include deleted systems in the results.

System Attributes Table¶

Attribute	Type	Example	Notes
tenant	String	designsafe	Name of the tenant for which the system is defined. tenant + id must be unique. Determined by the service at system creation time.
id	String	ds1.storage.default	Identifier for the system. URI safe, see RFC 3986. tenant + id must be unique. Allowed characters: Alphanumeric [0-9a-zA-Z] and special characters [-._~].
description	String	Default storage	Description
systemType	enum	LINUX	Type of system. Types: LINUX, S3, IRODS
owner	String	jdoe	username of owner. Variable references: ${apiUserId}. Resolved at create time. By default this is the resolved value for ${apiUserId}.
host	String	data.tacc.utexas.edu	Host name or ip address of the system
enabled	boolean	FALSE	Indicates if system currently enabled for use. May be updated using the enable/disable endpoints. By default this is true.
effectiveUserId	String	tg869834	User to use when accessing the system. May be a static string or a variable reference. Variable references: ${apiUserId}, ${owner} On output variable reference will be resolved.
defaultAuthnMethod	enum	PKI_KEYS	How access authentication is handled by default. Can be overridden as part of a request to get a system or credential. Methods: PASSWORD, PKI_KEYS, ACCESS_KEY
authnCredential	Credential		On input credentials to be stored in Security Kernel. effectiveUserId must be static, either a string constant or ${owner}. May not be specified if effectiveUserId is dynamic, i.e. ${apiUserId}. On output contains credential for effectiveUserId and requested authnMethod. Returned credential contains relevant information based on authnMethod. Credentials may be updated using the systems credentials endpoint. By default for LINUX the credentials are verified during create or update. Use query parameter skipCredentialCheck=true to bypass initial verification.
bucketName	String	tapis-ds1-jdoe	Name of bucket for an S3 system. Required if systemType is S3. Variable references: ${apiUserId}, ${owner}, ${tenant}
rootDir	String	/home/${apiUserId}	Required if systemType is LINUX or IRODS or isDtn = true. For LINUX or IRODS must begin with `/`. Optional for S3 and will typically not begin with `/`. Variable references are resolved at create time. Serves as effective root directory when listing or moving files. May not be updated. Contact support to request a change. Variable references: ${apiUserId}, ${owner}, ${tenant}
port	int	22	Port number used to access the system
useProxy	boolean	TRUE	Indicates if system should be accessed through a proxy.
proxyHost	String		Name of proxy host.
proxyPort	int		Port number for proxyHost
dtnSystemId	String	default.corral.dtn	An alternate system to use as a Data Transfer Node (DTN). This system and dtnSystemId must have shared storage.
dtnMountPoint	String	/gpfs/corral3/repl	Mount point (aka target) used when running the mount command on this system. Base location on this system for files transferred to rootDir on dtnSystemId.
dtnMountSourcePath	String	/gpfs/corral3/repl	Relative path defining DTN source directory relative to rootDir on dtnSystemId.
isDtn	boolean	FALSE	Indicates if system will be used as a data transfer node (DTN). By default this is false.
canExec	boolean		Indicates if system will be used to execute jobs.
canRunBatch	boolean		Indicates if system supports running jobs using a batch scheduler. By default this is false.
jobRuntimes	[Runtime]		List of runtime environments supported by the system. At least one entry required if canExec is true. Each Runtime specifies the Runtime type and version Runtime type is required and must be one of: DOCKER, SINGULARITY. Runtime version is optional.
jobWorkingDir	String	HOST_EVAL($SCRATCH)	Parent directory from which a job is run. Relative to the effective root directory rootDir. Required if canExec is true. Variable references: ${apiUserId}, ${owner}, ${tenant}
jobEnvVariables	[KeyValuePair]		Environment variables added to the shell environment in which the job is running. Added to environment variables specified in job and application definitions. Will overwrite job and application variables with same names. Each entry has a key (required) and a value (optional)
jobMaxJobs	int		Max total number of jobs . Set to -1 for unlimited.
jobMaxJobsPerUser	int		Max total number of jobs associated with a specific user. Set to -1 for unlimited.
batchScheduler	String	SLURM	Type of scheduler used when running batch jobs. Schedulers: SLURM
batchLogicalQueues	[LogicalQueue]		List of logical queues available on the system. Each logical queue maps to a single HPC queue. Multiple logical queues may be defined for each HPC queue.
batchDefaultLogical Queue	LogicalQueue		Default logical batch queue for the system.
tags	[String]		List of tags as simple strings.
notes	String	“{}”	Simple metadata in the form of a Json object.
uuid	UUID		Auto-generated by service.
deleted	boolean	FALSE	Indicates if system has been deleted. May be updated using the delete/undelete endpoints.
created	Timestamp	2020-06-19T15:10:43Z	When the system was created. Maintained by service.
updated	Timestamp	2020-07-04T23:21:22Z	When the system was last updated. Maintained by service.

Credential Attributes Table¶

Attribute	Type	Example	Notes
user	String	jsmith	Username associated with the credential.
authnMethod	String	PKI_KEYS	Indicates the authentication method associated with a retrieved credential. When a credential is retrieved it is for a specific authentication method.
loginUser	String		Optional native username valid on the system. May be used to map a Tapis user to a native login user.
password	String		Password for when authnMethod is PASSWORD.
privateKey	String		Private key for when authnMethod is PKI_KEYS.
publicKey	String		Public key for when authnMethod is PKI_KEYS.
accessKey	String		Access key used to authenticate to an S3 system.
accessSecret	String		Access secret used to authenticate to an S3 system.

LogicalQueue Attributes Table¶

Attribute	Type	Notes
name	String	Name of the tenant for which the system is defined. tenant + id must be unique. Determined by the service at system creation time.
hpcQueueName	String	Identifier for the system. URI safe, see RFC 3986. tenant + id must be unique. Allowed characters: Alphanumeric [0-9a-zA-Z] and special characters [-._~].
maxJobs	int	Maximum total number of jobs that can be queued or running in this queue.
maxJobsPerUser	int	Maximum number of jobs associated with a specific user that can be queued.
minNodeCount	int	Minimum number of nodes that can be requested when submitting a job to the queue.
maxNodeCount	int	Maximum number of nodes that can be requested when submitting a job to the queue.
minCoresPerNode	int	Minimum number of cores per node that can be requested when submitting a job. Default is 1
maxCoresPerNode	int	Maximum number of cores per node that can be requested when submitting a job.
minMemoryMB	int	Minimum memory in megabytes that can be requested when submitting a job. Default is 0
maxMemoryMB	int	Maximum memory in megabytes that can be requested when submitting a job. Default is unlimited
minMinutes	int	Minimum run time in minutes that can be requested when submitting a job. Default is 0
maxMinutes	int	Maximum run time in minutes that can be requested when submitting a job. Default is unlimited

Searching¶

The service provides a way for users to search for systems based on a list of search conditions provided either as query parameters for a GET call or a list of conditions in a request body for a POST call to a dedicated search endpoint.

Search using GET¶

To search when using a GET request to the systems endpoint a list of search conditions may be specified using a query parameter named search. Each search condition must be surrounded with parentheses, have three parts separated by the character . and be joined using the character ~. All conditions are combined using logical AND. The general form for specifying the query parameter is as follows:

?search=(<attribute_1>.<op_1>.<value_1>)~(<attribute_2>.<op_2>.<value_2>)~ ... ~(<attribute_N>.<op_N>.<value_N>)

Attribute names are given in the table above and may be specified using Camel Case or Snake Case.

Supported operators: eq neq gt gte lt lte in nin like nlike between nbetween

For more information on search operators, handling of timestamps, lists, quoting, escaping and other general information on search please see <TBD>.

Example CURL command to search for systems that have Test in the id, are of type LINUX, are using a port less than 1024 and have a default authentication method of either PKI_KEYS or PASSWORD:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems?search="(id.like.*Test*)~(system_type.eq.LINUX)~(port.lt.1024)~(DefaultAuthnMethod.in.PKI_KEYS,PASSWORD)"

Notes:

For the like and nlike operators the wildcard character * matches zero or more characters and ! matches exactly one character.
For the between and nbetween operators the value must be a two item comma separated list of unquoted values.
If there is only one condition the surrounding parentheses are optional.
In a shell environment the character & separating query parameters must be escaped with a backslash.
In a shell environment the query value must be surrounded by double quotes and the following characters must be escaped with a backslash in order to be properly interpreted by the shell:
- " \ `
Attribute names may be specified using Camel Case or Snake Case.
Following complex attributes not supported when searching:
- authnCredential jobRuntimes jobEnvVariables batchLogicalQueues notes

Dedicated Search Endpoint¶

The service provides the dedicated search endpoint systems/search/systems for specifying complex queries. Using a GET request to this endpoint provides functionality similar to above but with a different syntax. For more complex queries a POST request may be used with a request body specifying the search conditions using an SQL-like syntax.

Search using GET on Dedicated Endpoint¶

Sending a GET request to the search endpoint provides functionality very similar to that provided for the endpoint systems described above. A list of search conditions may be specified using a series of query parameters, one for each attribute. All conditions are combined using logical AND. The general form for specifying the query parameters is as follows:

?<attribute_1>.<op_1>=<value_1>&<attribute_2>.<op_2>=<value_2>)& ... &<attribute_N>.<op_N>=<value_N>

Attribute names are given in the table above and may be specified using Camel Case or Snake Case.

Supported operators: eq neq gt gte lt lte in nin like nlike between nbetween

For more information on search operators, handling of timestamps, lists, quoting, escaping and other general information on search please see <TBD>.

Example CURL command to search for systems that have Test in the name, are of type LINUX, are using a port less than 1024 and have a default authentication method of either PKI_KEYS or PASSWORD:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems/search/systems?name.like=*Test*\&enabled.eq=true\&system_type.eq=LINUX\&DefaultAuthnMethod.in=PKI_KEYS,PASSWORD

Notes:

For the like and nlike operators the wildcard character * matches zero or more characters and ! matches exactly one character.
For the between and nbetween operators the value must be a two item comma separated list of unquoted values.
In a shell environment the character & separating query parameters must be escaped with a backslash.
Attribute names may be specified using Camel Case or Snake Case.
Following complex attributes not supported when searching:
- authnCredential jobRuntimes jobEnvVariables batchLogicalQueues tags notes

Search using POST on Dedicated Endpoint¶

More complex search queries are supported when sending a POST request to the endpoint systems/search/systems. For these requests the request body must contain json with a top level property name of search. The search property must contain an array of strings specifying the search criteria in an SQL-like syntax. The array of strings are concatenated to form the full search query. The full query must be in the form of an SQL-like WHERE clause. Note that not all SQL features are supported.

For example, to search for systems that are owned by jdoe and of type LINUX or owned by jsmith and using a port less than 1024 create a local file named system_search.json with following json:

{
  "search":
    [
      "(owner = 'jdoe' AND system_type = 'LINUX') OR",
      "(owner = 'jsmith' AND port < 1024)"
    ]
}

To execute the search use a CURL command similar to the following:

$ curl -X POST -H "content-type: application/json" -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems/search/systems -d @system_search.json

Notes:

String values must be surrounded by single quotes.
Values for BETWEEN must be surrounded by single quotes.
Search query parameters as described above may not be used in conjunction with a POST request.
SQL features not supported include:
- IS NULL and IS NOT NULL
- Arithmetic operations
- Unary operators
- Specifying escape character for LIKE operator

Map of SQL operators to Tapis operators¶

Sql Operator	Tapis Operator
=	eq
<>	neq
<	lt
<=	lte
>	gt
>=	gte
LIKE	like
NOT LIKE	nlike
BETWEEN	between
NOT BETWEEN	nbetween
IN	in
NOT IN	nin

Sort, Limit, Select and ListType¶

When a list of Systems is retrieved the service provides for sorting, filtering and limiting the results. By default, only resources owned by you will be included. The service provides a way for you to request that all resources accessible to you be included. This is determined by the query parameter listType.

When retrieving either a list of resources or a single resource the service also provides a way to select which fields (i.e. attributes) are included in the results. Sorting, limiting and attribute selection are supported using query parameters.

Selecting¶

When retrieving systems the fields (i.e. attributes) to be returned may be specified as a comma separated list using a query parameter named select. Attribute names may be given using Camel Case or Snake Case.

Notes:

Special select keywords are supported: allAttributes and summaryAttributes

Summary attributes include:

id, systemType, owner, host, effectiveUserId, defaultAuthnMethod, canExec

By default all attributes are returned when retrieving a single resource via the endpoint systems/<system_id>.

By default summary attributes are returned when retrieving a list of systems.

Specifying nested attributes is not supported.

The attribute id is always returned.

For example, to return only the attributes host and effectiveUserId the CURL command would look like this:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems?select=host,effectiveUserId

The response should look similar to the following:

{
 "result": [
       {
           "id": "CSys_CltSrchGet_011",
           "host": "hostCltSrchGet_011",
           "effectiveUserId": "effUserCltSrchGet_011"
       },
       {
           "id": "CSys_CltSrchGet_012",
           "host": "hostCltSrchGet_012",
           "effectiveUserId": "effUserCltSrchGet_012"
       },
       {
           "id": "CSys_CltSrchGet_013",
           "host": "hostCltSrchGet_013",
           "effectiveUserId": "effUserCltSrchGet_013"
       }
   ],
   "status": "success",
   "message": "TAPIS_FOUND Systems found: 12 systems",
   "version": "1.0.0",
   "metadata": {
       "recordCount": 3,
       "recordLimit": 100,
       "recordsSkipped": 0,
       "orderBy": null,
       "startAfter": null,
       "totalCount": -1
   }
}

Sorting¶

The query parameter for sorting is named orderBy and the value is the attribute name to sort on with an optional sort direction. The general format is <attribute_name>(<dir>). The direction may be asc for ascending or desc for descending. The default direction is ascending.

Examples:

orderBy=id

orderBy=id(asc)

orderBy=name(desc),created

orderBy=id(asc),created(desc)

Limiting¶

Additional query parameters may be used in order to limit the number and starting point for results. This is useful for implementing paging. The query parameters are:

limit - Limit number of items returned. For example limit=10.

Use 0 or less for unlimited.

Default is 100.

skip - Number of items to skip. For example skip=10.

May not be used with startAfter.

Default is 0.

startAfter - Where to start when sorting. For example limit=10&orderBy=id(asc),created(desc)&startAfter=101

May not be used with skip.

Must also specify orderBy.

The value of startAfter applies to the major orderBy field.

Condition is context dependent. For ascending the condition is value > startAfter and for descending the condition is value < startAfter.

When implementing paging it is recommend to always use orderBy and when possible use limit+startAfter rather than limit+skip. Sorting should always be included since returned results are not guaranteed to be in the same order for each call. The combination of limit+startAfter is preferred because limit+skip is more likely to result in inconsistent results as records are added and removed. Using limit+startAfter works best when the attribute has a natural sequential ordering such as when an attribute represents a timestamp or a sequential ID.

ListType¶

By default, you will only see the resources that you own. The query parameter listType allows you to see additional resources that are available to you.

Options:

OWNED: Include only items owned by you (Default)
SHARED_PUBLIC: Include only items shared publicly
ALL: Include all items you are authorized to view.

Tapis Responses¶

For requests that return a list of resources the response result object will contain the list of resource records that match the user’s query and the response metadata object will contain information related to sorting and limiting.

The metadata object will contain the following information:

recordCount - Actual number of records returned.

recordLimit - The limit query parameter specified in the request. -1 if query parameter was not specified.

recordsSkipped - The skip query parameter specified in the request. -1 if query parameter was not specified.

orderBy - The orderBy query parameter specified in the request. Empty string if query parameter was not specified.

startAfter - The startAfter query parameter specified in the request. Empty string if query parameter was not specified.

totalCount - Total number of records that would have been returned without a limit query parameter being imposed. -1 if total count was not computed.

For performance reasons computation of totalCount is only determined on demand. This is controlled by the boolean query parameter computeTotal. By default computeTotal is false.

Example query and response:

Query:

$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/systems?limit=2&orderBy=id(desc)

Response:

{
 "result": [
   {
     "id": "testMin0",
     "systemType": "S3",
     "owner": "testuser",
     "host": "my.example.host",
     "defaultAccessMethod": "ACCESS_KEY",
     "canExec": false
   },
   {
     "id": "MinSystem1c",
     "systemType": "LINUX",
     "owner": "testuser",
     "defaultAccessMethod": "PASSWORD",
     "host": "data.tacc.utexas.edu",
     "canExec": true
   }
 ],
 "status": "success",
 "message": "TAPIS_FOUND Systems found: 2 systems",
 "version": "1.0.0",
 "metadata": {
   "recordCount": 2,
   "recordLimit": 2,
   "recordsSkipped": 0,
   "orderBy": "id(desc)",
   "startAfter": null,
   "totalCount": -1
 }

Systems¶

Overview¶

Getting Started¶

Creating a System¶

Registering Credentials for a System¶

Use of PKI_KEYS as credentials¶

Viewing Systems¶

Retrieving details for a system¶

Retrieving details for all systems¶

Minimal Definition and Restrictions¶

Permissions¶

Sharing¶

Authentication Credentials¶

Runtime¶

Logical Batch Queue¶

Deletion¶

System Attributes Table¶

Credential Attributes Table¶

LogicalQueue Attributes Table¶

Searching¶

Search using GET¶

Dedicated Search Endpoint¶

Search using GET on Dedicated Endpoint¶

Search using POST on Dedicated Endpoint¶

Map of SQL operators to Tapis operators¶

Sort, Limit, Select and ListType¶

Selecting¶

Sorting¶

Limiting¶

ListType¶

Tapis Responses¶