Files¶
The files service is the central point of interaction for all file operations in the Tapis ecosystem.
Overview¶
Through the Files service users can perform file listing, uploading, and various operations such as move, copy, mkdir and delete. The service also supports transferring files from one Tapis system to another.
Currently, the files service includes support for systems of type LINUX, S3, IRODS and GLOBUS.
Note that supported functionality varies by system type.
All file operations act upon Tapis System resources. 1 For more information on the Systems service please see Systems
Basic File Operations¶
Listing¶
Tapis supports listing files or objects on a Tapis system. The type for items listed will depend on system type. For example, for LINUX they will be posix files and for S3 they will be storage objects. See the next section below for additional considerations for S3 type systems. On S3 systems, for example, the recurse flag is ignored and all objects with keys matching the path as a prefix are always included.
For system types that support directory hierarchies the maximum recursion depth is 20.
To list the files in the effective rootDir directory of a Tapis system:
Using the official Tapis Python SDK:
t.files.listFiles(systemId="my-tapis-system", path="/")
Or using curl:
curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/ops/my-system/
And to list a sub-directory in the system, just add the path to the request:
Using CURL
curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/ops/aturing-storage/subDir1/subDir2/subDir3/
Query Parameters
- limit
integer - Max number of results to return, default of 1000
- offset
integer - Skip the first N listings
The JSON response of the API will look something like this:
{
"status": "success",
"message": "ok",
"result": [
{
"mimeType": "text/plain",
"type": "file",
"owner": "1003",
"group": "1003",
"nativePermissions": "drwxrwxr-x",
"url": "tapis://dev/aturing-storage/file1.txt",
"lastModified": "2021-04-29T16:55:57Z",
"name": "file1.txt",
"path": "file1.txt",
"size": 313
},
{
"mimeType": "text/plain",
"type": "file",
"owner": "1003",
"group": "1003",
"nativePermissions": "-rw-rw-r--",
"url": "tapis://dev/aturing-storage/file2.txt",
"lastModified": "2020-12-17T22:46:29Z",
"name": "file2.txt",
"path": "file2.txt",
"size": 21
}
],
"version": "1.1-84a31617",
"metadata": {}
}
Listings and S3 Support¶
File listings on S3 type systems have some special considerations. Objects in an S3 bucket do not have a hierarchical structure. There are no directories. Everything is an object associated with a key.
One thing to note is that, as mentioned above, for S3 the recurse flag is ignored and all objects with keys matching the path as a prefix are always included.
Note that for S3 this means that when the path is an empty string all objects in the bucket with a prefix matching rootDir will be included. This is especially important to keep in mind when using the delete operation to remove objects matching a path.
The attribute rootDir is optional for S3 type systems. When defined it will be prepended to all paths and the resulting path will become the key.
Note
When rootDir is defined for an S3 system it typically should not begin with /
.
For S3 keys are typically created and manipulated using URLs and do not have a leading /
.
Handling of symbolic links on Linux systems¶
If listing contains a symbolic link, it will show type of symbolic link:
{
"group": "1002",
"lastModified": "2023-05-09T19:53:53Z",
"mimeType": null,
"name": "x2",
"nativePermissions": "rwxrwxrwx",
"owner": "1002",
"path": "x2",
"size": 4,
"type": "symbolic_link",
"url": "tapis://mysystem/mySymLinkedFileOrDirectory"
},
If a listing for a path that is a symbolic link is requested, the symbolic link is followed, and the information is returned for path that the symbolic link points to. If the path doesn’t exist, and error will be returned. If the link points to a file, the file’s information will be returned (the type will be “file”). If the symbolic link points to a directory, the contents of that directory will be returned.
Move and Copy¶
To move or copy a file or directory using the files service, make a PUT request using the path to the current location of the file or folder.
For example, to copy a file located at /file1.txt to /subdir/file1.txt
curl -H "X-Tapis-Token: $JWT" -X PUT -d @body.json "https://tacc.tapis.io/v3/files/ops/aturing-storage/file1.txt"
with a JSON body of
{
"operation": "COPY",
"newPath": "/subdir/file1.txt"
}
Handling of symbolic links on Linux systems¶
During a move or copy, if a symbolic link is encountered, it will be handled as shown in the tables below. The first and second columns indicate whether the link is the source or target and if it points to a file or directory.
Copy
Symbolic Link |
Points To |
Notes |
---|---|---|
source |
file |
If the destination path is to a file (this means that the path ends in a component that does exist, and it’s a file or a component that does not exist):
If the destination path is to a directory (this means that the path ends in a component that does exist and it is a directory):
Note that if the link is to a relative path, moving it could change where it actually points because the exact relative path will remain the same. |
source |
directory |
If the destination path is to a file (this means that the path ends in a component that does exist, and it’s a file or a component that does not exist):
If the destination path is to a directory (this means that the path ends in a component that does exist and it is a directory):
Note that if the link is to a relative path, moving it could change where it actually points because the exact relative path will remain the same. |
destination |
file |
The destination is replaced by the source. The source could be a file, directory, or link to a file or directory. |
destination |
directory |
The new file, directory, or link is created inside of the existing directory. |
Move
Symbolic Link |
Points To |
Notes |
---|---|---|
source |
file |
If the destination path is to a file (this means that the path ends in a component that does exist, and it’s a file or a component that does not exist):
If the destination path is to a directory (this means that the path ends in a component that does exist and it is a directory):
Note that if the link is to a relative path, moving it could change where it actually points because the exact relative path will remain the same. |
source |
directory |
|
If the destination path is to a file (this means that the path ends in a component that does exist, and it’s a file or a component that does not exist):
If the destination path is to a directory (this means that the path ends in a component that does exist and it is a directory):
Note that if the link is to a relative path, moving it could change where it actually points because the exact relative path will remain the same. |
||
destination |
file |
The source link is renamed, to the destination path. The destination is replaced. The source could be a file, directory, or link to a file or directory. |
destination |
directory |
The file, directory, or link is moved inside of the existing directory. |
Making directories¶
To create a directory on a tapis system at the given path, issue a mkdir request. This is not supported for all system types. The mkdir operation is currently supported for LINUX, IRODS and GLOBUS type systems.
Using the Tapis Python SDK:
t.files.mkdir(systemId="my-system", path="/folderA/folderB/newDirectory")
Using CURL:
curl -X POST https://tacc.tapis.io/v3/files/ops/<systemId> -H "X-Tapis-Token: $JWT" -H Content-Type:application/json -d '{"path":"<directory_path"}'
Uploading¶
To upload a file use a POST request. The file will be placed at the location specified in the {path} parameter in the request. Not all system types support this operation. For example, given the system my-system, to upload file someFile.txt to directory /folderA/folderB/folderC:
Using the official Tapis Python SDK:
t.upload(source_file_path="experiment-results.hd5", system_id="my-system", dest_file_path="/folderA/folderB/folderC/someFile.txt")
curl -H "X-Tapis-Token: $JWT" -X POST -F "file=@someFile.txt" https://tacc.tapis.io/v3/files/ops/my-system/folderA/folderB/folderC/someFile.txt
For some system types (such as LINUX) any folders that do not exist in the specified path will automatically be created.
Note that for an S3 system an object will be created with a key of rootDir/{path}.
Deleting¶
To delete a file or folder, issue a DELETE request for the path to be removed.
curl -H "X-Tapis-Token: $JWT" -X DELETE "https://tacc.tapis.io/v3/files/ops/aturing-storage/file1.txt"
The request above would delete file1.txt
For an S3 system, the path will represent either a single object or all objects in the bucket with a prefix matching the system rootDir if the path is the empty string.
Warning
For an S3 system if the path is the empty string, then all objects in the bucket with a key matching the prefix rootDir will be deleted. So if the rootDir is also the empty string, then all objects in the bucket will be removed.
Handling of symbolic links and special files on Linux Systems¶
If Tapis encounters a symbolic link during a delete operation, the link will be deleted. The file or directory that the link points to will be unaffected. If the delete encounters a special file (such as a device file or fifo, etc), it will not be deleted, and an error will be returned. If this is in the middle of a recursive delete operation, some files may have been already deleted.
Creating a directory¶
To create a directory, use POST and provide the path to the new directory in the request body. Not all system types support this operation.
$ curl -H "X-Tapis-Token: $JWT" -X POST -d @body.json -X POST https://tacc.tapis.io/v3/files/ops/my-system
with a JSON body of
{
"path": "path/to/new/directory/"
}
Getting Linux stat information¶
Get native stat information for a file or directory for a system of type LINUX.
For example, for /subdir/file1.txt
curl -H "X-Tapis-Token: $JWT" "https://tacc.tapis.io/v3/files/utils/linux/aturing-storage/subdir/file1.txt"
Running a Linux native operation¶
Run a native operation on a path. Operations are chmod, chown or chgrp. For a system of type LINUX.
For example, to change the owner of a file located at /file1.txt to aeinstein
curl -H "X-Tapis-Token: $JWT" -X POST -d @body.json "https://tacc.tapis.io/v3/files/utils/linux/aturing-storage/file1.txt"
with a JSON body of
{
"operation": "CHOWN",
"argument": "aeinstein"
}
Content¶
Get file or directory contents as a stream of data. Not supported for all system types.
File Contents - Serving files¶
To return the actual contents (raw bytes) of a file:
$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/content/my-system/image.jpg > image.jpg
Query Parameters
- startByte
integer - Start at byte N of the file
- count
integer - Return this number of bytes after startByte
- zip
boolean - Zip the contents of a folder
Header Parameters
- more
integer - Return 1 KB chunks of UTF-8 encoded text from a file starting after page more. This call can be used to page through a text based file. Note that if the contents of the file are not textual (such as an image file or other binary format), the output will be bizarre.
Download using ZIP¶
The query parameter zip may be used to request a stream compressed using the ZIP file format. This is not allowed if system rootDir plus path would result in all files on the host being included. Please download individual directories, files or objects.
For example, on a linux system a directory may be downloaded as a compressed archive using a command similar to the following:
$ curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/content/my-linux-system/my_dir > my_dir.zip
The program unzip may then be used to extract the contents.
If the path being downloaded is a single file and the contents are placed in a file ending in the extension .gz then the gunzip utility may also be used to extract the contents.
Transfers¶
File transfers are used to move data between Tapis systems. They should be used for bulk data operations that are too large for the REST api to perform. Transfers occur asynchronously, and are executed concurrently where possible to increase performance. As such, the order in which the files are transferred is not deterministic.
When a transfer is initiated, a bill of materials is created that creates a record of all the files from the sourceUri that are to be transferred to the destinationUri. Unless otherwise specified, all files in the bill of materials must transfer successfully in order for the overall transfer to be considered successful. A transfer task has an attribute named status which is updated as the transfer progresses. The possible states for a transfer are:
- ACCEPTED
The initial request has been processed and saved.
- IN_PROGRESS
The bill of materials has been created and transfers are either in flight or waiting to begin.
- FAILED
The transfer failed.
- COMPLETED
The transfer completed successfully, all files have been transferred to the target system.
Unauthenticated HTTP endpoints are also possible to use as a source for transfers. This method can be utilized to include outputs from other APIs into Tapis jobs.
Note
For transfers involving Globus, both the source and destination system must be of type GLOBUS.
The number of files included in the bill of materials will depend on the system types and the sourceUri values provided in the transfer request. If the source system supports directories and sourceUri is a directory then the directory will be processed recursively and all files will be added to the bill of materials. If the source system is of type S3 then all objects matching the sourceUri path as a prefix will be included.
System types and supported functionality¶
As discussed above, the files included in a transfer will depend on the source system types and the sourceUri values provided in the transfer request. Here is a summary of the behavior:
- LINUX/IRODS to LINUX/IRODS
When the sourceUri is a directory a recursive listing is made and the files and directory structure are replicated on the destinationUri system.
- S3 to LINUX/IRODS
All objects matching the sourceUri path as a prefix will be created as files on the destinationUri system.
- LINUX/IRODS to S3
When the sourceUri is a directory a recursive listing is made. For each entry in the listing the path relative to the source system rootDir is mapped to a key for the S3 destination system. In other words, a recursive listing is made for the directory on the sourceUri system and for each non-directory entry an object is created on the S3 destinationUri system.
- S3 to S3
All objects matching the sourceUri path as a prefix will be re-created as objects on the destinationUri system.
- HTTP/S to ANY
Transfer of a directory is not supported. Destination system may not be of type GLOBUS. The content of the object from the sourceUri URL is used to create a single file or object on the destinationUri system.
- ANY to HTTP/S
Transfers not supported. Tapis does not support the use of protocol http/s for the destinationUri.
Creating Transfers¶
Lets say our user aturing
needs to transfer data between two systems that are registered in tapis. The source system
has an id of aturing-storage
with the results of an experiment located in directory /experiments/experiment-1/
that should be transferred to a system with id aturing-compute
curl -H "X-Tapis-Token: $JWT" -X POST -d @body.json https://tacc.tapis.io/v3/files/tranfers
{
"tag": "An optional identifier",
"elements": [
{
"sourceUri": "tapis://aturing-storage/experiments/experiment-1/",
"destinationUri": "tapis://aturing-compute/"
}
]
}
The request above will initiate a transfer that copies all files and folders in the experiment-1
folder on the source
system to the root directory of the destination system aturing-compute
HTTP Source¶
Unauthenticated HTTP/S endpoints can also be used as a source for a file transfer request. This can be useful, for instance, when the inputs for a job are from a separate web service, or perhaps stored in a public S3 bucket. Note that in this case the sourceUri does not refer to a Tapis system.
curl -H "X-Tapis-Token: $JWT" -X POST -d @body.json https://tacc.tapis.io/v3/files/tranfers
{
"tag": "An optional identifier",
"elements": [
{
"sourceUri": "https://some-web-application.io/calculations/12345/results.csv",
"destinationUri": "tapis://aturing-compute/inputs.csv"
}
]
}
The request above will place the output of the source URI into a file called inputs.csv
in the
aturing-compute
system.
Getting transfer information¶
To retrieve information about a transfer including status and bytes transferred, simply make a GET request to the transfers API with the UUID of the transfer.
curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/tranfers/{UUID}
The JSON response should look something like :
{
"status": "success",
"message": "ok",
"result": {
"id": 1,
"username": "aturing",
"tenantId": "tacc",
"tag": "some tag",
"uuid": "b2dcf71a-bb7b-409a-8c01-1bbs97e749fb",
"status": "COMPLETED",
"parentTasks": [
{
"id": 17,
"tenantId": "tacc",
"username": "aturing",
"sourceURI": "tapis://sourceSystem/file1.txt",
"destinationURI": "tapis://destSystem/folderA/",
"totalBytes": 100000,
"bytesTransferred": 100000,
"taskId": 1,
"children": null,
"errorMessage": null,
"uuid": "8fdccda6-a504-4ddf-9464-7b22sa66bcc4",
"status": "COMPLETED",
"created": "2021-04-22T14:21:58.933851Z",
"startTime": "2021-04-22T14:21:59.862356Z",
"endTime": "2021-04-22T14:22:09.389847Z"
}
],
"estimatedTotalBytes": 100000,
"totalBytesTransferred": 100000,
"totalTransfers": 1,
"completeTransfers": 1,
"errorMessage": null,
"created": "2021-04-22T14:21:58.933851Z",
"startTime": "2021-04-22T14:21:59.838928Z",
"endTime": "2021-04-22T14:22:09.376740Z"
},
"version": "1.1-094fd38d",
"metadata": {}
}
Handling of symbolic links on Linux Systems¶
Transfer will always “follow links”. If the source of the transfer is a symbolic link to a file or directory, it transfer the file or directory that is pointed to. If it doesn’t exist, it will be an error (i.e. the link points to something that doesn’t exist). In the case of a directory transfer where one of the entries in the directory encountered is a symbolic link it will be resolved in exactly the same way - the file is added to the archive to be downloaded, or the directory is walked, and it’s content added to the archive. Symbolic links can create situations where infinite recursion can occur - for example, suppose you have a directory with a link that points to “../”. That means that each time it’s expanded the current directory will be added, and the link will be expanded again. Transfers (and really all file operations that involve recursing subdirectories) are limited by a recursion depth. The current maximum depth is 20.
Support for Globus¶
Please note that your Tapis site installation must have been configured by the site administrator to support Globus. Please see Globus_Config.
The integration of Globus and Tapis allows users to configure and use Globus endpoints and collections just as they would other types of storage systems defined in Tapis. As mentioned previously, not all operations are supported for all system types. For systems of type GLOBUS, the following operations are supported:
listing
mkdir
move
delete
transfer between GLOBUS systems
For more information on setting up and registering credentials for a system of type GLOBUS, please see Systems_Globus.
File Permissions¶
The permissions model allows for fine grained access control of paths on a Tapis system. The system owner may grant READ and MODIFY permission to specific users. MODIFY implies READ.
Please note that Tapis permissions are independent of native permissions enforced by the underlying system host.
Getting permissions¶
Get the Tapis permissions for a user for the system and path. If no user specified then permissions are retrieved for the user making the request.
curl -H "X-Tapis-Token: $JWT" https://tacc.tapis.io/v3/files/perms/aturing-storage/experiment1?username=aeinstein
Granting permissions¶
Lets say our user aturing
has a system with ID aturing-storage
. Alan wishes to allow his collaborator
aeinstein
to view the results of an experiment located at /experiment1
curl -H "X-Tapis-Token: $JWT" -d @body.json -X POST https://tacc.tapis.io/v3/files/perms/aturing-storage/experiment1/
with a JSON body with the following shape:
{
"username": "aeinstein",
"permission": "READ"
}
Other users can also be granted permission to write to the system by granting the MODIFY
permission.
The JSON body would then be:
{
"username": "aeinstein",
"permission": "MODIFY"
}
Revoking permissions¶
Our user aturing
now wishes to revoke his former collaborators access to the folder above. He can
issue a DELETE request on the path and specify the username in order to revoke access:
curl -H "X-Tapis-Token: $JWT" -X DELETE https://tacc.tapis.io/v3/files/perms/aturing-storage/experiment1?username=aeinstein
File Path Sharing¶
In addition to fine grained permissions support, Tapis also supports a higher level approach to granting access. This approach is known simply as sharing. The sharing API allows you to share a path with a set of users as well as share publicly with all users in a tenant. Sharing a path grants users READ access to the path or, in the context of running a job, it grants users READ and MODIFY access to the path.
Please note that the underlying host associated with a system typically also has it’s own access controls.
Sharing a path with users¶
Create or update sharing information for a path on a system. The path will be shared with the list of users provided in the request body. Requester must be owner of the system. For LINUX systems path sharing is hierarchical. Sharing a path grants users READ access to the path or, in the context of running a job, it grants users READ and MODIFY access to the path.
curl -H "X-Tapis-Token: $JWT" -d @body.json -X POST https://tacc.tapis.io/v3/files/share/aturing-storage/experiment1/
with a JSON body with the following shape:
{
"users": [ "aeinstein", "rfeynman" ]
}
Sharing a path publicly¶
Share a path on a system with all users in the tenant. Requester must be owner of the system. Sharing a path grants users READ access to the path or, in the context of running a job, it grants users READ and MODIFY access to the path.
curl -H "X-Tapis-Token: $JWT" -X POST https://tacc.tapis.io/v3/files/share_public/aturing-storage/experiment1/
Unsharing a path with users¶
Update sharing information for a path on a system. The path will be unshared with the list of users provided in the request body. Requester must be owner of the system.
curl -H "X-Tapis-Token: $JWT" -d @body.json -X POST https://tacc.tapis.io/v3/files/unshare/aturing-storage/experiment1/
with a JSON body with the following shape:
{
"users": [ "rfeynman" ]
}
Unsharing a path publicly¶
Remove public sharing for a path on a system. Requester must be owner of the system.
curl -H "X-Tapis-Token: $JWT" -X POST https://tacc.tapis.io/v3/files/unshare_public/aturing-storage/experiment1/
PostIts¶
The PostIts service is a URL service that allows you to create pre-authenticated, disposable URLs to files, directories, buckets, etc in the Tapis Platform’s Files service. You have control over the lifetime and number of times the URL can be redeemed, and you can expire a PostIt at any time. The most common use of PostIts is to create URLs to files so that you can share with others without having to upload them to a third-party service.
Creating PostIts¶
To create a PostIt, send a POST request to the Files service’s Create PostIt endpoint. The url will contain the systemId and the path that will be shared. The body of the post will contain a json document which describes how long the PostIt is valid, and how many times it can be redeemed. There are default values for each of these parameters if they are not included. If the number of times the PostIt may be redeemed (allowedUses) is set to -1, the PostIt may be redeemed an unlimited number of times. The expiration (validSeconds) must always contain a value. If one is not provided, there is a default that is used. The maximum value of ‘validSeconds’ is the maximum integer value in Java (Integer.MAX_VALUE = 2147483647).
Note
The maximum value would result in a PostIt that would be valid for nearly 70 years, however it is important to remember that the authentication and authorization are built into the PostIt redeem url. This means anyone who has the url could redeem it. For this reason, it’s advisable to keep the uses and expiration times to the minimum required.
Default parameters:
allowedUses: 1 (One use)
validSeconds: 2592000 (30 days)
APPLICATION/JSON examples
Creating a postit with the default expiration and uses on a system called “tapisv3-storage” for the path “/myDirectory/myFile.txt”
Using curl
curl -X POST "https://tacc.tapis.io/v3/files/postits/tapisv3-storage/myDirectory/myFile.txt" -H "X-Tapis-Token: $JWT" -H "Content-Type: application/json"
Using python
tapis.files.createPostIt(systemId="tapisv3-storage", path="myDirectory/myFile.txt")
Creating a postit supplying expiration (validSeconds 600) and uses (allowedUses 3) on a system called “tapisv3-storage” for the path “/myDirectory/myFile.txt”
Using curl
curl -X POST "https://tacc.tapis.io/v3/files/postits/tapisv3-storage/myDirectory/myFile.txt" -H "X-Tapis-Token: $JWT" -H "Content-Type: application/json" -d '{"allowedUses": 3, "validSeconds": 600}'
Using python
tapis.files.createPostIt(systemId="tapisv3-storage", path="myDirectory/myFile.txt", allowedUses=3, validSeconds=600)
Creating a postit supplying allowing unlimited uses (allowedUses -1) and the default value for expiration (default value for validSeconds is 30 days) on a system called “tapisv3-storage” for the path “/myDirectory/myFile.txt”
Using curl
curl -X POST "https://tacc.tapis.io/v3/files/postits/tapisv3-storage/myDirectory/myFile.txt" -H "X-Tapis-Token: $JWT" -H "Content-Type: application/json" -d '{"allowedUses": -1}'
Using python
tapis.files.createPostIt(systemId="tapisv3-storage", path="myDirectory/myFile.txt", allowedUses=-1)
Example Postit Creation Response
{
"status": "success",
"message": "FAPI_POSTITS_OP_COMPLETE Operation completed. jwtTenant: tacc jwtUser: example_user OboTenant: tacc OboUser: examaple_user Operation: createPostIt System: tapisv3-storage Path: myDirectory/myFile.txt Id: e614ce8e-447c-4195-a3f7-55f5dec5d243-010",
"result": {
"id": "e614ce8e-447c-4195-a3f7-55f5dec5d243-010",
"systemId": "tapisv3-storage",
"path": "myDirectory/myFile.txt",
"allowedUses": 3,
"timesUsed": 0,
"jwtUser": "example_user",
"jwtTenantId": "tacc",
"owner": "example_user",
"tenantId": "tacc",
"redeemUrl": "https://tacc.tapis.io/v3/files/postits/redeem/e614ce8e-447c-4195-a3f7-55f5dec5d243-010",
"expiration": "2023-03-08T15:37:28.533641Z",
"created": "2023-03-08T15:27:28.534250Z",
"updated": "2023-03-08T15:27:28.534250Z"
},
"version": "1.3.1",
"commit": "0c13ee3c",
"build": "2023-03-07T21:56:41Z\n",
"metadata": {}
}
Note
The PostIt returned by the create will contain the redeemUrl. This url may be used to download the content pointed to by the PostIt. No Authentication will be done during this call. The credentials used to access this content will be the credentials of the PostIt owner. If the owner’s permissions change between creating the PostIt and redeeming the PostIt so that the owner is no longer allowed to read the content, redeeming the PostIt will fail.
Create PostIt parameters¶
Name |
Type |
Location |
Default |
Description |
---|---|---|---|---|
systemId |
String |
url |
<none> |
The systemId of the system containing the path to create the PostIt for. |
path |
String |
url |
<none> |
The path to create the PostIt for. |
allowedUses |
integer |
body |
1 |
The number of times a postit can be redeemed. Valid values are 1 - 2147483647, or -1 for unlimited uses. |
validSeconds |
integer |
body |
2147483647 |
The number of seconds from creation that the PostIt will be redeemable. An expiration time is computed by adding this value to the current date and time. |
Listing PostIts¶
PostIts can be listed by authenticated users. By default a listing of all PostIts owned by the authenticated user will be returned.
Using curl
curl "https://tacc.tapis.io/v3/files/postits" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.listPostIts()
To list all PostIts that are visible to the authenticated user, supply the query parameter listType and set it’s value to ALL. Typically users will only be able to see PostIts that they own, however tenant admins will be allowed to see all PostIts in their tenant.
Using curl
curl "https://tacc.tapis.io/v3/files/postits?listType=ALL" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.listPostIts(listType="ALL")
Paging is handled by the query parameters limit, skip, and startAfter. By default 100 PostIts are returned, however this can be changed by setting the query parameter “limit”. Skip is used to determine how many PostIts to skip. For example to get the second page of a list of PostIts containing 10 items per page, you would need to set the limit to 10 (10 items per page), and set skip to 10 (skip the first page). It’s probably a good practice to set orderBy also, so that the list is ordered in the same way each time. You could for example set orderBy to id.
Using curl
curl "https://tacc.tapis.io/v3/files/postits?listType=ALL&limit=10&skip=10&orderBy=id" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.listPostIts(listType="ALL", limit=10, skip=10, orderBy="id")
Using startAfter is similar to using skip. When using startAfter, you must provide a PostIt id, and the list will start immediately after that PostIt. You must provide the orderBy parameter. You may not use skip and startAfter together.
To control which fields are returned, you can supply the select query parameter, and select only certain fields. Setting the select query parameter to id, redeemUrl and expiration would return only those fields (select=id,redeemUrl,expiration). Setting “select” to allAttributes will return all attributes, and setting “select” to summaryAttributes will only return a preset collection of attributes. The default is summaryAttributes. You can also set the value to summaryAttributes with additional attributes (select=summaryAttributes,updated).
Using curl
curl "https://tacc.tapis.io/v3/files/postits?listType=ALL&select=id,redeemUrl,expiration" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.listPostIts(listType="ALL", select="id,redeemUrl,expiration")
Retrieving a Single PostIt¶
Retrieving a single PostIt can be done by issuing a GET request containing the id of the PostIt. This is not the same as redeeming, and does not add to the redeem count. This will allow the owner of the PostIt or a tenant admin to view the PostIt. This could be used to see the number of times it’s been retrieved, total number of uses allowed, expiration date, etc.
Using curl
curl "https://tacc.tapis.io/v3/files/postits/e614ce8e-447c-4195-a3f7-55f5dec5d243-010" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.getPostIt(postitId="e614ce8e-447c-4195-a3f7-55f5dec5d243-010")
For tenant admins, any PostIts in the tenant can be retreived in this way. For other users only PostIts that are owned by that user may be retreived, since access to the redeem url allows redemption of the PostIt.
Updating PostIts¶
The creator of a PostIt and tenant admins can update a PostIt. When updating a PostIt, the id of the posted is sent as part of the url, and a body containing allowedUses and/or validSeconds can be specified.
Note
The validSeconds parameter will add to the date and time as of the update request to compute the new expiration. It does not extend the current expiration by that many seconds.
If you need to update the url, you will need to delete or expire this PostIt and create a new one.
Update a PostIt to allow for 10 uses and to expire in 1 hour
Using curl / PATCH
curl -X PATCH "https://tacc.tapis.io/v3/files/postits/e614ce8e-447c-4195-a3f7-55f5dec5d243-010" -H "Content-Type: application/json" -H "X-Tapis-Token: $JWT" -d '{ "allowedUses":10, "validSeconds":3600 }'
Using curl / POST
curl -X POST "https://tacc.tapis.io/v3/files/postits/e614ce8e-447c-4195-a3f7-55f5dec5d243-010" -H "Content-Type: application/json" -H "X-Tapis-Token: $JWT" -d '{ "allowedUses":10, "validSeconds":3600 }'
Using python
tapis.files.updatePostIt(postitId="e614ce8e-447c-4195-a3f7-55f5dec5d243-010", allowedUses=10, validSeconds=3600)
Redeeming PostIts¶
To redeem a PostIt, use the redeemUrl from the PostIt to make a non-authenticated HTTP GET request.
Using curl
curl -JO "https://tacc.tapis.io/v3/files/postits/redeem/e614ce8e-447c-4195-a3f7-55f5dec5d243-010"
Note
The options -J and -O (specified above as -JO) tell curl to download the file content and use the filename in the content-disposition header. It’s worth noting that using this filename is not entirely without risk as it could overwrite a file of the same name. If you would prefer to use a name that you specify, you could replace -JO with --output filename
Using python
tapis.files.redeemPostIt(postitId="e614ce8e-447c-4195-a3f7-55f5dec5d243-010")
By default if the PostIt you are redeeming points to a path that is a directory, you will get a zip file, and if it’s a regular file, you will get an uncompressed file. If you want to force a file to be compressed, you can specify the query parameter zip and set it to true.
Using curl
curl -JO "https://tacc.tapis.io/v3/files/postits/redeem/e614ce8e-447c-4195-a3f7-55f5dec5d243-010?zip=true"
Using python
tapis.files.redeemPostIt(postitId="e614ce8e-447c-4195-a3f7-55f5dec5d243-010", zip=True)
Note
If you specify zip=false for a PostIt that points to a directory, you will get an error. Directories can’t be returned unless they are compressed.
The redeem URL can also be pasted into the address bar of your favorite browser, and it will download the file pointed to by the PostIt.
Expiring PostIts¶
There is no special endpoint for expiring a PostIt. To Expire a PostIt just update see Updating PostIts and set validSeconds to 0, or allowedUses to 0.
Deleting PostIts¶
PostIts can be deleted by specifying the PostIt id in the url. This is a hard delete, and cannot be undone. Only an owner or tenant admin can delete a PostIt.
Using curl
curl -X DELETE "https://tacc.tapis.io/v3/files/postits/e614ce8e-447c-4195-a3f7-55f5dec5d243-010" -H "X-Tapis-Token: $JWT"
Using python
tapis.files.deletePostIt(postitId="e614ce8e-447c-4195-a3f7-55f5dec5d243-010")
Footnotes
- 1
With the exception of the sourceUri in a transfer request when the protocol is http or https.