CLI Restricted Endpoint Examples

The DataBiosphere CLI provides several ways for users of the data store to access and download data sets from the data store. This page covers how to access the data store using the dbio command line utility.

The CLI calls listed here are restricted to those with upload or ingest permissions.

In the document that follows, privileged user refers to a user with proper credentials and permission to upload/ingest data into the DSS.

NOTE: The Data Biosphere CLI utility is compatible with Python 3.5+.

dbio delete-bundle

Deletes an existing bundle given a UUID, version, and replica.

Example call to dbio delete-bundle:

#!/usr/bin/env bash

dbio dss delete-bundle --reason test --replica gcp --uuid 98f6c379-cb78-4a61-9310-f8cc0341c0ea --version 2019-08-02T202456.025543Z

dbio put-bundle

Creates a bundle. A bundle can contain multiple files of arbitrary type.

Inputs:

• uuid - a unique, user-created UUID.
• creator-uid - a unique user ID (uid) for the bundle creator uid. This accepts integer values.
• version - a unique, user-created version number. Use the create_verson() API function to generate a DSS_VERSION.
• replica - which replica to use (corresponds to cloud providers; choices: aws or gcp)

• files - a valid list of file objects, separated by commas (e.g., [{<first_file>}, {<second_file>}, ...  ]). Each file object must include the following details:

  • Valid UUID of the file
  • Valid version number of the file
  • Name of the file
  • Boolean value - is this file indexed

Example call to dbio put-bundle:

#!/usr/bin/env bash

dbio dss put-bundle --creator-uid 0 --replica aws --uuid 98f6c379-cb78-4a61-9310-f8cc0341c0ea --version 2019-08-02T202456.025543Z --files '{"uuid": "2196a626-38da-4489-8b2f-645d342f6aab", "version": "2019-07-10T001103.121000Z" , "name": "process_1.json", "indexed":false}'

dbio patch-bundle

Allows user to pass in an optional list of files to add or remove from an exisiting bundle.

add_files/remove_files follow this format:

[
  {
    "path": "string",
    "type": "string",
    "uuid": "string",
    "version": "string"
  }
]

Example call to dbio patch-bundle:

 #!/usr/bin/env bash

dbio dss patch-bundle --replica aws --uuid 98f6c379-cb78-4a61-9310-f8cc0341c0ea --version 2019-08-02T202456.025543Z

dbio put-file

Creates a new version of a file, given an existing UUID, version, creator uid, and source URL.

Example call to dbio put-file:

#!/usr/bin/env bash

dbio dss put-file --uuid 38f6c379-cb78-4a61-9310-f8cc0341c0eb --version 2019-07-30T164352.961501Z --creator-uid 0 --source-url s3://ucsc-cgp-dss-cli-test/930a927d-0138-4a79-8c87-e45936fe4fc3/get_bundle.json

dbio get-collection(s), dbio put-collection, dbio patch-collection, dbio delete-collection

• dbio get-collection - Given a collection UUID, get the collection.
• dbio get-collections - Get a list of collections for a given user.
• dbio delete-collection - Given a collection UUID and replica, delete the collection from the replica.
• dbio put-collection - Create a collection.
• dbio patch-collection - Add or remove a given list of files from an existing collection.

To add or remove files with the CLI actions above, specify each file in the following format:

[
  {
    "path": "string",
    "type": "string",
    "uuid": "string",
    "version": "string"
  }
]

Example CLI calls:

#!/usr/bin/env bash

info_instance=$(dbio dss put-collection --uuid fff01947-bf94-43e9-86ca-f6ff6ae45d2c --description foo --details {} --version 2018-09-17T161441.564206Z --replica aws --name bar --contents '{"path": "https://dss.dev.ucsc-cgp-redwood.org/v1/bundles/ff818282-9735-45fa-a094-e9f2d3d0a954?version=2019-08-06T170839.843085Z&replica=aws", "version": "2019-08-06T170839.843085Z", "type": "bundle", "uuid": "ff818282-9735-45fa-a094-e9f2d3d0a954"}')

ID=`echo ${info_instance} | jq -r '.uuid'`
VERSION=`echo ${info_instance} | jq -r '.version'`

dbio dss get-collections

dbio dss patch-collection --replica aws --uuid $ID --verison $VERSION

dbio dss get-collection --replica aws --uuid $ID

dbio dss delete-collection --replica aws --uuid $ID

dbio upload

Uploads a directory of files from the local filesystem and creates a bundle containing the uploaded files.

Example call to dbio upload:

#!/usr/bin/env bash

dbio dss upload --src-dir data/ --replica aws --staging-bucket upload-test-unittest

aws s3 rm s3://upload-test-unittest --recursive

Links: Index / Module Index / Search Page