Start a bulk export job

📘

When exporting a file from the Platform to an attached volume, export is possible only to a volume that is in the same location (cloud provider and region) as the project from which the file is being exported.

Bulk export files from your project on the Seven Bridges Platform into your volume. One call can contain up to 100 items.

Files selected for export must not be public files or aliases. Aliases are objects stored in your cloud storage bucket which have been made available on the Platform. The volume you are exporting to must be configured for read-write access. To do this, set the access_mode parameter to RW when creating or modifying a volume.

Essentially, the call writes to your cloud storage bucket via the volume. If this call is successful, the original project files will become aliases to the newly exported objects on the volume. Source files will be deleted from the Platform and, if no more copies of the files exist, they will no longer count towards your total storage price on the Platform. In summary, once you export files from the Platform to a volume, they are no longer part of the storage on the Platform and cannot be exported again.

Learn more about using the Volumes API for for Amazon S3.

Request

https://api.sbgenomics.com/v2/bulk/storage/exports/create
https://eu-api.sbgenomics.com/v2/bulk/storage/exports/create

Example request

POST v2/bulk/storage/exports/create
Host: api.sbgenomics.com
X-SBG-Auth-Token: 3259c50e1ac5426ea8f1273259740f75
curl -X POST \
  https://api.sbgenomics.com/v2/bulk/storage/exports/create \
  -H 'Content-Type: application/json' \
  -H 'X-SBG-Auth-Token: b04aa2a2e98348a1a3a75ca677ea7fd7' \
  -d '{
    "items": [
        {
              "source": {
                "file": "5abce8051c948ec3c0ce2b7c"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output.vcf"
              },
              "overwrite": true,
              "properties": {
              "sse_algorithm": "AES256"
            }
        },
        {
              "source": {
                "file": "5abce8191c948ec3c0ce2b7f"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output.vcf"
              },
              "overwrite": false,
              "properties": {
              "sse_algorithm": "AES256"
            }
        }
         
    ]
}
'

Header Fields

Name

Description

X-SBG-Auth-Token
required

Your Seven Bridges Platform authentication token.

Content-Type
required

application/json

Query parameters

Key

Data type

Description

copy_only

Boolean

If set to true, the file will be copied to a volume but the source file will remain on the Platform.

Request body

For every item in the request body, you should enter the list of key-value pairs described in the table below. One API call can contain up to 100 items.

Key

Data type

Description

source
required

Boolean

If set to true, the file will be copied to a volume but the source file will remain on the Platform.

file
required

String

The ID of the file for export.

destination
required

Object

This object describes the destination to which the file will be exported.

volume

String

The ID of the volume to which the files will be exported.

location
required

String

Volume-specific location to which the files will be exported.
This location should be recognizable to the underlying cloud service as a valid key or path to the new files.

Please note that if this volume has been configured with a prefix parameter, the value of prefix will be prepended to location before attempting to create files on the volume.

properties

Object

Service-specific properties of the export. These values override the defaults from the volume.

sse_algorithm

String

S3 server-side encryption to use when exporting to this bucket.

Supported values:

  • AES256 (SSE-S3 encryption)
  • 'aws:kms`
  • null (no server-side encryption).
    default:AES256

sse_aws_kms_key_id
applies to type:s3

String

If AWS KMS encryption is used, this should be set to the required KMS key. If not set and aws:kms is set as sse_algorithm, default KMS key is used.

aws_canned_acl

String

If AWS KMS encryption is used, this should be set to the required KMS key. If not set and aws:kms is set as sse_algorithm, default KMS key is used.

overwrite

Boolean

If this is set to true and a named file exists in the project where the alias is about to be created, the existing file will be deleted.

default:false

Example request (normal)

POST /v2/bulk/storage/exports/create HTTP/1.1  
Host: api.sbgenomics.com
X-SBG-Auth-Token: 3259c50e1ac5426ea8f1273259740f74
Content-Type: application/json
from sevenbridges import Api

api = Api(url='https://api.sbgenomics.com/v2',
          token='3259c50e1ac5426ea8f1273259740f74')

# send POST request to SB API
exports_bulk = api.exports.bulk_submit(
    exports=[
        {
            "file": "5abce8051c948ec3c0ce2b7c",
            "volume": "rfranklin/output",
            "location": "output.vcf",
            "overwrite": True,
            "properties": {
                "sse_algorithm": "AES256"
            }
        },
        {
            "file": "5abce8191c948ec3c0ce2b7f",
            "volume": "rfranklin/output",
            "location": "output.vcf",
            "properties": {
                "sse_algorithm": "AES256"
            }
        }
    ]
)

for export_record in exports_bulk:
    if export_record.valid:
        export = export_record.resource
        print(export.id, export.state)
    else:
        print(export_record.error)
{
    "items": [
        {
              "source": {
                "file": "5abce8051c948ec3c0ce2b7c"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output.vcf"
              },
              "overwrite": true,
              "properties": {
              "sse_algorithm": "AES256"
            }
        },
        {
              "source": {
                "file": "5abce8191c948ec3c0ce2b7f"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output_2.vcf"
              },
              "overwrite": false,
              "properties": {
                "sse_algorithm": "AES256"
            }
        }
         
    ]
}

Example request (advance access - copy file only)

POST /v2/bulk/storage/exports?copy_only=true HTTP/1.1  
Host: api.sbgenomics.com
X-SBG-Auth-Token: 3259c50e1ac5426ea8f1273259740f74
Content-Type: application/json
from sevenbridges import Api

api = Api(url='https://api.sbgenomics.com/v2',
          token='3259c50e1ac5426ea8f1273259740f74',
          advance_access=True)

# send POST request to SB API
exports_bulk = api.exports.bulk_submit(
    exports=[
        {
            "file": "5abce8051c948ec3c0ce2b7c",
            "volume": "rfranklin/output",
            "location": "output.vcf",
            "overwrite": True,
            "properties": {
                "sse_algorithm": "AES256"
            }
        },
        {
            "file": "5abce8191c948ec3c0ce2b7f",
            "volume": "rfranklin/output",
            "location": "output.vcf",
            "properties": {
                "sse_algorithm": "AES256"
            }
        }
    ],
    copy_only=True
)

for export_record in exports_bulk:
    if export_record.valid:
        export = export_record.resource
        print(export.id, export.state)
    else:
        print(export_record.error)
{
    "items": [
        {
              "source": {
                "file": "5abce8051c948ec3c0ce2b7c"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output.vcf"
              },
              "overwrite": true,
              "properties": {
              "sse_algorithm": "AES256"
            }
        },
        {
              "source": {
                "file": "5abce8191c948ec3c0ce2b7f"
              },
              "destination": {
                "volume": "rfranklin/output",
                "location": "output.vcf"
              },
              "overwrite": false,
              "properties": {
              "sse_algorithm": "AES256"
            }
        }
         
    ]
}

Response

See a list of Seven Bridges Platform-specific response codes that may be contained in the body of the response.

Response body

The response object contains information about all of the exported items. The information is structured using the following key-value pairs:

Key

Data type

Description

id

string

ID of this import job.

state

string

The state of this import job. Possible values are:

  • PENDING: the import is queued;
  • RUNNING: the import is running;
  • COMPLETED: the import has completed successfully;
  • FAILED: the import has failed.

source

object

Import source, as passed when this job was started by the call to start an import job.

destination

object

Import destination, as passed when this job was start an import job.

result

object

File object that was imported.

error

object

In case of error in the import job, standard API error is returned here.

Example response body

{
    "items": [
        {
            "resource": {
                "href": "https://api.sbgenomics.com/v2/storage/exports/MiNOzItzc9HpRfZxoGp3YTZE8STJMZmI",
                "id": "MiNOzItzc9HpRfZxoGp3YTZE8STJMZmI",
                "state": "PENDING",
                "source": {
                    "file": "5abce8051c948ec3c0ce2b7c"
                },
                "destination": {
                    "volume": "rfranklin/output",
                    "location": "output.vcf"
                },
                "properties": {
                    "sse_algorithm": "AES256",
                    "sse_aws_kms_key_id": null,
                    "aws_canned_acl": null
                },
                "overwrite": true
            }
        },
        {
            "resource": {
                "href": "https://api.sbgenomics.com/v2/storage/exports/1sKqNbEI8nrb3TGui6OD3ATRz1XxDRIi",
                "id": "1sKqNbEI8nrb3TGui6OD3ATRz1XxDRIi",
                "state": "PENDING",
                "source": {
                    "file": "5abce8191c948ec3c0ce2b7f"
                },
                "destination": {
                    "volume": "rfranklin/output.vcf",
                    "location": "output.vcf"
                },
                "properties": {
                    "sse_algorithm": "AES256",
                    "sse_aws_kms_key_id": null,
                    "aws_canned_acl": null
                },
                "overwrite": false
            }
        }
    ]
}