Submission guide

This guide contains instructions on how to upload data to BigPicture. We will take you through the process step by step.

Sign and send Terms of Use form
Install the sda-cli tool
Download the encryption key
Encrypt the files
Submit your files
Notify us

Sign and send Terms of Use form

Please sign and send the Terms of Use form (internal) and submit it to submit@bigpicture.eu.

Install the sda-cli tool

Note

This guide expects you to perform the following steps on the system where you keep the data you intend to encrypt and submit.

Download the sda-cli executable that matches your system from the GitHub repository.
Extract the binary by using the tar command:
```
tar -xvzf sda-cli_.vX_Linux_x86_64.tar.gz
```
The sda-cli executable should now be in the same directory as the downloaded file.

Download the sda-cli executable that matches your system from the GitHub repository.
Extract the binary by using the tar command:
```
tar -xvzf sda-cli_.vX_Darwin_x86_64.tar.gz
```
The sda-cli executable should now be in the same directory as the downloaded file.

Download the sda-cli executable that matches your system from the GitHub repository.
Extract the binary by using the tar command:
```
tar -xvzf sda-cli_.vX_Windows_x86_64.zip
```
The sda-cli executable should now be in the same directory as the downloaded file.

Note

User documentation for sda-cli is available in the GitHub repository. This guide should however include the information needed to encrypt and upload data to BigPicture.

Download the encryption key

For BigPicture to be able to read the uploaded files, they need to be encrypted with the correct public key. This key can be downloaded with the following command:

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

Encrypt the files

Now that you have the public key, and the tools you need, you can encrypt the submission files. An encryption key will be created automatically by the tool.

./sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

./sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

The tool will automatically create checksum files called:

checksum_encrypted.md5
checksum_encrypted.sha256
checksum_unencrypted.md5
checksum_unencrypted.sha256

Make sure to save these files, you will need them during submission.

Submit your files

Once your files are encrypted, you are ready to start uploading them.

Obtain the configuration file

The sda-cli tool requires a configuration file with the relevant settings. You can get the configuration file by logging in to our service.

Upload the files

Files can be uploaded with or without folders. Files can be uploaded individually using:

./sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

./sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

The folder structure of the uploaded files will be preserved in the remote archive.

Many times it might be easier to upload a directory directly though. This can be done with the -r flag:

./sda-cli upload -config <configuration_file> -r <folder_to_upload>

./sda-cli upload -config <configuration_file> -r <folder_to_upload>

sda-cli upload -config <configuration_file> -r <folder_to_upload>

More information on the capabilites of the sda-cli can be found using the tool’s built-in help:

./sda-cli help

./sda-cli help

sda-cli help

Notify us

Once your submission is completed, please don’t forget to notify us by sending an email to bp-ops@nbis.se. The email should contain

Name of the uploader
Name of the dataset