Submission guide

This guide contains instructions on how to upload data to BigPicture. We will take you through the process step by step.

0. Test submission
1. Sign and send Terms of Use form
2. Install the sda-cli tool
3. Download the encryption key
4. Encrypt the files
5. Submit your files
6. Notify us

Test submission

We highly recommend that you first do a small submission to our staging environment, this way we can avoid requiring you to re-upload terabytes of data.

We recommend that you upload two images and generate metadata for them according to the preparation guide. Please include all XML files, the PRIVATE folder (always required), and the LANDING_PAGE folder (if you want a landing page).

Then follow the instructions below but when logging in to our platfrom, please use the staging login instead.

Sign and send Terms of Use form

Please sign and send the Terms of Use form (internal) and submit it to submit@bigpicture.eu.

Install the sda-cli tool

Note

This guide expects you to perform the following steps on the system where you keep the data you intend to encrypt and submit.

Linux

1. Download the sda-cli executable that matches your system from the GitHub repository.

2. Extract the binary by using the tar command:

   tar -xvzf sda-cli_.vX_Linux_x86_64.tar.gz

   The sda-cli executable should now be in the same directory as the downloaded file.

Mac

1. Download the sda-cli executable that matches your system from the GitHub repository.

2. Extract the binary by using the tar command:

   tar -xvzf sda-cli_.vX_Darwin_x86_64.tar.gz

   The sda-cli executable should now be in the same directory as the downloaded file.

Windows

1. Download the sda-cli executable that matches your system from the GitHub repository.

2. Extract the binary by using the tar command:

   tar -xvzf sda-cli_.vX_Windows_x86_64.zip

   The sda-cli executable should now be in the same directory as the downloaded file.

Note

User documentation for sda-cli is available in the GitHub repository. This guide should however include the information needed to encrypt and upload data to BigPicture.

Download the encryption key

For BigPicture to be able to read the uploaded files, they need to be encrypted with the correct public key. This key can be downloaded with the following command:

Linux

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

Mac

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

Windows

curl -OL https://raw.githubusercontent.com/NBISweden/EGA-SE-user-docs/main/crypt4gh_bp_key.pub

Encrypt the files

1. Encrypt the files

   Now that you have the public key, and the tools you need, you can encrypt the submission files. An encryption key will be created automatically by the tool.

   Linux

   ./sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

   Mac

   ./sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

   Windows

   sda-cli encrypt -key crypt4gh_bp_key.pub <file_1_to_encrypt> <file_2_to_encrypt> ...

   The tool will automatically create checksum files called:

   checksum_encrypted.md5
   checksum_encrypted.sha256
   checksum_unencrypted.md5
   checksum_unencrypted.sha256

   Make sure to save these files, you will need them during submission.

Submit your files

Once your files are encrypted, you are ready to start uploading them.

1. Obtain the configuration file

   The sda-cli tool requires a configuration file with the relevant settings. You can get the configuration file by logging in to our service (In case it’s a test, please use our staging login service).

2. Upload the files

   Files can be uploaded with or without folders. Files can be uploaded individually using:

   Linux

   ./sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

   Mac

   ./sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

   Windows

   sda-cli upload -config <configuration_file> <encrypted_file_1_to_upload> <encrypted_file_2_to_upload> ...

   Many times it might be easier to upload a directory directly though. This can be done with the -r flag:

   Linux

   ./sda-cli upload -config <configuration_file> -r <folder_to_upload>

   Mac

   ./sda-cli upload -config <configuration_file> -r <folder_to_upload>

   Windows

   sda-cli upload -config <configuration_file> -r <folder_to_upload>

   More information on the capabilites of the sda-cli can be found using the tool’s built-in help:

   Linux

   ./sda-cli help

   Mac

   ./sda-cli help

   Windows

   sda-cli help

Notify us

Once your submission is completed, please don’t forget to notify us by sending an email. The subject of the email should be in the format of:

submission - DATASET_{IDENTIFIER}*

While the email should contain:

• Name of the uploader
• Name of the dataset