Key concepts
A handful of words recur throughout this manual. The terms are listed alphabetically, so skim the names and expand any you don’t recognise. Nothing here needs to be read end to end; keep it open and look terms up while following this guide.
Batch
A processing run within a project where you ingest your metadata workbook and curate the records through SlideTap. A project can contain multiple batches — for example if your metadata arrives in parts — though a single batch is usually enough.
→ You create and complete a batch in H. Run a submission.Bundle
The DATASET_<alias> folder that is the pipeline’s finished product — the thing you validate and upload. It contains METADATA/ and IMAGES/, plus LANDING_PAGE/, ANNOTATIONS/, and PRIVATE/ where used. SlideTap writes it to the SlideTap outbox; from there you validate it and submit it to BigPicture.
Coded attribute
A metadata field whose value is a standardised code (usually SNOMED CT) rather than free text — a specimen type or a staining, for example. Your site’s local short codes are translated into coded attributes by the mapper workbook. An unmapped value leaves the attribute empty and makes the item invalid.Curation UI
The SlideTap web interface where you review and fix your metadata before submission — mapping values, catching invalid items, and correcting them in the browser. The bp-submission-pipeline path and the Direct components path both run it; only the Direct upload (sda-cli) path skips it.Dataset
The named entity within your project that becomes the dataset record in BigPicture. This is where you define secondary information: the policy, the metadata standard version, the type of dataset, and so on. The dataset’s identifier (short name / alias) becomes the folder name of your bundle and outbox.
→ You configure the dataset in H. Run a submission.DICOMize (pre- and post-process)
Converting whole-slide images into the DICOM format BigPicture requires. In the curation UI this is the Pre-process step (download and DICOMize the slides) followed by Post-process (finalise them into the export). Most of a real submission’s time is spent here.
→ Steps 5–6 of H. Run a submission.Encryption & upload tools (crypt4gh, sda-cli, bp-submitter)
The tools that encrypt the bundle and send it to BigPicture’s inbox. crypt4gh is the encryption format (encrypted files end in .c4gh); bp-submitter (used by the pipeline’s make upload) and sda-cli (the manual Direct upload path) are two alternative uploaders for the same job.
Invalid item
A record whose attribute a mapper could not resolve — the value stays None, the SlideTap validator flags it, and it blocks export until fixed. The operator’s original input isn’t lost: add the missing mapping row, re-upload mappers.xlsx, and re-map from the curation UI. Whole-slide images always show invalid until pre-processing.
Landing page
The LANDING_PAGE/ folder and its thumbnails, which appear publicly in BigPicture’s Discovery browser. SlideTap does not populate it — you choose representative, non-sensitive images by hand. landing_page.xml is optional at submission time and can be added later.
Mapper group
The named set of mappings a project uses, keyed by your institution’s short name (XLSX_MAPPER_GROUP_NAME, e.g. karolinska). You select it in Project Settings so the project resolves values against your mapper workbook.
Mapper workbook (mappers.xlsx)
A spreadsheet you fill in for your site that translates your institution’s local short codes (staining names, tissue types, preparation methods, …) into the coded attributes BigPicture expects. Each site maintains its own mapper workbook.
→ You prepare and provide this at E. Reference data.Metadata workbook
The Excel file you create for each submission. It contains the contextual metadata for your dataset — specimens, observations, whole-slide images, and their relationships. This is your data: the pipeline ingests it and uses it to build the submission. (Distinct from the mapper workbook, which is reference data you set up once.)
→ You upload this in H. Run a submission.MetaFlex
BigPicture’s metadata schema — the set of XSD files your exported metadata XML is validated against. A bundle whose XML doesn’t conform fails validation. The metadata standard version you pick in the UI (for example 2.0.0) selects which MetaFlex schema applies.
model.json
A file that describes the structure of a BigPicture-compatible dataset: what types of data it can contain (specimens, observations, images), how they relate, and your site’s attribute extensions. For most sites the provided default works unchanged.
→ You supply this file at E. Reference data.MSMDAD
BigPicture’s mandatory minimum attribute set — the floor of attributes every submission must carry. You get the entity floor (case, specimen, block, slide, image, …) automatically, even from an empty{} model.json. There is no equivalent built-in floor for observations: any MSMDAD attribute that lives inside an observation — a diagnosis code is the normal case — must still be declared in model.json by hand, or its mapper fails silently at boot and every item using it comes up unmapped. An observation missing a required MSMDAD attribute fails export. See E1. model.json for the full picture.
Project
A container you create in the curation UI that holds everything for one upload attempt — its dataset definition and one or more batches. Think of a project as “one dataset going through the pipeline.”
→ You create a project in H. Run a submission.Reference data
The two files the curation UI reads to understand your site: model.json (the attribute and observation model) and mappers.xlsx (local codes → coded attributes). You set them up once, not per submission.
→ E. Reference data.SlideTap outbox
The folder SlideTap writes when a batch finishes its export, named DATASET_<your dataset identifier>. This is the pipeline’s direct output — the bundle you then validate and upload.
SNOMED CT
The international clinical terminology whose Concept IDs BigPicture uses for coded attributes such as staining procedures and specimen types. You look codes up in the SNOMED CT Browser and record them in your mapper workbook.
→ Staining mapping.Staging vs production
BigPicture runs two upload targets. Staging is the test environment the A–J manual uploads to (staging-login.bp.nbis.se, no Terms-of-Use form). Production is the real one (login.bp.nbis.se, signed ToU required), with its own separate credentials. make upload requires you to name the TARGET.
WSI (whole-slide image)
A digitised microscopy slide — the image data at the heart of a submission. WSIs run to gigabytes each and are DICOMized during pre-processing; a real dataset can be terabytes of them.
→ Best Practices for handling large datasets.