H. Run a submission test

This step walks you through a complete test submission in the slidetap web UI: log in, configure a project, upload your operator workbook, process the images, curate any invalid items, and export a bundle to the outbox.

Requires: stack running (step G)

The UI must be reachable before you begin. Complete G. Preflight + boot and confirm the login page opens at the address make up printed.

1. Log in

Open the UI address in your browser (e.g. https://localhost:443). You will land on a Login page.

Enter the credentials you created in F. TLS + credentials:

  • User name — the username from make gen-credentials
  • Password — the password from make gen-credentials

Click Login. You will land on the Projects list.

2. Create a project

On the Projects list, click New project. The UI creates the project and opens its Settings page immediately. A Default batch is created automatically — you do not need to create one manually.

Two separate pages, two separate UPDATE buttons

Settings and Dataset are two different pages in the Project sidebar, each with its own UPDATE button. Filling in a field and clicking UPDATE on one page does not save anything on the other — do both parts below, in either order, and click UPDATE at the bottom of each page separately.

On the Settings page

  • General — set Project Name to something recognisable (e.g. test-submission).
  • Mapper groups — this connects the project to the mappers.xlsx mappings you placed in E. Reference data.
  • Attributes — the required fields (marked *) will already be filled in with test values. You can leave optional fields (Keywords, Geographical areas, Year of submission, etc.) blank for a test run.

Click UPDATE at the bottom of the Settings page.

On the Dataset page

Click Dataset in the Project sidebar (a separate page from Settings), and set two fields:

  • Dataset Name — a name for this dataset, distinct from the Project Name above. The Project and its Dataset are separate records that both start out called “New project”; renaming the project on the Settings page does not rename the dataset. Leave Dataset Name unset and every submission from this site exports under the same placeholder-derived bundle name, colliding with the last one. This field is what the export bundle directory and the identifier NBIS is notified about are derived from — see J. Upload to BigPicture for the exact command syntax once you reach that step — so give it something that will still identify this submission uniquely later.
  • Metadata standard — change it from its default (0.1.2) to 2.0.0. The metaflex XSD validation in I. Validate the bundle requires a 2.0.x value — submissions with the default 0.1.2 fail schema validation.

Click UPDATE at the bottom of the Dataset page.

3. Upload the workbook

In the left sidebar under Batch, click Search.

  1. Click Browse and select tests/data/metadata/metadata.xlsx — the example workbook that ships with the pipeline. Ensure its Image path already points to the slide you placed in step G.4.
  2. Click Parse.

The batch status changes to Metadata Searching while the pipeline reads the workbook and creates items. When parsing finishes, the status becomes Metadata Search Complete and the item list populates. You can also check the status under Search, where, after a refresh, the Status will read Complete.

4. Curate invalid items

Click Curate in the Batch sidebar. Items are listed by type across tabs (e.g. Sample, Image).

An invalid item is one where a mapper could not resolve an attribute — the attribute value stays None, the validator flags it as invalid, and it blocks the post-processing step downstream. See E2. mappers.xlsx for why this happens and how to fix the mapping.

To fix an invalid item:

  1. Click the row to open the detail panel. The unmapped attribute is shown in red.
  2. Add the missing mapping row to mappers.xlsx and re-upload it via E. Reference data.
  3. Select the affected items in the Curate table and click Remap to re-apply the updated mappers. No workbook re-upload is needed.

For a test run with the provided example workbook, items should already be valid if your mappers.xlsx covers the test values and the correct mapper group is selected in Project Settings.

Note: WSIs will always be invalid at this stage; they can become valid after Pre-process.

Green in Curate is not proof the submission will export

A valid (green) item means a mapper resolved a value — not that the value was actually saved. For an attribute nested inside another attribute (a diagnosis or other pathology code inside an observation is the normal case), Remap can compute the right value, mark the item valid, and still never write that value to the database. Curate cannot show you this; the item looks identical to one that is genuinely fine.

If Remap doesn’t clear an invalid item after you’ve fixed its mapping row, stop remapping it — that is the signature of a nested attribute, and remapping again will keep reporting success without fixing anything. Delete the batch and re-parse the workbook instead (upload it again at step 3). See If something goes wrong below.

The only check that confirms the submission is actually intact is a completed Export (step 9): the project status reads Export Complete, METADATA/ inside the bundle holds XML files, and IMAGES/ holds .dcm files — see the Verify callout under step 9 for exactly where to look and a caveat about make export-check. Treat every green status before that as provisional — worth knowing now, before the image conversion in the next two steps, rather than after.

5. Pre-process images

Click Pre-process in the Batch sidebar. Click the Pre-process button.

The batch status changes to Image Pre-processing while the pipeline downloads and DICOMizes the slide images. The table updates live — wait until the status reaches Image Pre-processing Complete. Alternatively, watch in the Pre-process view to see all WSIs get status Pre-Processed.

If any image shows a failed status, click Retry next to it or select multiple rows and use the bulk retry action.

6. Post-process images

Click Post-process in the Batch sidebar.

Important

The Start button is only enabled when all items are valid. If it is greyed out, return to Curate (step 4), resolve the remaining invalid items, and come back.

Click Start. The batch status changes to Image Post-processing, then Image Post-processing Complete.

7. Validate (optional)

Click Validate in the Batch sidebar to browse image thumbnails and visually confirm the slides look correct. Use the Included / Excluded toggles to show only the images you want to include in the export.

You can exclude individual slides here by clicking their thumbnail — this marks them as excluded and they will not appear in the export bundle.

8. Complete the batch

Click Complete in the Batch sidebar, then click the Complete button.

The batch is sealed. Once all batches are completed, the project status updates to Completed and the Export item in the Project sidebar becomes active.

9. Export the bundle

Click Export in the Project sidebar (under Project, not Batch). Click the Submit button.

The project status changes to Exporting, then Export Complete. The pipeline writes the export bundle to the outbox ($PIPELINE_STORAGE/storage).

make export-check currently reports FAIL on a good bundle

make export-check looks for METADATA/ and IMAGES/ directly inside the project folder, but the pipeline actually writes them one level deeper, inside a DATASET_<alias> subfolder. This is a known defect in the check itself, not a sign your export failed. Until it’s fixed, a FAIL from make export-check is inconclusive — verify the bundle by hand instead, as below.

Verify

You can still run make export-check as a quick first look:

In Git Bash, run:

make export-check
make export-check
make export-check

But because of the defect above, confirm the bundle by hand: find the project’s folder under $PIPELINE_STORAGE/storage (named <project name>.<project uid>), then check inside its DATASET_<alias> subfolder:

  • METADATA/ should contain the required XML files (at minimum: dataset.xml, image.xml, observation.xml, policy.xml, sample.xml, staining.xml). Depending on your dataset, observer.xml and annotation.xml may also be present.
  • IMAGES/ should contain .dcm files.
  • LANDING_PAGE/ and PRIVATE/ should also exist (a missing landing_page.xml is allowed; see I. Validate the bundle).

The bundle is good when the project status reads Export Complete and both folders above are populated — not when make export-check says so, while this defect is open. Once confirmed, it is ready for I. Validate the bundle.

10. Prepare the landing page thumbnails

SlideTap writes a THUMBNAILS/ folder inside the project directory (the images you browsed in step 7). These are local validation thumbnails only. The submission bundle also needs a LANDING_PAGE/THUMBNAILS/ folder inside DATASET_<alias>/ with the images you want to appear publicly in BigPicture. This is a manual step — SlideTap does not populate it automatically.

Warning

Everything under LANDING_PAGE/ is world-visible after submission. Choose representative images and avoid frames that carry sensitive information.

If something goes wrong

  • Parse produces no items. The workbook format does not match the pipeline’s expected structure. Check that you are uploading the correct .xlsx and that the sheet layout matches the import schema.
  • Images stay failed after retry. Run make logs and check the image worker output. Common causes: slide files are not reachable at the path referenced in the workbook, or DICOM conversion failed on a specific file format.
  • Post-process Start button is greyed out. One or more items are still invalid. Return to Curate, identify the invalid items, fix the mapping in mappers.xlsx, and remap.
  • Submit button is greyed out on the Export page. The project is not yet in Completed status. Confirm the batch was completed (step 8) and check the batch status chip in the Batch sidebar.
  • Export never reaches “Export Complete” — the project status just reverts to “Completed” with no visible error. An observation is missing a required coded attribute (e.g. diagnose) that has no "optional": true flag in model.json. Check make logs for a ValueError naming the missing attribute’s tag — export aborts at the first one it hits, so fixing one observation can just surface the next; expect a few rounds if several are affected. A green status in Curate doesn’t rule this out — see the warning under step 4 (Curate) above. Specifically: if the missing attribute is nested inside another attribute (an observation’s coded value, e.g. diagnose, is the normal case), Remap will not fix it, even though it reports success — delete the batch and re-upload the workbook to re-parse it (step 3) instead. For a plain, non-nested attribute, re-checking the mapping in mappers.xlsx and remapping does work. Either way, you can also mark the attribute optional in model.json instead of supplying a value — see E1. model.json. Editing model.json needs a stack restart — see When do I need to restart?.
  • export-check reports FAIL outbox not found. The PIPELINE_STORAGE path in .env does not exist or is not mounted correctly — see D. Configure.
  • export-check reports FAIL no project directories. The export did not finish writing. Check make logs and confirm the project status shows Export Complete in the UI — if it instead shows Completed, see the missing-required-attribute entry above; that failure also prevents the bundle from being written.