NDA Data Submissions

Most data collected through an NIH funded grant mechanisms are required to share complete study data with the National Data Archive (NDA).

Please confirm with your program officer if your grant is required to submit data to the NDA. If so, please review requirements before data collection on the NDA website.

Once study personnel have generated GUID (global participant identifiers) for each study participant, you may proceed with neuroimaging data upload. Follow the guidance below to get started. We also highly recommend consulting with an INC staff member for guidance on the most up to date methods and standards.

We recommend working within CURC’s compute and data storage resources for NDA data submissions. INC manages a communal code repository with up-to-date NDAR tools for data submission. If users elect to use a local compute envrionment, you will need to be sure to install the prerequisite code environments before getting started.

Using CURC resources (OpenOndemand)?

Great! Skip over the next section. Quickly refresh yourself on how to access CURC’s resources in our early documentation Using Core Desktop and Setting Up Conda Environments.

module load anaconda
conda activate nda-tools-2025
cd /scratch/alpine/<identikey>

(Extra Steps) Prerequisite software packages

BIDS formatted files

Easily export all of your unprocessed nifti imaging files (and event timing, dicom files) from Flywheel.io using the command line interface.

fw export bids <target_path_for_bids> --project PROJECT --group GROUPID --subject [subjects] --session [sessions]

  • Be sure to include your event timing data! The event timing data may automatically be exported in the BIDS export or you may need to manually export depending on the project settings. If you have questions or concerns, contact INC staff

Here is an example to download a single subject using BIDS formatting.

# load conda environment (including python) on CURC
module load anaconda
conda activate nda-tools-2025

# define paths
BIDSDIR="/scratch/alpine/<identikey>/nda_submission_data/bids"
mkdir -p $BIDSDIR

# export bids data
fw export bids --group ics --project flanker --subject 01 $BIDSDIR

# check downloads
ls -l $BIDSDIR

Generate image03.csv file

Consult the NDA official documentation on the best practices of uploading neuroimaging and BIDS formatted data. Current recommendations are to upload data as image03 or fmriresults01 format. image03 data submission spreadsheets require addtional dicom related metadata to be associated with each file/manifest entry. We recommend using the dicom metadata from the main fMRI acquisition.

Sample image03 table. Notice, using the manifest requires an entry for the manifest column and requires image_file column to be left empty.

SUBJECTKEY

SRC_SUBJECT_ID

INTERVIEW_DATE

INTERVIEW_AGE

SEX

COMMENTS_MISC

IMAGE_FILE

IMAGE_THUMBNAIL_FILE

MANIFEST

IMAGE_DESCRIPTION

IMAGE_FILE_FORMAT

IMAGE_MODALITY

SCANNER_MANUFACTURER_PD

SCANNER_TYPE_PD

SCANNER_SOFTWARE_VERSIONS_PD

MAGNETIC_FIELD_STRENGTH

MRI_REPETITION_TIME_PD

FLIP_ANGLE

ACQUISITION_MATRIX

MRI_FIELD_OF_VIEW_PD

PATIENT_POSITION

PHOTOMET_INTERPRET

RECEIVE_COIL

TRANSMIT_COIL

TRANSFORMATION_PERFORMED

TRANSFORMATION_TYPE

IMAGE_HISTORY

IMAGE_NUM_DIMENSIONS

IMAGE_EXTENT1

IMAGE_EXTENT2

IMAGE_EXTENT3

IMAGE_EXTENT4

EXTENT4_TYPE

IMAGE_EXTENT5

EXTENT5_TYPE

IMAGE_UNIT1

IMAGE_UNIT2

IMAGE_UNIT3

IMAGE_UNIT4

IMAGE_UNIT5

IMAGE_RESOLUTION1

IMAGE_RESOLUTION2

IMAGE_RESOLUTION3

IMAGE_RESOLUTION4

IMAGE_RESOLUTION5

IMAGE_SLICE_THICKNESS

IMAGE_ORIENTATION

QC_OUTCOME

QC_DESCRIPTION

QC_FAIL_QUEST_REASON

PET_ISOTOPE

PET_TRACER

DECAY_CORRECTION

TIME_DIFF_INJECT_TO_IMAGE

TIME_DIFF_UNITS

FRAME_START_UNIT

FRAME_END_UNIT

DATA_FILE2

DATA_FILE2_TYPE

SCAN_TYPE

SLICE_ACQUISITION

SOFTWARE_PREPROC

PULSE_SEQ

EXPERIMENT_ID

SCAN_OBJECT

FRAME_END_TIMES

FRAME_START_TIMES

STUDY

WEEK

EXPERIMENT_DESCRIPTION

VISIT

SLICE_TIMING

BVECFILE

BVALFILE

BVEK_BVAL_FILES

DEVICESERIALNUMBER

PROCDATE

VISNUM

MRI_ECHO_TIME_PD

0

NDAR_INV25KAJM0U

NDAR_INV25KAJM0U

6/4/17

109

M

MB2 fMRI Fieldmap P

100206_3T_Diffusion_preproc_manifest.json

ABCD-fMRI-FM-PA

fMRI

MRI

Philips Medical Systems

Achieva dStream

[“5.3.0”, “5.3.0.0”]

3

7

52

[92, 0, 0, 89]

HFS

MONOCHROME2

MULTI COIL

No

0

2.4

NA

Field Map

Live

baseline_year_1_arm_1

anonb2d4

0.07

Generate manifest.json

After downloading necessary fMRI data and organizing entries in the image03 format, you will need to generate a unique manifest.json for each image03.csv file entry (generally on data collection session). Uploading fMRI data using manifest.json instead of a single zipped archive has a few advantages. It improves visibility within NDA’s registry and affords users wishing to download the dataset flexibility to select a subset of the full imaging protocol. To learn more about generate the manifest.json files, check out NDAR’s repo manifest-data.

1# load conda environment (including python) on CURC
2module load anaconda
3conda activate nda-tools-2025
4
5# define paths
6BIDSPATH="<path-to-bids>"            # path containing all BIDS data
7MANIFESTPATH="<path-to-manifests>"   # output path where manifest files are written
8SUBJECTID="sub-1001"
9python /projects/ics/software/NDA/manifest-data/nda_manifests.py -id $BIDSPATH/$SUBJECTID -of MANIFESTPATH/$SUBJECTID.json

Check out another example to pull BIDS data from Flywheel.io and generate manifest files using some BASH scripting.

 1# Example putting it all together... with a subject id list
 2FILE="subject_list.txt"
 3
 4#setup file paths
 5BIDSDIR=/scratch/alpine/$USER/nda_submission_data/bids
 6MANIFESTDIR=/scratch/alpine/$USER/nda_submission_data/manifests
 7#make bids and manifest directories if missing
 8mkdir –p $BIDSDIR
 9mkdir –p $MANIFESTDIR
10
11# Loop through file list, subject by subject
12while IFS= read -r SUBJECT; do
13   echo "Processing subject: ${SUBJECT}"
14   # strip the "sub-" prefix for flywheel
15   fw_subject=`echo $SUBJECT | cut -d"-" -f2`
16
17   # download BIDS formatted fmri data from Flywheel.io
18   fw  bids $BIDSDIR --project PROJECT --group GROUP --subject ${fw_subject}
19
20   # generate a manifest files containing all bids formated data for subject XX
21   python /projects/ics/software/NDA/manifest-data/nda_manifests.py -id $BIDSDIR/${SUBJECT} -of $/MANIFESTDIR/${SUBJECT}.json
22
23done < "$FILE"

Validate and Upload

NIMH Data Archive (NDAR) retains a set of python packages which can be used to batch upload fMRI neuroimaging data. The nda-tools code base can come with software bugs. Feel free to reach out to INC staff for support troubleshooting any nda-tools error or warning messages.

  • Start by ensuring your credentials are stored in your working environment (looking for help?). You can also just enter your password when prompted from the command line. Never share or store your password in a group readable file!

  • You are ready for the data submission! Have your image03.csv submission file handy, directory with all manifest.json files, and of course access to path with the fMRI data.

    vtcmd sample_image03.csv -m <path/to/manifests>

Once you are ready to validate and upload fMRI data to NDA, check out this example code.

 1# load conda environment (including python) on CURC
 2module load anaconda
 3conda activate nda-tools-2025
 4
 5# define paths
 6BIDSPATH="<path-to-bids>"            # path containing all BIDS data
 7MANIFESTPATH="<path-to-manifests>"   # output path where manifest files are written
 8COLLECTIONID="XXXX"                  # collection-ID can be found on NDA portal. It
 9                                     #   is a 4 digit number starting with "C". For example,
10                                     #   C1234, you will pass 1234.
11
12# validation and upload command. Test with 1 subject first!!
13# title and description should be descriptive for each unique upload (e.g. image_03_year)
14vtcmd <image03.csv> -m $MANIFESTPATH -w -j -c $COLLECTIONID -b -t <title> -d <description>

INC generally maintains a error free version of the nda-tools software package. Please contact INC staff for instructions to access the shared code environment.

Important

Users should start the process of uploading NDA data at least 6 weeks before any upload deadlines! The upload process is time consuming!