LION.data_loaders.LIDC_IDRI

Classes

LIDC_IDRI(mode[, operator_params, parameters])

The LIDC-IDRI dataset: lung CT scans of 1010 patients with nodules annotated by up to four radiologists, pre-processed into 512x512 slices (LIDC_IDRI/pre_process_lidc_idri.py).

Functions

choose_random_annotation(nodule_annotations_list)	One of the annotations, drawn with torch's random numbers (so rng.seeded makes the choice reproducible).
create_consensus_annotation(...)	The consensus mask of one nodule on one slice: the pixels that at least a fraction clevel of its annotations (nodule_annotations_list) mark.
format_index(index)	A patient number as LIDC-IDRI writes it, 4 digits: 7 -> "0007".
load_json(file_path)	The contents of the JSON file file_path (FileNotFoundError if there is none).
npy_problem(path)	None if the .npy file exists and holds all the data its header says, else what is wrong.
select_slices(non_nodule_slices, ...[, legacy])	Slices to load for one patient, sorted.

---

class LION.data_loaders.LIDC_IDRI.LIDC_IDRI(mode, operator_params=None, parameters=None)

Bases: LIONDataset

The LIDC-IDRI dataset: lung CT scans of 1010 patients with nodules annotated by up to four radiologists, pre-processed into 512x512 slices (LIDC_IDRI/pre_process_lidc_idri.py). Patients are split into training, validation and test (mode); each patient’s slices stay in one split.

What a sample is depends on parameters.task:

• “reconstruction”: (sinogram, image): the image in mass density, and its sinogram with the operator of operator_params (the geometry), with sinogram_transform (e.g. an experiment’s noise) applied;

• “segmentation”, “joint”, “end_to_end”: (image, mask): the mask is (background, nodule) channels, from the annotations (annotation);

• “diagnostic”: the patient’s diagnosis.

parameters (default_parameters()):

• training_proportion, validation_proportion: of the patients (0.8, 0.1; testing_proportion is the rest, recorded only);

• max_num_slices_per_patient: slices taken per patient (5; -1: all);

• pcg_slices_nodule: the fraction of them with a nodule (0.5; more of the other kind when a patient has too few);

• legacy_slice_selection, legacy_patient_list: reproduce the selection of LION before October 2026 (False);

• task, folder (the pre-processed dataset);

• operator_params: the geometry of the sinograms, for “reconstruction” (default_geometry(), for which the slices are made);

• annotation: “consensus” (the pixels at least clevel of the annotations mark; 0.5) or “random” (one annotation per nodule, chosen with annotation_seed; 0).

Samples are on LION’s device (default_device()). Cite it with cite().

The mode split (“train”, “validation” or “test”) with parameters (default_parameters(operator_params) if None); operator_params: the CT geometry, replacing parameters.operator_params if given.

Pixel spacing: the pre-processing resamples every slice to PIXEL_SPACING_MM (500 mm over 512x512 pixels, the largest field of view in the dataset, so no scan is cropped; smaller ones are surrounded by air), so all images have the same physical size. LIDC_IDRI.default_geometry() is the matching CT geometry. Datasets processed before October 2026 have each scan’s own spacing (0.46 to 0.98 mm) and no pre_processing.json: the loader warns, and python LION/data_loaders/LIDC_IDRI/pre_process_lidc_idri.py --resample-only resamples them in place (no DICOM files needed).

Damaged data: the download and pre-processing can leave incomplete files. The loader leaves out, with a warning, selected slices whose files are incomplete or unreadable, and the patients pre_processing.json lists as not resampled.

Images: for reconstruction, the CT images (Hounsfield units) are turned into mass density in g/cm^3 (ct_utils.from_HU_to_density: air 0.0012, water 1.0137, 500 HU 1.52). With LION’s geometries in mm, the sinograms are density line integrals in g/cm^3 * mm, which is what the noise model of ct_utils.sinogram_add_noise expects: the experiments (ct_experiments) add the noise.

static patient_list(path_to_processed_dataset, legacy=False)

The patients of the dataset, in order, and the number the train/validation/test proportions are taken of.

The patients are the folders named LIDC-IDRI-NNNN, so patients without a scan (0238 and 0585) or that failed pre-processing are not in the list, and other folders (e.g. a leftover LIDC-IDRI-0002.tmp) are ignored.

legacy=True is LION’s list before October 2026 (parameter files without legacy_patient_list use it): patients 1 to N, N the number of entries named LIDC-IDRI-*, with proportions of N + 1. As 0238 and 0585 have no scan, it leaves out the last two patients (1011 and 1012), and any other entry (a missing patient, a leftover folder) moves the splits.

static read_not_resampled(path_to_processed_dataset)

Patients the pre-processing could not resample (pre_processing.json).

drop_damaged_slices(slices_to_load)

Leave out, with a warning, the slices that cannot be loaded: those of patients the pre-processing could not resample, and incomplete or unreadable files (and, for tasks with masks, their mask files).

A damaged file can come from an interrupted download, pre-processing or copy. Only the selected slices are checked (their headers and sizes), so this is quick unless every slice is selected.

static read_pixel_spacing(path_to_processed_dataset)

The pixel spacing (mm) of the processed slices, or None for a dataset processed before October 2026 (each scan’s own spacing).

check_geometry()

Warn if the geometry’s pixel size is not the data’s pixel spacing: then sizes, and the physical noise model’s path lengths, are wrong.

static default_geometry(angles=None)

The fan-beam geometry of the pre-processed slices: 512x512 pixels of PIXEL_SPACING_MM (500 mm), source to centre 575 mm, source to detector 1050 mm, as LION’s default geometry, with a 1024 x 1 mm detector: it covers a 504 mm circle, like the ~50 degree fan of a clinical scanner. angles: in radians (None: 360 over 360 degrees).

static default_parameters(operator_params=None, task='reconstruction')

The parameters (see the class), for task, with the geometry operator_params: for “reconstruction”, default_geometry() if None.

get_slices_to_load(patient_list, non_nodule_slices_dict, nodule_slices_dict, num_slices_per_patient, pcg_slices_nodule)

The slices to load of each patient of patient_list: {patient: [slice, ...]}, num_slices_per_patient of them (-1: all), a fraction pcg_slices_nodule with a nodule (select_slices), from each patient’s slices without (non_nodule_slices_dict) and with (nodule_slices_dict) a nodule.

get_patient_id_to_first_index_dict(patient_with_slices_to_load)

The index of each patient’s first sample: {patient: index}, from the slices to load (patient_with_slices_to_load).

get_slice_index_to_patient_id_list(patient_with_slices_to_load)

The patient of each sample, in order: each patient once per slice it has to load.

get_reconstruction_tensor(file_path)

The slice saved at file_path, (1, 512, 512), on the dataset’s device.

set_sinogram_transform(sinogram_transform, seed=None)

A function applied to each clean sinogram, e.g. the noise of an experiment.

seed: an integer makes its random numbers depend only on the seed and the slice (patient and slice file): each slice gets the same noisy sinogram in every epoch, run, order and DataLoader worker, as a scan measures it once. None: fresh random numbers each time.

set_image_transform(image_transform)

A function applied to each image, after loading it.

compute_clean_sinogram(image=None)

The sinogram of image with the dataset’s operator (from operator_params), without noise.

get_mask_tensor(patient_id, slice_index)

The nodule mask of one slice (slice_index of patient_id): (2, 512, 512), background and nodule channels, from the annotations as annotation says; empty if the slice has no nodule.

sample_ids()

What each sample is, in order: "LIDC-IDRI-NNNN/slice_K".

sample_groups()

The group of each sample, in order: its patient. Slices of one patient are near-copies, so a patient’s slices are kept in one split, and evaluations check that no test patient was trained on.

sample_files()

The files the samples are read from, in order: each slice, and for tasks with masks the slice’s mask files.

get_specific_slice(patient_index, slice_index)

The image and mask of one slice (slice_index of patient_index), whether or not it is in the dataset’s selection.

static cite(cite_format='bib')

The LIDC-IDRI dataset: the paper and the data (TCIA), BibTeX (“bib”) or “MLA”.

LION.data_loaders.LIDC_IDRI.choose_random_annotation(nodule_annotations_list)

One of the annotations, drawn with torch’s random numbers (so rng.seeded makes the choice reproducible).

LION.data_loaders.LIDC_IDRI.create_consensus_annotation(path_to_patient_folder, slice_index, nodule_index, nodule_annotations_list, clevel)

The consensus mask of one nodule on one slice: the pixels that at least a fraction clevel of its annotations (nodule_annotations_list) mark.

LION.data_loaders.LIDC_IDRI.format_index(index)

A patient number as LIDC-IDRI writes it, 4 digits: 7 -> “0007”.

LION.data_loaders.LIDC_IDRI.load_json(file_path)

The contents of the JSON file file_path (FileNotFoundError if there is none).

LION.data_loaders.LIDC_IDRI.npy_problem(path)

None if the .npy file exists and holds all the data its header says, else what is wrong. Reads the header only.

LION.data_loaders.LIDC_IDRI.select_slices(non_nodule_slices, nodule_slices, max_slices, pcg_slices_nodule, legacy=False)

Slices to load for one patient, sorted.

Takes up to max_slices slices (all of them if -1), a fraction pcg_slices_nodule of them with a nodule. If the patient has too few slices of one kind, more of the other kind are taken; patients without nodule slices contribute non-nodule slices.

legacy=True reproduces LION before October 2026: at most min(max_slices, #non-nodule, #nodule) slices, so patients without nodule slices contributed none, and -1 meant at most 1000 of each.