Datasets

Reference information for the multimodal data Datasets API.

eva.multimodal.data.datasets.TextImageDataset

Bases: MultimodalDataset[TextImageSample[TargetType]], TextDataset, ABC, Generic[TargetType]

Base dataset class for text-image tasks.

Parameters:

Name	Type	Description	Default
*args		Positional arguments for the base class.	()
transforms	TransformsSchema | None	The transforms to apply to the text, image and target when loading the samples.	None
**kwargs		Keyword arguments for the base class.	{}

Source code in src/eva/multimodal/data/datasets/text_image.py

def __init__(self, *args, transforms: TransformsSchema | None = None, **kwargs) -> None:
    """Initializes the dataset.

    Args:
        *args: Positional arguments for the base class.
        transforms: The transforms to apply to the text, image and target when
            loading the samples.
        **kwargs: Keyword arguments for the base class.
    """
    super().__init__(*args, **kwargs)

    self.transforms = transforms

load_image abstractmethod

Returns the image content.

Parameters:

Name	Type	Description	Default
index	int	The index of the data sample.	required

Returns:

Type	Description
Image	The image content.

Source code in src/eva/multimodal/data/datasets/text_image.py

@abc.abstractmethod
def load_image(self, index: int) -> tv_tensors.Image:
    """Returns the image content.

    Args:
        index: The index of the data sample.

    Returns:
        The image content.
    """
    raise NotImplementedError

eva.multimodal.data.datasets.PatchCamelyon

Bases: TextImageDataset[int], PatchCamelyon

PatchCamelyon image classification using a multiple choice text prompt.

Parameters:

Name	Type	Description	Default
root	str	The path to the dataset root. This path should contain the uncompressed h5 files and the metadata.	required
split	Literal['train', 'val', 'test']	The dataset split for training, validation, or testing.	required
download	bool	Whether to download the data for the specified split. Note that the download will be executed only by additionally calling the :meth:prepare_data method.	False
transforms	TransformsSchema | None	A function/transform which returns a transformed version of the raw data samples.	None
prompt	str | None	The text prompt to use for classification (multple choice).	None
max_samples	int | None	Maximum number of samples to use. If None, use all samples.	None

Source code in src/eva/multimodal/data/datasets/multiple_choice/patch_camelyon.py

def __init__(
    self,
    root: str,
    split: Literal["train", "val", "test"],
    download: bool = False,
    transforms: TransformsSchema | None = None,
    prompt: str | None = None,
    max_samples: int | None = None,
) -> None:
    """Initializes the dataset.

    Args:
        root: The path to the dataset root. This path should contain
            the uncompressed h5 files and the metadata.
        split: The dataset split for training, validation, or testing.
        download: Whether to download the data for the specified split.
            Note that the download will be executed only by additionally
            calling the :meth:`prepare_data` method.
        transforms: A function/transform which returns a transformed
            version of the raw data samples.
        prompt: The text prompt to use for classification (multple choice).
        max_samples: Maximum number of samples to use. If None, use all samples.
    """
    super().__init__(root=root, split=split, download=download, transforms=transforms)

    self.max_samples = max_samples
    self.prompt = prompt or self._default_prompt

    if self.max_samples is not None:
        self._expected_length = {split: max_samples}