Datasets

Reference information for the multimodal data Datasets API.

eva.multimodal.data.datasets.TextImageDataset

Bases: MultimodalDataset[TextImageSample[TargetType]], TextDataset, ABC, Generic[TargetType]

Base dataset class for text-image tasks.

Parameters:

Name	Type	Description	Default
*args		Positional arguments for the base class.	()
transforms	TransformsSchema | None	The transforms to apply to the text, image and target when loading the samples.	None
**kwargs		Keyword arguments for the base class.	{}

Source code in src/eva/multimodal/data/datasets/text_image.py

def __init__(self, *args, transforms: TransformsSchema | None = None, **kwargs) -> None:
    """Initializes the dataset.

    Args:
        *args: Positional arguments for the base class.
        transforms: The transforms to apply to the text, image and target when
            loading the samples.
        **kwargs: Keyword arguments for the base class.
    """
    super().__init__(*args, **kwargs)

    self.transforms = transforms

load_image abstractmethod

Returns the image content.

Parameters:

Name	Type	Description	Default
index	int	The index of the data sample.	required

Returns:

Type	Description
Image	The image content.

Source code in src/eva/multimodal/data/datasets/text_image.py

@abc.abstractmethod
def load_image(self, index: int) -> tv_tensors.Image:
    """Returns the image content.

    Args:
        index: The index of the data sample.

    Returns:
        The image content.
    """
    raise NotImplementedError

eva.multimodal.data.datasets.PatchCamelyon

Bases: TextImageDataset[int], PatchCamelyon

PatchCamelyon image classification using a multiple choice text prompt.

Parameters:

Name	Type	Description	Default
root	str	The path to the dataset root. This path should contain the uncompressed h5 files and the metadata.	required
split	Literal['train', 'val', 'test']	The dataset split for training, validation, or testing.	required
download	bool	Whether to download the data for the specified split. Note that the download will be executed only by additionally calling the :meth:prepare_data method.	False
transforms	TransformsSchema | None	A function/transform which returns a transformed version of the raw data samples.	None
max_samples	int | None	Maximum number of samples to use. If None, use all samples.	None
prompt_template	PromptTemplate | None	The template to use for rendering prompts. If None, uses the default template which enforces JSON output.	None
prompt_render_kwargs	Dict[str, Any] | None	The kwargs to use when rendering the prompt template.	None

Source code in src/eva/multimodal/data/datasets/multiple_choice/patch_camelyon.py

def __init__(
    self,
    root: str,
    split: Literal["train", "val", "test"],
    download: bool = False,
    transforms: TransformsSchema | None = None,
    max_samples: int | None = None,
    prompt_template: templates.PromptTemplate | None = None,
    prompt_render_kwargs: Dict[str, Any] | None = None,
) -> None:
    """Initializes the dataset.

    Args:
        root: The path to the dataset root. This path should contain
            the uncompressed h5 files and the metadata.
        split: The dataset split for training, validation, or testing.
        download: Whether to download the data for the specified split.
            Note that the download will be executed only by additionally
            calling the :meth:`prepare_data` method.
        transforms: A function/transform which returns a transformed
            version of the raw data samples.
        max_samples: Maximum number of samples to use. If None, use all samples.
        prompt_template: The template to use for rendering prompts. If None, uses the
            default template which enforces JSON output.
        prompt_render_kwargs: The kwargs to use when rendering the prompt template.
    """
    super().__init__(root=root, split=split, download=download, transforms=transforms)

    self.max_samples = max_samples

    if self.max_samples is not None:
        self._expected_length = {split: max_samples}

    prompt_template = prompt_template or self._default_prompt_template
    prompt_render_kwargs = prompt_render_kwargs or self._default_render_kwargs
    self.prompt = prompt_template.render(**prompt_render_kwargs)