Transforms

Reference information for the model Transforms API.

eva.models.transforms.ExtractCLSFeatures

Extracts the CLS token from a ViT model output.

Parameters:

Name	Type	Description	Default
cls_index	int	The index of the CLS token in the output tensor.	0
num_register_tokens	int	The number of register tokens in the model output.	0
include_patch_tokens	bool	Whether to concat the mean aggregated patch tokens with the cls token.	False

Source code in src/eva/core/models/transforms/extract_cls_features.py

def __init__(
    self, cls_index: int = 0, num_register_tokens: int = 0, include_patch_tokens: bool = False
) -> None:
    """Initializes the transformation.

    Args:
        cls_index: The index of the CLS token in the output tensor.
        num_register_tokens: The number of register tokens in the model output.
        include_patch_tokens: Whether to concat the mean aggregated patch tokens with
            the cls token.
    """
    self._cls_index = cls_index
    self._num_register_tokens = num_register_tokens
    self._include_patch_tokens = include_patch_tokens

eva.models.transforms.ExtractPatchFeatures

Extracts the patch features from a ViT model output.

Parameters:

Name	Type	Description	Default
has_cls_token	bool	If set to True, the model output is expected to have a classification token.	True
num_register_tokens	int	The number of register tokens in the model output.	0
ignore_remaining_dims	bool	If set to True, ignore the remaining dimensions of the patch grid if it is not a square number.	False

Source code in src/eva/core/models/transforms/extract_patch_features.py

def __init__(
    self,
    has_cls_token: bool = True,
    num_register_tokens: int = 0,
    ignore_remaining_dims: bool = False,
) -> None:
    """Initializes the transformation.

    Args:
        has_cls_token: If set to `True`, the model output is expected to have
            a classification token.
        num_register_tokens: The number of register tokens in the model output.
        ignore_remaining_dims: If set to `True`, ignore the remaining dimensions
            of the patch grid if it is not a square number.
    """
    self._has_cls_token = has_cls_token
    self._num_register_tokens = num_register_tokens
    self._ignore_remaining_dims = ignore_remaining_dims