Networks

Reference information for the language model Networks API.

eva.language.models.modules.TextModule

Bases: ModelModule

Text-based LLM module for inference.

Uses LLM wrappers for text generation and supports evaluation using configurable metrics and post-processing transforms.

Parameters:

Name	Type	Description	Default
model	Module	An LLM wrapper (PyTorch-compatible) for text generation.	required
prompt	str	The prompt to use for generating text.	required
metrics	MetricsSchema | None	Metrics schema for evaluation.	None
postprocess	BatchPostProcess | None	A helper function to post-process model outputs before evaluation.	None

Source code in src/eva/language/models/modules/text.py

def __init__(
    self,
    model: nn.Module,
    prompt: str,
    metrics: metrics_lib.MetricsSchema | None = None,
    postprocess: batch_postprocess.BatchPostProcess | None = None,
) -> None:
    """Initializes the text inference module.

    Args:
        model: An LLM wrapper (PyTorch-compatible) for text generation.
        prompt: The prompt to use for generating text.
        metrics: Metrics schema for evaluation.
        postprocess: A helper function to post-process model outputs before evaluation.
    """
    super().__init__(metrics=metrics, postprocess=postprocess)

    self.model = model
    self.prompt = prompt

forward

Generates text responses for a batch of prompts.

Parameters:

Name	Type	Description	Default
prompts	List[str]	List of input texts to generate responses.	required
args	Any	Additional arguments.	()
kwargs	Any	Additional keyword arguments.	{}

Returns:

Type	Description
List[str]	List of generated responses.

Source code in src/eva/language/models/modules/text.py

@override
def forward(self, prompts: List[str], *args: Any, **kwargs: Any) -> List[str]:
    """Generates text responses for a batch of prompts.

    Args:
        prompts: List of input texts to generate responses.
        args: Additional arguments.
        kwargs: Additional keyword arguments.

    Returns:
        List of generated responses.
    """
    return self.model(prompts)

validation_step

Validation step that runs batch inference and evaluates metrics.

Parameters:

Name	Type	Description	Default
batch	TEXT_BATCH	An input batch.	required
args	Any	Additional arguments.	()
kwargs	Any	Additional keyword arguments.	{}

Returns:

Type	Description
STEP_OUTPUT	Dictionary with predictions, ground truth, and evaluation metrics.

Source code in src/eva/language/models/modules/text.py

@override
def validation_step(self, batch: TEXT_BATCH, *args: Any, **kwargs: Any) -> STEP_OUTPUT:
    """Validation step that runs batch inference and evaluates metrics.

    Args:
        batch: An input batch.
        args: Additional arguments.
        kwargs: Additional keyword arguments.

    Returns:
        Dictionary with predictions, ground truth, and evaluation metrics.
    """
    return self._batch_step(batch)