langsmith.evaluation.evaluator.DynamicRunEvaluator

class langsmith.evaluation.evaluator.DynamicRunEvaluator(func: Callable[[Run, Example | None], EvaluationResult | EvaluationResults | dict | Awaitable[EvaluationResult | EvaluationResults | dict]], afunc: Callable[[Run, Example | None], Awaitable[EvaluationResult | EvaluationResults | dict]] | None = None)

A dynamic evaluator that wraps a function and transforms it into a RunEvaluator.

This class is designed to be used with the @run_evaluator decorator, allowing functions that take a Run and an optional Example as arguments, and return an EvaluationResult or EvaluationResults, to be used as instances of RunEvaluator.

func

The function that is wrapped by this evaluator.

Type:

Callable

Initialize the DynamicRunEvaluator with a given function.

Parameters:

• func (Callable) – A function that takes a Run and an optional Example as

• arguments –

• ComparisonEvaluationResult. (and returns a dict or) –

• afunc (Optional[Callable[[Run, Optional[Example]], Awaitable[_RUNNABLE_OUTPUT]]]) –