langsmith.evaluation.evaluator.DynamicComparisonRunEvaluator

class langsmith.evaluation.evaluator.DynamicComparisonRunEvaluator(func: Callable[[Sequence[Run], Example | None], ComparisonEvaluationResult | dict | Awaitable[ComparisonEvaluationResult | dict]], afunc: Callable[[Sequence[Run], Example | None], Awaitable[ComparisonEvaluationResult | dict]] | None = None)

Compare predictions (as traces) from 2 or more runs.

Initialize the DynamicRunEvaluator with a given function.

Parameters:

• func (Callable) – A function that takes a Run and an optional Example as

• arguments –

• EvaluationResults. (and returns an EvaluationResult or) –

• afunc (Optional[Callable[[Sequence[Run], Optional[Example]], Awaitable[_COMPARISON_OUTPUT]]]) –