langsmith.client.Client.create_feedback#

Client.create_feedback(run_id: UUID | str | None, key: str, *, score: float | int | bool | None = None, value: str | dict | None = None, correction: dict | None = None, comment: str | None = None, source_info: Dict[str, Any] | None = None, feedback_source_type: FeedbackSourceType | str = FeedbackSourceType.API, source_run_id: UUID | str | None = None, feedback_id: UUID | str | None = None, feedback_config: FeedbackConfig | None = None, stop_after_attempt: int = 10, project_id: UUID | str | None = None, comparative_experiment_id: UUID | str | None = None, feedback_group_id: UUID | str | None = None, extra: Dict | None = None, trace_id: UUID | str | None = None, **kwargs: Any) → Feedback[source]#

Create a feedback in the LangSmith API.

Parameters:

  • run_id (str or UUID) – The ID of the run to provide feedback for. Either the run_id OR the project_id must be provided.

  • trace_id (Optional[ID_TYPE] = The trace ID of the run to provide feedback for. Enables batch ingestion.) – The trace ID of the run to provide feedback for. This is optional.

  • key (str) – The name of the metric or ‘aspect’ this feedback is about.

  • score (float or int or bool or None, default=None) – The score to rate this run on the metric or aspect.

  • value (float or int or bool or str or dict or None, default=None) – The display value or non-numeric value for this feedback.

  • correction (dict or None, default=None) – The proper ground truth for this run.

  • comment (str or None, default=None) – A comment about this feedback, such as a justification for the score or chain-of-thought trajectory for an LLM judge.

  • source_info (Dict[str, Any] or None, default=None) – Information about the source of this feedback.

  • feedback_source_type (FeedbackSourceType or str, default=FeedbackSourceType.API) –

    The type of feedback source, such as model (for model-generated feedback)

    or API.

  • source_run_id (str or UUID or None, default=None,) – The ID of the run that generated this feedback, if a “model” type.

  • feedback_id (str or UUID or None, default=None) – The ID of the feedback to create. If not provided, a random UUID will be generated.

  • feedback_config (langsmith.schemas.FeedbackConfig or None, default=None,) – The configuration specifying how to interpret feedback with this key. Examples include continuous (with min/max bounds), categorical, or freeform.

  • stop_after_attempt (int, default=10) – The number of times to retry the request before giving up.

  • project_id (str or UUID) – The ID of the project_id to provide feedback on. One - and only one - of this and run_id must be provided.

  • comparative_experiment_id (str or UUID) – If this feedback was logged as a part of a comparative experiment, this associates the feedback with that experiment.

  • feedback_group_id (str or UUID) – When logging preferences, ranking runs, or other comparative feedback, this is used to group feedback together.

  • extra (dict) – Metadata for the feedback.

  • trace_id –

  • kwargs (Any) –

Return type:

Feedback