JevMade hello@JevMade.com
← Back to guides

JevMade field notes / Instrumentation guide

How to track AI decisions and hide sensitive information

This guide explains how to connect an AI decision tool to a tracking program. It shows how to record the questions asked and answers given, and how to turn on settings that hide sensitive information.

Original by Arize AIIntegrations

Listen to this guide

JevMade’s plain-English explanation

0:00 /

AI narration

Credits

“TypeSafe AI Tracing with Phoenix” by Arize AI. Read the original source.

This expanded guide is an AI-narrated adaptation prepared by JevMade. It expands the source’s essential ideas, examples and caveats in JevMade’s own words and is not a word-for-word reading. The synthetic voice does not imitate the author or imply their endorsement.

Our summary

People use tracking software to understand how an AI assistant makes choices. This guide shows how to connect a specific decision-making tool to a tracking program. It helps you see exactly what information the software received and what answers it produced.

The tracking software records each AI decision as a single event. Instead of saving a normal conversation, it logs the exact situation, the specific questions asked, and the structured answers returned. It also counts the amount of text the software processed.

This setup is useful for developers who need to monitor AI choices. You must deliberately turn on privacy settings to hide sensitive questions or answers. Different programming languages require slightly different setup steps, and short tasks must finish saving before closing.

Key takeaways

  1. Connect the decision tool to the tracker to record inputs and outputs as structured data.
  2. Look at the exact questions and answers instead of searching for a standard chat transcript.
  3. You must deliberately turn on privacy settings to hide sensitive information before the tracker saves it.

The provided examples were reviewed by reading the source documents. They were not tested against a live tracking system.

Phoenix documentation · Source reviewed

Read the original guide Opens the author’s site in a new tab.