Monitor Your Agents

Once your AI agent is published and being used, ongoing monitoring is essential to ensure optimal performance and user satisfaction. Shipable provides comprehensive tools to track usage, analyze conversations, and continuously improve your agents.

Accessing Analytics

To view performance data for your agents:

1. Navigate to the Analytics section in your Shipable dashboard
2. Select the specific agent you want to monitor
3. Choose your preferred time range for data analysis

Key Metrics to Track

Usage Statistics

Monitor how frequently your agent is being used:

• Total Conversations: Track the overall number of interactions
• Active Users: See how many unique users are engaging with your agent
• Session Duration: Understand how long users typically interact with your agent
• Peak Usage Times: Identify when your agent receives the most traffic

Performance Metrics

Evaluate how well your agent is performing:

• Response Time: Measure how quickly your agent responds to queries
• Completion Rate: Track the percentage of conversations that reach a successful conclusion
• Tool Usage: See which connected tools are being utilized most frequently
• Knowledge Base Hits: Identify which parts of your knowledge base are most referenced

User Satisfaction

Gauge how users feel about their interactions:

• Feedback Scores: Review user ratings if you’ve enabled feedback collection
• Abandonment Rate: Track how often users leave conversations before completion
• Follow-up Questions: Measure how frequently users need to ask for clarification

Reviewing Conversations

Shipable allows you to review actual agent-user interactions:

1. Navigate to the Conversations section
2. Browse through conversation history
3. Use filters to find specific types of interactions
4. Analyze both successful conversations and potential problem areas

What to Look For

When reviewing conversations, pay attention to:

• Misunderstandings: Instances where the agent didn’t properly interpret the user’s intent
• Knowledge Gaps: Questions the agent couldn’t answer adequately
• Successful Patterns: Interactions that led to positive outcomes
• Common Queries: Frequently asked questions that might warrant specialized handling

Improving Your Agent

Knowledge Base Enhancements

1. Identify information gaps from unanswered questions
2. Add new documents or data to address these gaps
3. Refine existing content to improve relevance and accuracy
4. Remove or update outdated information

Instruction Refinement

1. Adjust your agent’s instructions based on conversation analysis
2. Provide more specific guidelines for handling common scenarios
3. Update the agent’s persona if the tone isn’t resonating with users
4. Add examples of ideal responses for frequently asked questions