AI Performance Report — ROI & Automation Metrics

What it does

The AI Performance Report shows you exactly how much time and money Vanio AI saves your operation. See the breakdown of AI-handled versus human-handled messages, estimated cost savings, action counts by type, per-listing automation rates, and response time comparisons — all with configurable date ranges.

Getting started

Accessing the report

1. Navigate to Settings > AI Assistant from the main sidebar
2. Click the Performance tab
3. The report loads with the current month selected
4. Use the date picker to analyze different periods — the URL updates so you can bookmark specific views

Reading the headline metrics

The top section displays four KPI cards:

• Messages Handled — Total messages processed, split between AI-automated and human-handled
• Automation Rate — Percentage of messages fully handled by AI without human intervention
• Estimated Savings — Dollar value of time saved, calculated from average message handling time multiplied by your configured hourly cost
• Avg Response Time — Median response time for AI versus human, shown side by side

How it works

AI vs. human messaging timeline

The main chart shows message volume over time with two series:

• Blue line — Messages handled entirely by AI (auto-responded, auto-classified, auto-actioned)
• Orange line — Messages that required human intervention (escalated, manually responded)

Hover over any point to see the exact counts for that day. A widening gap between the lines means your AI is taking on more of the workload over time.

Action breakdown

Below the timeline, a bar chart categorizes every action the AI performed:

• Auto-reply — Guest messages answered without human involvement
• Task creation — Cleaning or maintenance tasks created automatically
• Payment processing — Charges, refunds, or payment links generated
• Reservation updates — Date changes, cancellations, or modifications handled
• Escalation — Cases sent to a human team member for review

This shows you not just that AI is working, but what it is doing.

Per-listing automation rate

A table lists every property with its individual AI automation rate:

• Property name — The listing
• Total messages — Number of guest messages received
• AI-handled — Messages fully handled by AI
• Automation rate — Percentage
• Avg AI response time — Median time for AI responses on this property

Sort by automation rate to find properties where AI struggles — these may need updated knowledge base articles or procedure adjustments.

Response time comparison

A side-by-side comparison shows:

• AI median response time — Typically under 30 seconds
• Human median response time — Varies by team staffing and time of day
• Time saved per message — The difference, extrapolated to total messages for a dollar savings estimate

Key features

• ROI quantification — Dollar estimates of AI savings based on your actual message volume and labor costs
• AI vs. human timeline — Visualize automation growth over time
• Action breakdown — See exactly what types of actions AI performs
• Per-listing rates — Identify properties that need AI tuning
• Response time metrics — Compare AI and human response speeds with median calculations
• Configurable date ranges — Analyze any period with URL-persistent date selection
• Export — Download the full report data for stakeholder presentations or owner reports

Tips & best practices

Review monthly and share with stakeholders. The AI Performance Report makes a compelling case for the value of automation. Include it in monthly reports to property owners.

Investigate low-automation properties. If a property has a significantly lower AI automation rate, check its knowledge base and procedures. Missing information is usually the cause.

Track response time trends. If AI response times increase, it may indicate more complex queries or system load. Human response time increases may signal staffing gaps.

Use the savings estimate in ROI calculations. Configure your hourly labor cost in Settings > AI Assistant > Economics to get accurate savings projections. This is useful for justifying Vanio AI to new clients or expanding your portfolio.