|
| 1 | +# Value Modeling & Targeting Documentation |
| 2 | + |
| 3 | +This document outlines the rationale and logic behind the calculated metrics and targets used in the "Value Modeling & Targeting" dashboard. Each section corresponds to a category of metrics displayed in the dashboard. |
| 4 | + |
| 5 | +## Org Metrics |
| 6 | + |
| 7 | +### Seats |
| 8 | +- **Logic**: Based on the average total active seats (licenses) across top 10 recent days for the organization. |
| 9 | +- **Max**: Set to known total developer headcount. |
| 10 | + |
| 11 | +### Adopted Devs |
| 12 | +- **Logic**: Average of total active developers using AI tooling (e.g. Copilot) from top 10 recent days for the organization. |
| 13 | +- **Max**: Total known developer count. |
| 14 | + |
| 15 | +### Monthly Devs Reporting Time Savings |
| 16 | +- **Logic**: Count of distinct users who responded to time-savings surveys in past 30 days. |
| 17 | +- **Target**: Double the current, indicating intent to increase reporting. |
| 18 | + |
| 19 | +### % of Seats Reporting Time Savings |
| 20 | +- **Logic**: (Monthly reporting users / total seats) * 100. |
| 21 | +- **Purpose**: Shows how broadly time savings are captured. |
| 22 | + |
| 23 | +### % of Seats Adopted |
| 24 | +- **Logic**: (Adopted Devs / Total Seats) * 100. |
| 25 | +- **Use**: Adoption penetration relative to seat assignments. |
| 26 | + |
| 27 | +### % of Max Adopted |
| 28 | +- **Logic**: (Adopted Devs / Total Developer Count) * 100. |
| 29 | +- **Use**: Indicates potential ceiling for adoption. |
| 30 | + |
| 31 | +## Daily User Metrics |
| 32 | + |
| 33 | +### Daily IDE Suggestions |
| 34 | +- **Logic**: Averaged from last 5 valid daily records. |
| 35 | +- **Target/Max**: Calibrated based on observed high-performing usage. |
| 36 | + |
| 37 | +### Daily IDE Acceptances |
| 38 | +- **Logic**: Suggestions * 30% (default assumed acceptance rate). |
| 39 | +- **Target/Max**: Reflects healthy usage from productive orgs. |
| 40 | + |
| 41 | +### Daily IDE Chat Turns |
| 42 | +- **Logic**: Average of chat turns per day per user from recent week. |
| 43 | +- **Target/Max**: Reflects healthy usage from productive orgs. |
| 44 | + |
| 45 | +### Daily Dot-Com Chats |
| 46 | +- **Logic**: Chat Turns * 33% (estimated portion on dot-com). |
| 47 | +- **Target**: Not yet set pending more data. |
| 48 | + |
| 49 | +### Weekly PR Summaries |
| 50 | +- **Logic**: Total PR summaries / daily active users from last week. |
| 51 | + |
| 52 | +### Weekly Time Saved |
| 53 | +- **Logic**: Weekly average from time savings reports per developer. |
| 54 | +- **Calculation**: |
| 55 | + ``` |
| 56 | + const weeklyHours = hoursPerYear / 50; // Assuming 50 working weeks |
| 57 | + const weeklyDevHours = weeklyHours * (percentCoding / 100); |
| 58 | + const avgWeeklyTimeSaved = weeklyDevHours * (avgPercentTimeSaved / 100); |
| 59 | + ``` |
| 60 | + |
| 61 | +## Calculated Impacts |
| 62 | + |
| 63 | +### Monthly Time Savings (hrs) |
| 64 | +- **Formula**: Adopted Devs * Weekly Time Saved * 4. |
| 65 | +- **Max**: 80 hours/month * total seats (full work month). |
| 66 | + |
| 67 | +### Annual Time Savings (Dollars) |
| 68 | +- **Formula**: Weekly Time Saved * 50 weeks * $100/hr * Adopted Devs. |
| 69 | +- **Note**: $100/hr is assumed average developer cost. |
| 70 | + |
| 71 | +### Productivity / Throughput Boost |
| 72 | +- **Formula**: ((40 + Weekly Time Saved) / 40 - 1) * 100. |
| 73 | +- **Purpose**: Estimates effective increase in output per dev. |
| 74 | + |
| 75 | +## Source of Calculations |
| 76 | + |
| 77 | +All calculations were derived from one or more of: |
| 78 | +- Recent metric exports (5 most recent days) |
| 79 | +- Monthly time-savings surveys |
| 80 | +- Developer seat and activity data |
| 81 | +- Assumed baselines (e.g., 40-hr weeks, $100/hr, 70% acceptance) |
| 82 | + |
| 83 | +Targets are either: |
| 84 | +- Reflective of past top 10 org benchmarks |
| 85 | +- Strategically aspirational (2x current, known limits) |
| 86 | + |
| 87 | +This model provides a structured framework for tracking usage, estimating impact, and guiding adoption investments. |
| 88 | + |
| 89 | +> Edits can include notes on thresholds, cohort segmentation, or more nuanced modeling (e.g., p50/p90 range breakdowns). |
0 commit comments