Defining the AI problem
Before committing to AI, I worked through whether it was the right tool and what it would need to do. This page lays out that reasoning using two exercises from the "User Needs + Defining Success" chapter of Google's People + AI Guidebook. The first asks whether AI is the right tool for the user need. The second defines what the AI must do, what is easy to leave unspecified, and how the AI could learn the wrong goal. The last part sets what would count as success.
Everything below comes from the research and build in the case study. The short answer: AI can help, but only with the instructor reviewing before formal feedback reaches a student.
User research summary
Both exercises start from the same evidence.
| Date | Source | Summary of findings |
|---|---|---|
| Summer 2026, before the build | Informal conversations: Lumyr (instructor, Salsa Y Control), Debra and Andres (students) | Lumyr already watches every homework video and types timestamped corrections for each student. Students want expert feedback, but private lessons and workshops cost money, time, and energy. Andres asked whether instructors would trust AI feedback. |
| Aug 22, 2026 | Prototype demo and feedback: Ashwin, studio owner and instructor outside Salsa Y Control | Poor accuracy comes from the video capture, not the model: inconsistent angles, framing, and lighting. Suggested on-screen guidance while recording. Flagged aligning counts between the instructor's and student's videos as a separate problem. |
| Aug 22, 2026 | Interview: Nick, team student | Practices by drilling with a mirror and slowed-down music. Recording and comparing himself isn't a habit. Motivated by group accountability. Most team material is partnered. |
| Aug 27, 2026 | Second interview (two parts): Lumyr | Reviews about 34 videos a week (1–2 hours) and doesn't mind the time. A correction often hasn't taken hold by the next class. Technique is a path to a look shaped by each teacher. Reviews only on a laptop with two monitors. Would trust another person's read only if they could explain his standard. |
| Aug 30, 2026 | Working session with a live accuracy test: Andres | The analysis found real differences between a beginner and an advanced dancer (about 51% more head movement for the beginner), but anyone could see that by eye. Accent and shoulder ratios measure style, not mistakes. Decision: pause building and validate the feedback. |
| Aug–Sep 2026 | Five rounds of accuracy tests on real takes | A video compared with itself went from a match score of 75 to 100. The gap between a correct and a deliberately wrong take grew from 2.4 to 11.2 points. Two compressed copies of one video agreed on weight transfer for only 87.5% of counts. A failed pose detection scores 0, which looks like bad dancing. |
| Sep 8, 2026 | Group interview: Salsa Y Control instructors (Andres G., Tara, Kelly, Lumyr) | Assessment needs human judgment. "Why don't I move up?" is students' top complaint. Level names and teaching differ between studios. One instructor reviewed about 700 videos. Single named steps are easier to judge than choreography. |
| Oct 2026 | Competitive analysis of 8 options | No product combines the studio's own reference, AI feedback, and instructor review. DanceBetter returned no result after a full day; DanceBuddy has about 10,000 users and no paying customers. Private lessons cost $50–224 an hour. The real competitor is free solo practice. |
1Is AI the right tool?
Exercise: Evidence of User Need.
Statement of user need
How might we help team dancers keep practicing an instructor's correction between weekly classes until it sticks, without waiting a week or paying for a private lesson?
Pose analysis can compare every practice take with the instructor's own reference video, count by count, at any hour. No instructor can review every practice take, so AI fills the week between classes. The instructor still reviews the final take before formal feedback reaches the student.
| How pervasive is the problem? | It recurs every week of the team season. One instructor reviews about 34 homework videos a week; another reviewed about 700. Corrections often haven't taken hold by the next class, and every Salsa Y Control instructor named "why don't I move up?" as students' top complaint. |
|---|---|
| Whom does it affect? | Students on studio dance teams, who practice choreography between classes. Instructors, who repeat corrections and must explain placement decisions. Indirectly, dance partners, because poor technique can get partners hurt. |
| How does it affect various individuals? | Students wait a week for feedback or pay $50–224 an hour for private lessons; some travel to workshops. Students who stall get frustrated that they aren't moving up. Instructors spend 1–2 hours a week on homework review and still see fixes that don't take. |
| How does it vary across dimensions of identity or historically marginalized communities? | The research did not study this, so it is an evidence gap. What the evidence does show: the paid alternatives (private lessons, workshops) favor students with more money and time. The app runs only on Android, though iPhone users are the larger group. Body type, camera distance, and lighting change how large a movement looks, so accuracy may differ between dancers. The sample is one Boston scene. |
| Do the effects change over time? | Yes. A correction fades over the week between classes. Within a season, counts videos appear only in the first weeks, and teams move to group and partner work, which single-dancer pose tracking can't follow. Over months, students who stall stop seeing progress. |
AI suitability
Checked statements describe Dance Coach. I added one statement to each side.
AI is probably better for
- Personalization: the core experience requires personalizing content to individual users or a domain.Feedback is specific to each student's take and to their own studio's reference, because there is no universal correct technique.
- Predictions: requires predicting future events.
- Scale: need to recognize a general class of things too large to articulate every case.Every dancer, room, and take is different; pose tracking reads body positions without a rule for each case.
- Volume: need to detect low-occurrence events that keep evolving.
- Language: requires natural language interactions or an agent.
- Diversification: the experience is better when outcomes vary.
- Availability (added): feedback is needed at the moment of practice, at home, when no instructor is present.Homework feedback now waits until the next class, a week later.
AI is probably not better for
- Consistency: the most valuable part is predictability regardless of input.Not checked: the value is feedback on each take, which varies by design.
- Cost: running AI at the needed frequency is very high.Not checked yet: storage and processing costs arrive with real users and still need measuring.
- Stakes: the consequences of errors are very high.Wrong feedback can teach wrong technique, and instructors said assessment needs human judgment. Mitigated by instructor review before formal feedback is released.
- Explainability: people need to understand what happens.Lumyr would trust another person's read only if they could explain his standard, and students need to know which count to fix.
- Momentum: getting to market first matters more than AI's value.
- Directive (partly): people say they don't want a task automated.Instructors don't want assessment or level placement automated. They welcome support for practice between classes.
- Input quality (added): accuracy depends on conditions users control.Angle, distance, lighting, and cramped rooms caused most errors, so recording guidance comes before model tuning.
Summary of evidence
I think AI can help solve team dancers' need to keep practicing an instructor's correction between weekly classes, because pose analysis can check every practice take against the studio's own reference video at any hour. No instructor can do that for every take, and no current product does it with the student's own instructor involved.
It helps only with the instructor in the loop. Instructors said assessment needs human judgment, the analysis is not yet reliable on subtle mistakes or style, and accuracy depends on recording conditions. So the AI supplies count-by-count evidence for practice, and the instructor reviews the final take before formal feedback is released.
I'll revisit this after the controlled accuracy test, after testing recording guidance, or if partner work comes into scope.
2What exactly should the AI do?
Exercise: Specify and Align the AI Problem.
Primary goal
to dance the team's choreography the way their instructor teaches it, by fixing specific mistakes between weekly classes instead of waiting a week for the next correction.
- Find the dancer's body points (hips, knees, shoulders, feet) reliably in an ordinary phone video.
- Line up the student's take with the instructor's reference video count by count, including detecting which way each video was filmed.
- Compare timing and technique (hip movement, weight transfer, footwork, frame, knee depth) against the instructor's reference, not against fixed rules.
- Point the student to the specific counts to work on, with only the top one or two items during practice.
- Say when it can't judge a take, such as a failed detection or a poor recording, instead of showing it as a low score.
Sub-goals
Before the primary goal
- Learn the piece from the reference: the counts video early in the season, the music video after.
- Know which correction the instructor gave them to work on.
- Record a usable take: full body in frame, enough light, a clear angle, and room to dance.
While working on it
- Hit movements on the right counts.
- Work on the technique the instructor stressed, such as hip action, without losing the choreography.
- Decide when a take is good enough to send to the instructor.
These sub-goals are mostly consistent with the primary goal, because each moves the student toward dancing the piece as taught. Two are in tension with it: recording a usable take takes time away from practice, and most team material is partnered, which a student can't fully practice alone and single-dancer tracking can't follow.
Underspecification
Parts of the problem that are easy to assume or overlook, what goes wrong if they stay unknown, and how to pin them down.
| If this is unknown | It can result in | It can be specified if we |
|---|---|---|
| The studio's standard: what "correct" means at this studio, and which differences are style rather than mistakes | The AI scoring stylistic choices as errors and contradicting the instructor. In testing, accent ratio and shoulder ratio measured style, not mistakes. | Bring in an instructor to say which measures matter and where the AI is wrong, and show students only the measures that hold up. |
| Recording conditions: camera angle, distance, lighting, framing, and the dancer's body type | Movements looking larger or smaller than they are, and failed detections scoring 0, which looks like bad dancing. | Add a framing guide while recording and a quality check before submitting, then run a controlled accuracy test with the same dancer, clothes, and lighting. |
| Where count 1 falls: how the counts line up between the two videos | Feedback naming the wrong count. The analysis assumes count 1 has the most movement, and Ashwin flagged count alignment as its own problem. | Take the beat grid from the music version of the reference whenever it exists, flag counts-only references as lower confidence, and include count alignment in the controlled test. |
| Partner and group work: whether a section is danced alone, with a partner, or as a group | Feedback that judges only one dancer on partnered material, or tracks the wrong person, as teams move to partner work later in the season. | Say plainly that the AI doesn't judge partner connection, and keep AI feedback to solo sections until partner detection is in scope. |
Optimization
Meeting the primary goal can
- Give students feedback on every practice take, instead of waiting a week for the next class.
- Reinforce the corrections instructors already give, so fewer fade between classes.
- Hold students to their own studio's standard, and show instructors which counts each student missed when they review a final take.
But what if the model
- Rewards a take that roughly matches the overall movement, so a student scores well while dancing the wrong steps? It hasn't yet been tested with deliberately wrong choreography.
- Treats every difference from the instructor as a mistake, pushing all students to copy one person's styling?
- Scores a bad recording as bad dancing, discouraging students with scores they didn't earn?
Where sub-goals compete. More feedback per take works against trustworthy feedback, because the extra measures (style, weight transfer) are the least reliable. The design response: show students fewer measures than instructors see, and keep the instructor's review before any formal feedback.
3What would count as success?
The criteria the controlled accuracy test has to meet. The exact thresholds, such as how often an instructor must agree with the AI, will be set with an instructor during that test.
Timing and technique need different criteria. A move is either on the count or it isn't, but correct technique depends on each studio's standard.
| Timing | On the controlled test, the deliberately wrong take scores below the correct take every time, and the counts the AI flags match the counts the instructor flags. |
|---|---|
| Technique | An instructor agrees with the AI's top one or two items for most takes, and rarely has to remove an AI note before releasing feedback. |
| Takes it can't judge | A failed detection or a poor recording is labeled as such, and never shown as a low score. |
| For students | The next practice take improves on the flagged counts, and instructors repeat fewer corrections by the next class. |
The tradeoff: precision over coverage. For practice feedback no one has checked, a wrong note does more harm than a missed one. So practice mode should favor fewer, correct notes over flagging everything.
Exercises: "Evidence of User Need" and "Specify and Align the AI Problem," User Needs + Defining Success chapter, People + AI Guidebook (Google PAIR).