Testing whether AI can coach dance technique
Dance Coach was my hands-on project for the first three units of the Open Vector curriculum, which teaches designers to build real products by directing AI agents. I used it to answer a product question: can pose analysis give Latin dance students technique feedback they can trust between classes? I researched the need, built the full system with Claude Code, and tested it on real footage. The evidence so far shows that MediaPipe, the pose-estimation library I used, can't judge dance technique. The part it can judge, timing, isn't worth switching from what teams already use.
At a glance
- Interviewed instructors and students in Boston's Latin dance community.
- Built the full system end to end: an Android app for students, a web dashboard for instructors, and a pose-analysis backend, tested with real accounts.
- Ran five rounds of accuracy work on real footage. Technique feedback stayed unreliable.
- Concluded MediaPipe can't do the job students need, and stopped building.
I've danced in Boston's Latin dance community for over 8 years, so I could draw on existing relationships for candid research. I spoke with instructors Lumyr, Tara and Kelly at Salsa Y Control (SyC), studio owner Ashwin, and students Debra, Andres and Nick, before and during the build.
Watches every homework video his team submits each week and types timestamped corrections for each student.
Pays for private lessons and travels to dance workshops to improve her technique with leading instructors. AI that could cover some of that would help her.
Asked whether instructors would trust AI feedback, and stressed that students need to master the basics first.
Instructors already give individual video feedback on team homework. What holds students back is the week between corrections, and the cost of getting expert feedback any other way: private lessons run $50–224 an hour. Three findings shaped the product:
Corrections don't always stick
A correction given once often hasn't taken hold by the next class. Lumyr reviews 34 homework videos a week and doesn't mind the work; what frustrates him is a fix that works for one student and not others.
There is no universal correct technique
Each teacher shapes their own standard, and level names don't carry over between studios.
Instructors want support, not a verdict
Assessment needs human judgment.
The top-ranked job to be done came from students: when I'm practicing choreography at home between classes, I want to know where I'm off on the count or off on the movement, so I can fix it before the next class instead of waiting a week.
The real competitor was free solo practice: rewatching the class video, a mirror, and slowed-down music, alongside Band, the group-messaging app teams already use to share videos. Dance Coach had to offer what those can't: feedback.
All six findingsThe evidence behind each, and what it meant for the design
Finding | Evidence | What it meant for the design |
|---|---|---|
Video homework already exists, but corrections don't always stick | Lumyr reviews 34 videos a week (1–2 hours) and doesn't resent it, but his fix for one move worked for one student and others never got there. One SyC instructor reviewed about 700 videos. Feedback waits until the next class. | Reinforce feedback instructors already give. An AI check on every practice take covers the week between corrections. |
There is no universal correct technique | Lumyr: technique is a path to a particular look, shaped by each teacher. SyC: level names and teaching differ between studios. | Every reference video comes from the studio's own instructor. A generic "correct salsa" model would contradict how they teach. |
Instructors want support, not a verdict | SyC: assessment needs human judgment. Lumyr would trust another person's read only if they could explain his standard. | The AI output is evidence the instructor reviews and releases. No pass/fail scores. |
Poor accuracy is mostly a recording problem | Angle, distance, lighting, and cramped rooms came up from Ashwin, SyC, Lumyr, and Andres. | Recording guidance and a quality check before submission come before model tuning. |
Instructors review on large screens | Lumyr reviews only on a laptop with two monitors. On a phone he can't see detail or watch and type at once. | Instructors review on a web dashboard, not a second mobile app. |
Students rewatch and repeat; they don't record and compare | Nick drills with a mirror and slowed music. Recording himself wasn't a habit. | Recording and comparing is a new habit. Team obligation is the only force found that would drive it. |
Leveling was the most consistently raised problem, even though I didn't ask about it. Every SyC instructor named "why don't I move up?" as students' top complaint.
Method and constraintsHow I ran the research and what it can support
Conversations happened after class, at events, and in a few scheduled sessions, using Erika Hall's book Just Enough Research as a guide. There was no budget for formal, paid sessions.
- Bias check: as a team dancer for 8+ years, I recorded my own view too, but marked it as a separate source in the affinity map and left it out of the findings.
- Limits: a convenience sample from one scene. Good for deciding what to build and test next, not for sizing a market.
- Method: broke summaries into single observations, grouped them, and rated each insight by how many sources agreed.
All five jobsEvery job statement, by role and type
I used Open Vector's format, "When [situation], I want to [motivation], so I can [outcome]."
When I've given a student a correction, I want them to keep practicing it between classes until it sticks, so my feedback actually turns into improvement.
When I'm practicing choreography at home between classes, I want to know where I'm off the count or off the movement, so I can fix it before the next class instead of waiting a week.
When I compare myself to people who started when I did, I want to see what specifically is holding me back, so I can feel I'm progressing rather than stuck.
When a student asks why they haven't moved up, I want to point to clear standards and a record of what I've told them, so I can explain my decision without it feeling personal.
When I dance with my team or a social partner, I want to not be the one who's visibly off, so I don't hold the group back or make a partner's dance harder.
Evidence for each jobWhat supports it, and what it competes with today
Job | Evidence |
|---|---|
1 | Lumyr doesn't mind the 1–2 hours a week he spends on homework videos. What frustrates him is a correction that doesn't take: his fix for one move worked for one student, and others never got there. Hip movement drilled at the start of class disappears once students run the routine. |
2 | Lumyr: homework feedback waits until the next class. Private lessons, the alternative, cost $50–224 an hour. Andres and me: we know there's technique we aren't executing properly, but can't fix it without paying for private lessons. |
3 | SyC: "why don't I move up?" is the top complaint; students expect fast progress and stall. |
4 | SyC: no criteria and no record. Lumyr: documented technique would make placement easier to explain. |
5 | Nick: motivated by group accountability. Andres: poor technique gets partners hurt. Thin evidence. |
What each job competes with today.
- Job 1: repeating the correction at the next class, a week later, or in a paid private lesson.
- Job 2: rewatching the class video, a mirror, and slowed-down music. None of these judges the student against their own instructor's standard.
- Jobs 3 and 4: an ad hoc conversation after class. No product offers placement or a level path.
Ranking the jobsHow each job scored on the three questions
Question from the exercise | Job | Why |
|---|---|---|
Most frequent | 1 and 2 | Both happen every week during the season |
Most underserved | 3 and 4 | Nothing addresses them today: no criteria, no record, no tool |
Would make users love it if solved perfectly | 2 | It removes the week-long wait, the gap students feel most directly |
Competitive analysisEight options compared, and what they showed
Before designing, I looked at what dancers use today. I compared eight options: the team's current tool (Band), two online courses, three AI movement-analysis apps, and the two things people fall back on, private lessons and practicing alone. View the full analysis (Google Sheet)
No one combines the studio's own reference, AI feedback, and instructor review
Every AI app is sold to individual learners and compares them against generic dancers, a video they pick, or the original artist's video. None involves the student's own instructor.
AI dance apps haven't shown their feedback is reliable, so the instructor's review is what makes Dance Coach credible
When I tested DanceBetter, it returned no result after a full day. In Dance Coach, the student's own instructor reviews the AI's analysis before it's released as formal feedback.
The real competitor is free solo practice, with Band alongside it
Students already rewatch class videos, slow the music down, and practice in a mirror, at no cost. Dance Coach has to offer what solo practice can't: feedback.
All seven takeawaysIncluding partner work, pricing, and player features
- No competitor combines the studio's own reference video, AI feedback, and instructor review. Every AI product is sold to individual learners and compares against generic dancers or a video the learner picks. The closest threat is DanceBetter's quote-only studio tier, which sends AI reports under the coach's name, with no instructor review step and no team assignments. The closest model is JustMove, a K-pop app that compares a student's clip side by side with the original choreography and returns an AI report on timing and technique. Its reference is the artist's video, not the student's instructor.
- AI dance feedback hasn't shown it works or that people will pay for it. DanceBetter has no reviews and no sample output, and in my July test it returned no result after a full day. DanceBuddy has about 10,000 users and no paying customers. Dance Coach's own accuracy isn't reliable yet either, so the instructor review step is what makes the product credible while the model improves.
- The real competitor is free solo practice, with Band alongside it. Students already rewatch class video, slow the music, and use a mirror, at no cost. Band has 4.8 stars from 784k reviews and is already installed, even though students mute it. I still need to decide whether Dance Coach sits next to Band or replaces it.
- Partner work is the largest functional gap. Most team material is partnered, according to Nick and Lumyr, and Dance Coach detects one person.
- The value is speed and cost. Homework feedback through Band takes about a week; private lessons cost $50–224/hr. Dance Coach sits between the two.
- Practice-player features are expected, and mostly missing. VDanceClub and DanceBuddy offer mirror, loop, and 8-count segments. Dance Coach only has speed control.
- No one assesses named fundamentals or offers level placement. Course platforms teach fundamentals but never assess them.
Pose estimation is common in fitness products for checking exercise form, such as Kemtai, which tracks physical therapy exercises through a phone or laptop camera, and Tempo, a home gym that gives real-time form feedback. MediaPipe's own documentation describes its pose model as optimized for fitness applications. AI grading tools in higher education already give a first pass that a teacher reviews. Together, those suggested AI could cover the week between classes: the AI does the repetitive checking, and the instructor keeps the relationship. My hypothesis was that AI movement analysis could help students work on technique between classes, as a supplement to instructors rather than a substitute.
The studio's own instructor is the reference
Every reference video comes from the student's own instructor, not a generic model of correct technique, because each studio defines its own standard. Tradeoff: the AI can only compare, never say what is correct on its own.
The instructor reviews AI feedback before it becomes formal feedback
Feedback comes in two tiers. On practice takes, students get AI feedback immediately, as often as they want, and only they see it. When they're ready, they submit one final take: the instructor sees the same AI analysis, removes or corrects anything wrong, adds their own notes, and only then releases it to the student as formal feedback. Tradeoff: in practice mode, students see AI feedback no one has checked. That became the project's central risk.
Students on a mobile app, instructors on a web dashboard
Students record on their phones, and Lumyr reviews only on a laptop with two monitors. Tradeoff: two front ends to build and maintain.
- 1Instructor uploads reference video
- 2Assignment published to team
- 3Student records a practice take
- 4AI compares poses, scores each count
- 5Student submits final take
- 6Instructor reviews and adds notes
- 7Instructor releases feedback
- 8Student reads feedback
↺ Students repeat steps 3–4 as often as they want before submitting.
The app follows the format teams already use. Instructors post a counts video (the steps danced to spoken counts instead of music), a music video, or both. Counts videos usually appear only in the first weeks of a season. Students submit one take for each reference video.
Since I don't have a coding background, I built the system by directing Claude Code, following the Open Vector curriculum. Open Vector has you plan the system first, check what the agent produces, and understand why it works. My job was to set the plan, give the agent context, and catch it when it was wrong. This project covers units 1–3. Unit 3, The Pipeline, ends with validation, which is where this project reached its answer.
- 01
Plan the system before prompting.
I modeled the data as nouns and verbs (team, choreo piece, practice take, analysis, instructor note) and wrote VECTOR.md, one document describing the users, the loop and the data model. The agent read it at the start of every session.
- 02
Stage the work.
Thirteen prompts built the core loop, one feature each, in dependency order: storage, then pose analysis, then the dashboard, then sign-in. I checked each one worked before writing the next.
- 03
Check the agent's output.
It modeled bachata hip action and weight transfer wrong, and only my training as a dancer caught that. It also left unreleased feedback readable by anyone with the URL, which I caught by reviewing API responses; roles are now verified on the server.
- 04
Keep the docs honest.
VECTOR.md once said sign-in was built when it wasn't. A script now checks the document's countable claims against the code.
The result runs end to end in production: an Android app for students, a web dashboard for instructors, and a backend that analyzes movement with MediaPipe, Google's open-source pose-estimation library.
When I ran real student videos through the app, the technique feedback wasn't accurate enough to trust. I paused feature work and ran five rounds of accuracy work, each measured against real takes.
The model's own measures improved:
- A video compared with itself went from a match score of 75 out of 100 to 100.
- The gap between a correct and a deliberately wrong take grew from 2.4 to 11.2 points.
But the technique feedback stayed unreliable:
- It scored stylistic choices, like accents and shoulder styling, as errors.
- Two differently compressed copies of the same video agreed on weight transfer for only 87.5% of counts.
- Camera angle, distance, lighting and body type changed how large a movement looked.
- In a live test with Andres, it found real differences between a beginner and an advanced dancer, but anyone could see those by eye. The tool would only earn trust by catching the small mistakes a student can't see.
Why. MediaPipe's documentation explains most of this. It tracks 33 points on the body. The hips are two points and there are no spine points, so pelvic tilt, the core of bachata hip action, can only be estimated indirectly. Each foot has three points, too few to judge foot placement. Depth from a single phone camera is an estimate. Unlike several other MediaPipe tasks, pose detection can't be retrained, so it can't learn Latin dance technique. And it's built for fitness form checks, where correct form is fixed, while correct dance technique depends on each studio.
What I haven't tested. The controlled accuracy test (same dancer, clothing and lighting, including a deliberately wrong take) hasn't been run, and the backend used MediaPipe's smallest model at 10 frames per second. A larger model and higher frame rate could improve timing, but wouldn't add the missing body points. I also didn't test heavier whole-body models, such as RTMPose or ViTPose, which track more points on the feet. They could change the conclusion.
Accuracy testsWhat improved and what's still unreliable
Results after five rounds of accuracy work, each measured against real takes:
- Same video compared with itself: a match score of 75 out of 100, now 100
- Gap between a correct and a deliberately wrong take: 2.4 points, now 11.2
- Beginner vs. advanced dancer, a live run with Andres: found real differences in the right shoulder, head stability (about 51% more head movement for the beginner), and spine tilt, and flagged core engagement without being asked
Still unreliable.
- Style is scored as error: accent ratio and shoulder ratio measure stylistic choices, not mistakes.
- Some measures are limited by the pose tracking itself. Two differently compressed copies of the same video agreed on weight transfer for only 87.5% of counts, so weight transfer is shown but no longer scored.
- No test yet with deliberately wrong choreography, so the score may be too generous toward it.
- Recording conditions distort results: body type, distance from the camera, lighting, and framing change how large a movement looks.
- A failed pose detection shows as a score of 0, which looks like bad dancing.
- Count numbering assumes count 1 has the most movement, so feedback can name the wrong count.
The key lesson. The beginner-vs-advanced run found the right answer on a case where the answer was already obvious. That is exactly the case where nobody needs the tool, so it doesn't show the AI can be trusted on subtle cases.
MediaPipe can likely judge timing: whether a student moves on the count. But timing alone isn't worth switching for.
- Students' problem is technique they can't see themselves. Timing is the part they can already partly check against the music.
- Instructors don't need the time saved. Lumyr spends 1–2 hours a week on homework videos and doesn't mind it.
- Switching has a cost. It means a new app next to Band, plus a habit of recording and comparing that students don't currently have.
So I stopped building, and I wouldn't release practice feedback to students. Wrong technique feedback that no one has checked does more harm than no feedback.
What I learned
- 01
Test AI on the cases that are hard to judge.
The analysis was right where the answer was obvious, which is exactly where nobody needs it.
- 02
An AI agent builds fast, so deciding when to stop matters more.
The most important decision in the project was to stop and validate.
- 03
The agent is only as right as the domain knowledge I give it.
My dance training caught errors the agent couldn't.
- 04
Building is a way to answer a product question.
Directing an agent let me test the AI on real footage without an engineering team.
- 05
Match the method to the goal.
Learning to build with agents was my main goal, so building the full system was the point. But for the product question alone, I'd test the riskiest assumption first: a standalone script comparing a few videos, reviewed with an instructor, would likely have shown MediaPipe's limits within a week or two.
For AI technique feedback to work, at least one of these would need to exist:
- A heavier whole-body model, such as RTMPose or ViTPose, that tracks more points on the feet. This is the cheapest test, and I haven't run it.
- A pose model trained on dance footage, with more points on the hips, spine and feet.
- Recording from more than one camera, so depth is measured instead of estimated.
- A narrower task: single named steps instead of full choreography. SyC's instructors pointed out that single steps have clean framing and one clear target, and the build showed that comparing two videos filmed in different rooms and angles caused most of the errors.
Of the ideas that change the task rather than the model, the last is the most realistic. It would also serve the second need that came up in research: helping instructors decide when a student is ready to move up a level.