[ GUIDE ]

How to run a six-week AI pilot

A pilot that answers the questions schools actually have: do the rules hold, do teachers find it useful, what does it cost, and can students do the work alone afterward.

Plan a pilot Watch the demo

Facts as of Oct 18, 2026Last reviewed Oct 19, 20264 min read

[ PILOT ]

What a six-week pilot can and cannot show

A short pilot cannot prove a learning gain. With 200 students, the smallest effect a pilot can reliably detect is about 0.40 SD, and published AI-tutoring effects run 0.06-0.31 SD. So use a pilot to test reliability and usefulness: do the rules hold, do teachers find it useful, do costs stay inside limits, and do students solve practiced problems alone afterward.

Scope

Item	Plan
Courses	1-2 high-school courses chosen by the school
People	2-4 teachers; about 100-200 students; one school safety designee; one IT contact
Length	One teacher-only week, then six weeks with students
Modes	Chosen by each teacher per class
Measures	Delayed unaided success, help after an attempt, mode reliability, cost and teacher usefulness

A six-week AI pilot

Before any student starts

Requirements, not features. Do not start without them:

A signed agreement: the school owns the data; use only to run the pilot; no sale, ads or model training; deletion and breach timelines; a parent-access path.
A named school approver, not a teacher, for the agreement and the parent notice.
Parent notice through the school, consent recorded where a provider requires it, and a non-AI alternative for families who opt out.
Accounts by invitation or roster only, with age from the school's roster.
Minors' model rules checked on the deployed build.
A safety route set to the school's protocol and tested with its designee.
In-product notices: AI, not a person; who can review class chats; where to get help.
The filter allow-list tested on a school device on school Wi-Fi.
Spending ceilings sized to the contract.
A real-model check that the learning modes hold.

Schedule

When	What
Week -3	Share the demo with the school approver and pilot teachers; agree courses and dates
Week -2	Agreement, parent notice, roster, filter allow-list, safety route test
Week -1	Teacher-only week: set modes and instructions, try student views, list what is missing
Week 0	Unaided pre-check (15-20 minutes); each teacher predicts which concepts each student will miss; student onboarding and survey
Weeks 1-5	Normal use; day-7 unaided rechecks; weekly teacher check-in under 2 minutes; weekly spot check of 20 sampled replies
Week 3	Mid-pilot call: what to change
Week 6	Unaided post-check with new items; student survey; interviews
Week 7	Readout and continuation decision
Weeks 8-10	Delayed unaided check, 2-4 weeks after the last practice

Pilot schedule

What to measure

Delayed unaided success (primary): practiced concepts solved alone on new items at 7 days and 2-4 weeks.
Attempt before help: the share of help requests that follow the student's own attempt.
Answer before attempt: problems where the AI gave a final answer before any attempt, by mode; the target is near zero in Hints and Guided.
Teacher usefulness: weekly yes-or-no check-ins with specific examples.
Cost: model spend per active student per month.
Safety: whether every flag reached a person within the agreed time.
Off-platform AI: per assignment, "Did you also use another AI tool?"
Equity: results by group, with small groups suppressed.

A continue rule, agreed in advance

Continue to a paid term only if all four hold:

Teachers answer yes in at least half of the weekly check-ins from week 3 on, and name specific uses.
Learning modes hold: final answers before an attempt stay near zero in Hints and Guided.
Every safety flag reached a person within the agreed time.
Cost per active student stayed inside the contract's ceiling.

What the vendor should not do

Use pilot data for grades, placement, discipline, special-education decisions or college reports.
Infer emotion, ability, disability, mental health, family life or beliefs.
Run AI-writing detection, or train or fine-tune models on student data.
Contact students or parents directly, or publish anything about the school without written permission.

At the end

The school chooses: continue, export or delete, with deletion covering backups and derived data and confirmed in writing. Our pilot evaluation rubric turns the readout into a score you can compare across products.

[ FAQ ]

Questions and answers

How long should a school AI pilot be?

A common shape is one teacher-only week and six weeks with students, then a readout and a delayed unaided check 2-4 weeks after the last practice.

What should an AI pilot measure?

Delayed unaided success on practiced concepts, help requests after the student's own attempt, whether learning modes hold, teacher usefulness, cost per active student and safety response times. Not message counts.

Can a six-week pilot prove AI improves learning?

No. With 200 students the smallest detectable effect is about 0.40 SD, larger than published AI-tutoring effects. A pilot tests reliability and usefulness.

What has to be in place before students join a pilot?

A signed data agreement, parent notice, roster-based accounts, minors' model rules, a tested safety route, a tested filter allow-list and a real-model check of the learning modes.

[ SOURCES ]