The pilot schools

The planned scale

The trials were designed to reach roughly 600 pupils in three countries, Spain, Austria and the United Kingdom, matching the countries of the consortium partners, across somewhere between nine and twelve school subjects. The spread was deliberate. The question was not whether motivated children enjoy making games, but whether game creation survives contact with a normal timetable and a normal mixed-ability class.

The number of schools depends on which document you read. CORDIS puts the planned scale at five schools; the published evaluation describes the same two-cycle experiment as reaching over 600 students from eight different schools. Both are project-level statements about the same project, so we give both rather than quietly picking one. For the findings below it makes no difference: the Spanish pilot was two schools, and that is what the results rest on.

The Spanish pilot

Two state schools took part, one in Úbeda and one in Puerto de Santa María, both in Andalusia, over two consecutive academic years between 2015 and 2017.

• Schools Two: Úbeda and Puerto de Santa María, Andalusia
• Pupils 308 used and validated the tools; 115 rated the experience
• Ages 8 to 17, 6th to 11th grade
• Split About 45% girls, 54% boys (as printed)
• Educators 16
• Devices 338 tablets and mobile phones
• Duration Two academic years, one per cycle

The two pupil figures are not interchangeable. 308 pupils used the tools and technically validated them across both cycles; 115 filled in the surveys that produced the ratings. Of the 16 educators, six worked only with the preliminary Pocket Code version, six only with Create@School and four with both. The gender split is printed as about 45% girls and 54% boys, which sums to 99, and we are not going to guess at the missing point.

Two cycles, two sets of subjects

The first academic year used Pocket Code in class, in a version that came before the school edition existed, to train pupils and teachers in coding and to produce a first set of templates and technical modules. The subjects that year were science, enrichment (a multidisciplinary programme available in Andalucía), mathematics, and PEMAR, a programme for children with attention deficit disorders.

In the second year Create@School and the teachers' Project Management Dashboard were completed, refined and validated in lessons, and the subject list widened: computing, mathematics, science, programming basics, biology, geology, language, social sciences and enrichment. So the pilot did not test a finished product for two years. It built the product while teaching with it, which is worth holding in mind when you read the ratings.

The British and Austrian sites

We can describe these two as planned and no further. The evaluation's authors explain that because the three pilot sites had different contexts, three separate evaluation studies were planned, one each for Spain, the United Kingdom and Austria, and that the paper published in 2019 presents the Spanish one. We have not located the British or the Austrian study. Readers who came here for the Graz end of the project, and there will be some, should know that nothing on this page speaks to it.

That limit matters for anyone citing this material. A finding from two schools in Andalusia is not a finding about Nottingham or Graz, and this page will not present it as one. If you know where either of the other two evaluations was published, we would be glad to hear it.

What the evaluation measured

Acceptance and user experience, using the Hassenzahl model with AttrakDiff surveys, which ask whether people found a tool practical, stimulating, identity-fitting and attractive. Not attainment. Nothing in the study shows that pupils learned more mathematics, or more of anything else, than they would have done otherwise, and it was never designed to. The headline results are that teachers rated the dashboard a desired product and the 115 pupils rated the app neutral.

The full practical account, including the pupil and teacher ratings, the consent arrangements and what the school network did to both tools, is onthe Spanish pilot schools.