The school network reality

The pupils blamed the app for the Wi-Fi

The most useful finding in the published evaluation of theSpanish pilot has nothing to do with coding. The authors report that inadequate connectivity generated disconnections, and that pupils perceived those disconnections as a fault of the Create@School app itself rather than as an external factor. That perception, the paper says, is what led the app to be characterised as unpredictable and unruly.

Read that from a staffroom rather than a research office and it is a warning about every tool you will ever trial. A class of thirteen-year-olds does not run a traceroute. They see something that stopped working, and the thing that stopped working has a name and an icon on the home screen. The network has neither. So the tool absorbs the reputational cost of a dependency chain it does not own, in exactly the weeks you are deciding whether to keep it.

That turns the sensible question around. It is not "is this tool any good". It is "what has to be true about this building before a tool like this can survive a timetable slot".

What actually broke in Spain

The evaluation lists its problems plainly, which is rarer in this literature than it should be. Wi-Fi in the classrooms was poor. Authentication problems arising from the schools' security and privacy measures prevented free internet navigation, so pupils could not browse or fetch images during lessons, and the authors say the app was appraised more conservatively as a result. The school infrastructure did not support enough devices connected at one time: network collapse and slow connectivity lowered the usability experience. The device operating system did not meet some of the schools' own requirements, with no parental control application and no customisable settings for managing the device account. And theProject Management Dashboard was not interoperable with the school management system, so project grades could not be transferred automatically. The authors close that list by noting that these drawbacks were time-consuming, and that avoiding them could have improved the validation results for both tools.

• Devices 338 tablets and mobile devices
• Schools Two, in Úbeda and Puerto de Santa María
• Hardware 7-inch and 10-inch Android: Nexus 7, MOTOG-2, BQ Edison 3
• Pupils 308 used the tools, 115 rated the experience
• Named blockers Wi-Fi, authentication, device OS, grade export

Note what is not on that list: the app crashing, the block language being confusing, pupils finding the templates too hard. Those were not the reported obstacles. Four of the five things that got in the way were owned by the schools' IT setup, not by the software under test.

Four questions to answer before the first lesson

Every school considering a device-based tool inherits the same dependency chain, and it is short enough to check in an afternoon.

Can the room carry a whole class connecting at once?

Not the building average, and not one tablet on a free period. Twenty-five devices, in the room you will actually teach in, at the time of day you will actually teach. This is the failure the Spanish evaluation names most directly, and it is the one most often tested under the wrong conditions.

Will the security policy silently break it?

The Spanish pilot did not fail on a firewall that blocked the app outright. It failed on a policy that let the app run and then stopped pupils reaching the images they needed inside it. Ask your network manager for the actual filtering rules and check them against the actual domains the tool calls. A tool that half works is harder to teach with than one that does not launch at all, because the lesson has already started before you find out.

Does device management meet your own requirements?

The pilot schools found the device operating system short on parental controls and on manageable device accounts. That is a safeguarding question, not a technical one, and the person who signs it off should see a device before anyone orders thirty of them.

Is there a route into whatever already holds the grades?

This is the detail we would put in front of any senior leadership team. The dashboard was the best-received component in the whole study, rated by teachers as a desired product, and it still lost marks because grades had to be moved by hand. The evaluation says the missing integration diminished the application's attractiveness. If the strongest thing in the trial was marked down over an export, assume yours will be too. More on how each component scored is on how it was rated.

What the evaluation does not tell you

Everything above rests on the published paper. What follows is our own judgement, and we would rather label it than smuggle it in.

The evaluation records that connectivity failed. It does not describe what a failure does to a lesson. From experience: at minute twelve, eight of twenty-five devices drop. Three pupils tell you at once, four go quiet, and one has lost work. You cannot debug a network from the front of a classroom, and the person who could is not timetabled to be there. So the practical questions are the ones the paper never reaches. Who is expected to fix a device mid-period, and what is the escalation route when it is not you? What do the eight pupils do for the remaining half hour that is worth doing on its own terms, rather than watching a peer's screen? And is any of the lesson's assessment resting on work that only exists on a device that just dropped?

Our answer, offered as professional judgement rather than as a finding, is that a lesson built on a network needs a version of itself that runs without one: paired working so a dropped device costs half a pair rather than a whole pupil, a paper or offline stage of the task that is genuinely part of the work, and a saving point early enough that minute twelve is not expensive. None of that is in the study. It is what we would want in place before running the same lesson.

This is not a 2017 problem

It would be comfortable to file this under old hardware and move on. The Andalusian schools were running 7-inch Android tablets in classrooms wired for far less traffic than a modern school carries. But the dependency chain has not changed shape, and cloud-first tools have made it longer rather than shorter. Bandwidth for a class connecting at once, a security policy that does not silently break the tool, device management that meets the school's own requirements, and a data path into the system that already holds the grades: a tool is judged on all four, whether or not it controls any of them.

The practical consequence is a small one. Whoever runs your network should be in the room for the procurement conversation, not told about it afterwards. Other things worth checking before a tool reaches a timetable are collected on the classroom hub.