Using data to reduce teacher shortages
DEPARTMENT FOR EDUCATION
The Department for Education (DfE) is a UK Government department responsible for children’s services, education, apprenticeships and wider skills in England.
The Teacher Services division of the Department for Education is responsible for the careers of teachers in the UK - from recruitment and training, through continuous professional development to leaving teaching and retirement.
Teacher Services contains a number of teams, each providing a different service catering to a different stage in a teacher’s journey:
The recruitment pipeline starts with Get Into Teaching, which leads prospective teachers through to Apply for Teacher Training.
Then, private interfaces for teacher training institutions allow them to manage applications for teacher training and register trainee teachers for the award of qualified teacher status.
After that, prospective employers can check a teacher's record, accusations of teacher misconduct can be reported for investigation, teachers can find their teacher reference number if they’ve lost it, and access their pensions.
The challenge
The UK is facing a teacher shortage, and the Department for Education has been tasked with addressing this. A key part of their strategy is to understand the teacher career pipeline.
An effort was already underway to integrate analytics data from the different services to try to construct an overall view of the effectiveness of recruitment programs, but it was facing technical obstacles due to the fact that the separate services had not been built with such integration in mind. There were also ongoing problems with the transactional integrations between services that needed to share data as part of their operation, such as passing details of training applicants from the application system to the registration system. These stemmed from the pressures placed upon teams to rapidly build services that met the teams’ mandates, with inter-service integration not being a priority.
Our objective was broad: to facilitate better data sharing between services, for better analytics and to ease transactional integrations between services; addressing existing acute problems, and building culture and processes to ensure that future problems were avoided.
What we did
Register Dynamics supplied an experienced Technical Architect, embedded in the cross-service analytics team.
We started by conducting a review of databases within Teacher Services, documenting the flows of data between them and which services used the different databases, in order to accumulate and summarise architectural knowledge that was previously distributed throughout the department.
This process also drove the identification of the key people involved in various projects and where their domain expertise lay. While this was ongoing, we developed written principles for the use and storage of data, documenting what teams needed to know to operate safely and ethically while guiding the development of a culture of data management that would allow teams to cater for the needs of the department as a whole while still meeting their immediate needs.
During this process, we observed that an underlying problem in both analytic and transactional integration between services was a lack of a shared means for managing reference data, such as lists of types of qualifications, subjects, and nationalities. Different services using different lists made it hard to share data between services, and services duplicated effort maintaining their own copies of lists; so we designed and built a shared repository for reference data lists.
The canonical reference data was stored as text files in a Git repository, a familiar interface for service teams to collaborate on changes, and an automated build system published approved changes into a versioned Ruby gem (easy to use for service teams) and into tables in the shared analytics database (easy to use for the analytics team).
On the technical side, this made it easy for all the teams that need a specific list to share the burden of maintaining that list and to be sure they are using the same list while also being able to control when changes are accepted into their services. At the same time, on the non-technical side it provided a means for different teams with common data interests to come together and discuss how they should handle them.
We also assisted with various smaller projects, such as building shared functionality in the analytics system to classify qualifications from different nations as meeting requirements for teacher training or not.
The result
The Reference Data Repository solved various long-standing problems relating to inconsistencies between supposedly equivalent lists maintained by different services.
It made reference data available to the Analytics System and in doing so, was incredibly valuable to the Service Teams. It saved them a significant amount of time by being easier to use than their original ad-hoc internal reference data systems, and allowing the burden of updating lists to match changes in legislation to be shared.
By creating a shared repository where proposed changes would have to be discussed and agreed upon, it created a community of data-literate people in different Service Teams and the Analytics Team, who were in contact with each other, were generally aware of each other’s needs and interests, and could easily discuss other topics of interest.
This, combined with a published set of general principles for data management for teams to consult, enabled a cross-team community of people with an interest in data sharing to guide teams in building systems that would make it easy to interoperate with other teams for transactional purposes and share data for analytical purposes; a lasting cultural change.
The repository is still in active use and is updated regularly by civil servant data architects.
Tags: