Data Engineer

There will be an opportunity to qualify for additional allowances and a skills-related payment under the Government Digital and Data Profession., Technology sits at the heart of everything we do at MI5. We're looking for experienced Data Engineers prepared to take on a more senior role within a multi-disciplinary product team. As a Data Engineer and working closely with technical specialists such as Product Owners, Business Analysts, Delivery Managers, Software Engineers, Data Scientists and Machine Learning Engineers, you'll build and run secure data applications and data products.

You'll promote Agile technical practises, using Agile methodologies to deliver digital products that are core to MI5's operation, making use of modern approaches and cloud technologies such as AWS and Azure, as well as on-premises platforms and long-established technologies and frameworks.

You'll gain experience in a challenging environment as part of a forward-thinking team and strive to develop digital solutions. Managing tasks, you'll be responsible for the design and implementation of numerous moderate-to-complex data flows, to connect operational systems, data for analytics, or business intelligence (BI) systems.

Ensuring the customer's needs are represented at every stage, you'll support the products owned by the team, working with users to identify and fix defects (providing on-call support if necessary), developing automated tests to maintain the assurance of our products and deploy through continuous integration pipelines.

You'll frame problems and set priorities for yourself and your team, taking ownership of large problems, breaking them down and working with the team to deliver new features throughout the engineering lifecycle. As a Data Engineer, you'll focus on creating professional and comprehensive digital experiences and share a collective responsibility in keeping the country safe, by staying one step ahead in an evolving digital world.

You'll set the standard for what good looks like, sharing digital and innovative ways of working with the team and across the skills community. Importantly, you'll have a passion for sharing knowledge, whether it's a customer or colleague you'll help them to understand what great engineering looks like.

Promoting best practises, you'll participate in guilds, and encourage cross-organisation initiatives to help build our community of engineers, whilst mentoring and developing junior members of the team and influencing them with your vision.

Our tech stacks vary between products (such as OracleDB, MongoDB, Elastic Search and Hadoop for data storage and a mixture of commercial-off-the-shelf products and custom applications. We embrace a DevSecOps (Development, Security, and operations) mindset, and our teams automate, run and support what we build.

Whether it's building a platform to enable the analysis of data systems to track the progress of our investigations, developing robust and scalable capabilities for our Data Science teams, delivering compliance services to ensure we are operating legally, or tooling to improve our developer experience, the nature of our work presents unique and fascinating engineering challenges, the likes of which you will never have seen before., We're committed to developing our staff and recognise this will look different for everyone. We use the Government Digital and Data Profession Capability Framework (formerly the Digital Data and Technology (DDaT) Profession Capability Framework) as a compass to provide guidance and tools to support you, and which recognises and rewards specialist expertise.

There'll be opportunities to gain qualifications and pursue specialist pathways, as well as undertaking tailored training, coaching, and mentoring. Individuals can also attend hackathons and have up to 20% innovation and personal development time.

As you develop, grow, and maintain your technical skills, you can progress through the Government Digital and Data pay points. You'll need to regularly demonstrate and accredit your skills based on the framework. If you don't maintain the required skills, you'll be given the opportunity and support to rectify this, as you could revert to a lower level of capability and pay., MI5 is proud to have achieved Leader status within the DWP's Disability Confident scheme. This is aimed at encouraging employers to think differently about disability and take action to improve how they recruit, retain, and develop disabled people. Being Disability Confident, we aim to offer a person-to-person interview to any candidate who self-identifies as disabled and meets the essential criteria for the role. This is our 'Offer of Interview' (OOI). To secure an interview for this vacancy, the essential criteria (in order of application process) are:

• Able to demonstrate ability in developing data solution (assessed at application form sift)


• Can demonstrate an understanding of the principles of modern standard approaches such as continuous integration and delivery, test driven development and cloud services (assessed at application form sift)


• Able to demonstrate taking on a more senior role within a team. Provide technical direction and the ability to guide and support others with regards to developing data products and solutions (assessed at application form sift)


What to Expect
Our recruitment process is fair, transparent, and based on merit. Here is a brief overview of each stage, in order:

• You'll be invited to upload your CV and complete an online application form.


• If you pass, you'll be invited to an interview where you will be asked questions relating to your motivation and our core competencies.


• If you pass, you'll be invited to a technical skill-based interview based on the Government Digital and Data Capability Profession framework for the role.


This will draw on skills and experience in the area(s) detailed below:

• Data Analysis and Synthesis


• Data Modelling


• Data Integration Design


• Metadata Management


• Technical Understanding (Data)


• If you pass, you'll start our vetting process.


Please note, you must successfully pass each stage of the process to progress to the next. Your application may take around 6 - 9 months to process including vetting, so we advise you to continue any current employment until you have received your final job offer.
Before You Apply
To work at MI5, you need to be a British citizen or hold dual British nationality. You can read our full eligibility criteria here.

This role requires the highest security clearance, known as Developed Vetting (DV). It's something everyone in the UK Intelligence Community undertakes. You can find out more about the vetting process here.

Please note we have a strict drugs policy, so once you start your application, you can't take any recreational drugs and you'll need to declare your previous drug usage at the relevant stage.

Before you apply, we advise you to consider setting up a separate email address for your contact with us, to ensure your personal and application correspondence remain separate.
Try to avoid having identifying features in your email address, such as your first and/or surname and date of birth. This is good practise and will help you to manage your application with us more discretely.

The role is based in London or Greater Manchester, so you'll need to live within a commutable distance. Please consider any financial implications and practicalities before submitting an application. An interest-free loan via our benevolent fund is available to assist with relocating into privately rented accommodation to take up the offer of employment.

As a Data Engineer you'll be part of a team which supports and delivers innovative digital products.

You'll need to demonstrate a proficiency in developing data solutions and data integration design. Crucially, you'll be comfortable using common data engineering technologies and you'll need to be familiar with the principles of a modern standards approach, such as continuous integration and delivery, test-driven development, and cloud services. You'll have an awareness of design patterns and how to implement them appropriately with security in mind and demonstrate proficiency in the use of the Agile methodology.

You'll be able to communicate information clearly and effectively, influencing technical and non-technical colleagues, defining, and shaping problems while working with a wide range of stakeholders to meet customer needs. You'll demonstrate leadership qualities, supporting others to upskill and build their career in an inclusive environment, while continuously looking for opportunities to develop and learn new engineering practises and approaches.

You'll have experience delivering moderate-to-complex data flows as part of a development team in collaboration with others. You'll be confident using technologies such as: Apache Kafka, Apache NiFi, SAS DI Studio, or other data integration platforms.

You can implement, deliver, and translate several data models, including unstructured data, document formats, relational models, and formats to support geo-enabled applications and graph database structures, using knowledge of various data formats including XML, JSON, flat file, Parquet etc. You can also manage metadata across your development teams systems and beyond, with a practitioner knowledge of tracking data provenance and data lineage.

You'll work to agreed and recognised standards to build solutions using various traditional or big data languages such as: SQL, PL/SQL, SAS Macro Language, Python, Scala, Apache Spark, Java, JavaScript etc, using various tools including SAS, Hue (Hive/Impala), Kibana (Elastic Search). Knowledge of data management on Cloud platforms is especially advantageous.

MI5 keeps the country safe from serious threats like terrorism and attempts by states to harm the UK, its people and way of life. We carry out investigations by obtaining, analysing, and assessing intelligence, and then work with a range of partners including MI6 and GCHQ to disrupt these threats. Through our protective security arm, we provide advice and guidance to government, businesses, and other organisations about how to keep themselves safe. A role in MI5 means you'll do unique and challenging work in a supportive and encouraging environment, making a real difference to UK national security.

Flexible Working: we offer a strong work-life balance, supporting full-time, part-time, and flexible hours. You'll be predominantly office-based, with working from home limited and subject to business need., You'll receive a starting salary of £59,479 - £66,100 dependent on skills, experience, and location plus other benefits including:

• 25 Days Annual Leave automatically rising to 30 days after 5 years' service, and an additional 10.5 days public and privilege holidays.


• Opportunities to be recognised through our employee performance scheme


• Personal Learning Budget


• Interest-free season ticket loan (travel and parking)


• Excellent pension scheme


• Cycle to work scheme


• Facilities such as a gym, restaurant, and on-site coffee bars (at some locations)


• Paid parental and adoption leave.


Equal Opportunities
At MI5 diversity and inclusion is critical to our mission. To protect the UK, we need a truly diverse workforce that reflects the society we serve. This includes diversity in every sense of the word: those with different backgrounds, ages, ethnicities, gender identities, sexual orientations, ways of thinking and those with disabilities or neurodivergent conditions. We therefore welcome and encourage applications from everyone, including those from groups that are under-represented in our workforce such as women, those from an ethnic minority background, people with disabilities and those from low socio-economic backgrounds.

Find out more about our culture, working environment and diversity on our website here.