Senior Data Engineer, N and I Business Operations

Home Office Digital designs, builds and develops services for the rest of the department and for government. Every year our systems support up to 3 million visa applications, checks on 100 million border crossings, up to 8 million passport applications and deliver 140 million police checks on people, vehicles and property.

Senior Data Engineers design and implement complex data flows to connect operational systems with analytics and BI systems. Youll build and manage data flows, collaborating with the Data Science community and Product Owners to identify strategies and services to best support business needs. Youll help set the direction for service technology and data architecture, while mentoring junior team members, promoting collaboration and encouraging an agile approach to work.

Youll get to work with some of the largest and most varied datasets around, and benefit from a wealth of continuous professional development resources and career opportunities. Youll play a key role in delivering joined-up, intelligent services that unlock the value from data and deliver better outcomes for the UK.

Where business needs allow, some roles may be suitable for a combination of office and home-based working. Where this is the case, employees will be expected to spend a minimum of 60% of their working time in the office. Applicants can raise any queries to the email address at the bottom of the advert.  

Watch this short video to hear from members of Home Office Digital talking about the projects they work on and their experience of working here: Working for Home Office Digital.

This role sits in the Central FinOps team and plays a critical role in enabling accurate cloud cost allocation, recharge, and financial transparency across Home Office. The role focuses on collecting, transforming and maintaining large volumes of AWS and Azure cost and usage data, ensuring that cloud expenditure can be accurately attributed to portfolios, programmes and services. This supports the Central FinOps team's recharge and chargeback processes, providing a reliable basis for financial management and accountability across the organisation.

A key responsibility is the development and maintenance of data pipelines, datasets and reporting solutions that underpin cloud cost dashboards, forecasting, invoicing and recharge activities. The engineer works closely with finance teams, cloud platform teams and business stakeholders to ensure data quality, consistency and traceability. They support the production of recharge datasets and dashboards, enabling portfolio teams to understand their cloud consumption and associated costs.

The role also contributes to cost optimisation, tagging compliance, data governance and cloud unit economics. By automating data collection, improving reporting accuracy and developing analytical capabilities, the role helps Home Office manage significant public cloud expenditure effectively.

Senior Data Engineers design and lead the implementation of complex data flows to connect operational systems, data for analytics and BI systems.

As a Senior Data Engineer, you will identify, build, modify and manage data flows between complex systems using the appropriate solution.

You will be working with the Data Science community, who set the scope of data engineering work through user research and data pipelines. Through sharing work with Product Owners, you will identify strategies for how the service is best able to support business needs.

You will work with fellow members of the Data Engineering community to set the direction of the service technology and data architecture. You will also mentor more junior members of the team - promoting challenge, collaborating and encouraging an agile approach to working.

Main Responsibilities

• Analysing problems and experimenting with possible solutions to find the underlying causes of issues or discrepancies to identify problems and to assist in the development of innovative solutions.
• Analysing and reporting test activities and results, supporting data engineers and stakeholders on data analysis issues.
• Designing and implementing a data streaming service, including the development of new data models and ETL processes.
• Ensuring the successful delivery of completed data loads for customers, Data Analysts and Data Scientists, troubleshooting where required.
• Designing, building and testing data products and solutions that are complex or large scale, through full development, test and deployment lifecycles.
• Applying concepts and principles of data modelling and producing relevant and varied data models across multiple subject areas and providing guidance on how to use them.

Working Pattern

Due to the business requirements of this role, it is only available on a full-time basis. However, compressed hours are available.

Essential Skills

• Working experience of finance processes, procedures, and controls (ideally in UK Government), processing financial transactions accurately and in line with delegated authorities, financial regulations, and audit requirements, including an ability to identify, investigate, and resolve financial discrepancies in a timely manner.
• Strong understanding of financial governance, risk management, and internal control frameworks, excellent attention to detail and commitment to accuracy in financial processing.
• Practical experience of receipting and invoicing, including purchase-to-pay (P2P) and order-to-cash (O2C) processes, working with finance systems and enterprise resource planning (ERP) platforms, e.g. Oracle.
• Experience of working collaboratively with suppliers, commercial teams, and operational stakeholders to ensure compliance with contractual and financial requirements, with an ability to communicate complex financial information clearly to a range of stakeholders.
• The delivery of projects in data analysis, solution design and end user reporting.
• Experience of any of the following: IT programming and automation, using tools such as industry standard ETL tools, network databases and scheduling and orchestration tools, using modern / open source programming languages in order to deliver data development / engineering products, using public cloud data technologies.
• Effectively managing and communicating with non-technical and senior stakeholders about performance and analysis.

SFIA capability framework 

Skills for the Information Age (SFIA) version 8 is the technical framework that sets the standard capability and development of all levels in the Home Office. This is a link to the capability framework:  All skills A - Z English (sfia-online.org).

We use set SFIA technical skills to form our interview questions and we will assess you against these technical skills during the selection process.

The essential skills listed above are reflective of the Home Office Government Digital and Data Profession Career Framework (based on the industry standard SFIA framework). Use the SFIA Levels of responsibilityto understand what would be expected for each technical skills listed below.