M23 - Data Engineer

Overview Join the Digital Excellence & Products Division (DXD) within the Ministry of Education (MOE), where technology, data, and education come together to build impactful digital solutions. As a Data Engineer in the Geospatial Team, you will design, develop, and maintain scalable data platforms and pipelines that power geospatial analytics and spatial modelling. Your work will support evidence-based planning by enabling insights into future education demand, school locations, infrastructure planning, and resource optimisation. Responsibilities Data Engineering & Solution Design Collaborate with business users, data analysts, and stakeholders to gather requirements and translate them into scalable data solutions. Design technical architectures that align with enterprise data strategy, governance, and security standards. Develop robust and maintainable data models to support analytics and reporting. Data Pipeline Development Design, build, test, and deploy end-to-end data pipelines for batch and real-time data processing. Develop data ingestion, transformation, validation, and orchestration workflows using modern data engineering frameworks. Implement automated data quality checks, error handling, and monitoring processes. Data Platform & Infrastructure Design and optimise data architectures across Data Lakes, Data Warehouses, Lakehouses, and related platforms. Monitor, maintain, and optimise data infrastructure to ensure scalability, reliability, and cost efficiency. Troubleshoot pipeline failures, resolve data quality issues, and continuously improve platform performance. Collaboration & Continuous Improvement Work closely with cross-functional teams to deliver data-driven solutions that support business objectives. Ensure compliance with data governance, metadata management, lineage, and security standards. Evaluate and recommend new technologies, tools, and best practices to modernise the data platform. Requirements Experience Minimum 3–5 years of experience in Data Engineering, Data Analytics, or a related technical field. Proven experience designing, building, and maintaining production-grade data pipelines and analytics platforms. Technical Skills Strong proficiency in Python, including libraries such as Pandas and NumPy. Advanced SQL skills for data querying, transformation, and database management. Solid understanding of modern data architecture concepts, including: Data Lake Data Warehouse Data Lakehouse Data Mesh Experience with data ingestion, transformation, orchestration, and data quality management. Knowledge of cloud platforms, preferably AWS, and experience with Databricks is advantageous. Familiarity with infrastructure-as-code, DevOps practices, and cloud-native data platforms. Understanding of data governance, metadata management, and data lineage principles. Experience with GIS technologies such as ArcGIS, geospatial APIs, routing engines, or 2D/3D mapping technologies is a plus. Soft Skills Strong analytical and problem-solving abilities with the capability to troubleshoot complex data engineering challenges. Excellent communication skills with the ability to translate business requirements into technical solutions. Strong stakeholder management and collaboration skills across technical and non-technical teams. Self-motivated, proactive, and committed to continuous learning and improvement. Ability to work effectively in Agile, cross-functional environments while delivering scalable, high-quality data solutions.