Verified Job
Engineering - Architecture
Data Engineering Lead (GCP, Bigquery, ETL, Prompt Engineering) Associate Director
Engineering - Architecture
#824465
Remote / WFH
HSBC
Job Description
If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.
HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.
We are currently seeking an experienced professional to join our team in the role of Associate Director, Data and Analytics Specialist
In This Role, You Will
• Own end-to-end data platform and pipeline architecture (batch/streaming), defining patterns, standards, and reference designs aligned to HSBC engineering and risk principles.
• Provide hands-on technical leadership across core GCP services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Composer (Airflow), Cloud Run/GKE, IAM, Cloud KMS, VPC, and related tooling. Lead development of resilient ELT/ETL pipelines, implementing orchestration, scheduling, monitoring, alerting, and operational runbooks to meet SLAs/SLOs.
• Drive robust data modelling approaches (dimensional, data vault, lakehouse patterns where appropriate), metadata strategy, and performance optimisation for analytical workloads (especially BigQuery). Ensure solutions meet HSBC requirements for data governance, lineage, access control, encryption, auditability, retention, and privacy-by-design; partner with Risk/Compliance/InfoSec as needed.
• Establish engineering quality bars including CI/CD, automated testing, code reviews, data quality checks, release governance, and incident/problem management practices.
Act as a senior technology partner to Product Owners, Architects, Data Analysts/Scientists, and business stakeholders; translate requirements into clear delivery plans and manage expectations. Lead, coach, and develop a team of data engineers (and partner resources), shaping ways of working, capacity planning, and delivery execution in an agile environment.
• Manage dependencies across squads and third parties, ensuring timely delivery, transparent reporting, and effective resolution of blockers, risks, and issues. Drive continuous improvement and FinOps practices—optimising GCP spend (e.g., BigQuery optimisation), improving performance, and standardising best practices for reusability and scalability.
• Define and track key performance indicators (KPIs), including DORA metrics, to measure the effectiveness of engineering practices; report progress and outcomes to senior management and stakeholders.Stay up to date with emerging technologies and industry trends to drive innovation within the engineering practice; mentor and coach team members in application architecture, data management, and AI technologies.
• Accountable with the Head of Architecture to review requirements from a testability perspective and formulate the non-functional testing strategy; ensure service resilience, sustainability, and recovery time objectives are met for all software solutions delivered. Leverage AI productivity tools (e.g., Microsoft Copilot, Claude, and similar platforms) to improve engineering effectiveness and support delivery outcomes.
To be successful in this role, you should meet the following requirements:
• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred. Significant experience in data engineering and cloud platform delivery, including end-to-end solution design and architecture for data platforms and pipelines.
• Strong hands-on experience with GCP data/platform services (e.g., BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, Cloud Storage) and cloud foundations (IAM, networking, encryption/KMS).
Proven expertise in data modelling and designing performant analytical data structures (e.g., dimensional modelling; data vault/lakehouse patterns beneficial).
• Experience building and operating production-grade ELT/ETL pipelines with strong observability (monitoring/alerting), operational readiness, and runbook-driven support.
• Strong understanding of data governance and control requirements in regulated environments (security, privacy-by-design, lineage, auditability, retention). Demonstrable experience establishing engineering best practices (CI/CD, automated testing, code quality, release governance) and improving reliability.
• Proven leadership experience managing and developing engineers, including coaching, performance support, and delivery execution in an agile environment.
• Excellent stakeholder management, communication, and interpersonal skills with the ability to influence across a complex organisation.
• Experience working with third-party vendors/partners and managing delivery dependencies. Exposure to AI concepts and/or participation in AI-related projects is advantageous.
HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.
We are currently seeking an experienced professional to join our team in the role of Associate Director, Data and Analytics Specialist
In This Role, You Will
• Own end-to-end data platform and pipeline architecture (batch/streaming), defining patterns, standards, and reference designs aligned to HSBC engineering and risk principles.
• Provide hands-on technical leadership across core GCP services such as BigQuery, Cloud Storage, Pub/Sub, Dataflow, Dataproc, Composer (Airflow), Cloud Run/GKE, IAM, Cloud KMS, VPC, and related tooling. Lead development of resilient ELT/ETL pipelines, implementing orchestration, scheduling, monitoring, alerting, and operational runbooks to meet SLAs/SLOs.
• Drive robust data modelling approaches (dimensional, data vault, lakehouse patterns where appropriate), metadata strategy, and performance optimisation for analytical workloads (especially BigQuery). Ensure solutions meet HSBC requirements for data governance, lineage, access control, encryption, auditability, retention, and privacy-by-design; partner with Risk/Compliance/InfoSec as needed.
• Establish engineering quality bars including CI/CD, automated testing, code reviews, data quality checks, release governance, and incident/problem management practices.
Act as a senior technology partner to Product Owners, Architects, Data Analysts/Scientists, and business stakeholders; translate requirements into clear delivery plans and manage expectations. Lead, coach, and develop a team of data engineers (and partner resources), shaping ways of working, capacity planning, and delivery execution in an agile environment.
• Manage dependencies across squads and third parties, ensuring timely delivery, transparent reporting, and effective resolution of blockers, risks, and issues. Drive continuous improvement and FinOps practices—optimising GCP spend (e.g., BigQuery optimisation), improving performance, and standardising best practices for reusability and scalability.
• Define and track key performance indicators (KPIs), including DORA metrics, to measure the effectiveness of engineering practices; report progress and outcomes to senior management and stakeholders.Stay up to date with emerging technologies and industry trends to drive innovation within the engineering practice; mentor and coach team members in application architecture, data management, and AI technologies.
• Accountable with the Head of Architecture to review requirements from a testability perspective and formulate the non-functional testing strategy; ensure service resilience, sustainability, and recovery time objectives are met for all software solutions delivered. Leverage AI productivity tools (e.g., Microsoft Copilot, Claude, and similar platforms) to improve engineering effectiveness and support delivery outcomes.
To be successful in this role, you should meet the following requirements:
• Bachelor’s degree in Computer Science, Engineering, or a related field; Master’s degree preferred. Significant experience in data engineering and cloud platform delivery, including end-to-end solution design and architecture for data platforms and pipelines.
• Strong hands-on experience with GCP data/platform services (e.g., BigQuery, Dataflow, Pub/Sub, Composer/Airflow, Dataproc, Cloud Storage) and cloud foundations (IAM, networking, encryption/KMS).
Proven expertise in data modelling and designing performant analytical data structures (e.g., dimensional modelling; data vault/lakehouse patterns beneficial).
• Experience building and operating production-grade ELT/ETL pipelines with strong observability (monitoring/alerting), operational readiness, and runbook-driven support.
• Strong understanding of data governance and control requirements in regulated environments (security, privacy-by-design, lineage, auditability, retention). Demonstrable experience establishing engineering best practices (CI/CD, automated testing, code quality, release governance) and improving reliability.
• Proven leadership experience managing and developing engineers, including coaching, performance support, and delivery execution in an agile environment.
• Excellent stakeholder management, communication, and interpersonal skills with the ability to influence across a complex organisation.
• Experience working with third-party vendors/partners and managing delivery dependencies. Exposure to AI concepts and/or participation in AI-related projects is advantageous.
