Verified Job
Engineering - Architecture
Sr. Manager - Data Engineer
Engineering - Architecture
#2642
Remote / WFH
Transunion
Job Description
Job description
About Us
We are India’s leading credit information company and have established ourselves as a key anchor in the credit ecosystem of the country over the past 25 years. As an information and analytics provider, we aim to bridge the gap between stakeholders across multiple ecosystems. We use information for good by analyzing credit information to generate solutions for institutions and consumers. Our solutions facilitate easy access to credit for consumers and provide valuable insights to lenders when evaluating potential borrowers. We are in the business of building trust between organizations and consumers. It is our core belief that trust can help organizations and consumers transact with ease and help them achieve great things. We are committed to not just being an equal opportunity employer but also actively cultivating a culture of inclusion and belonging. We celebrate and respect the unique backgrounds and experiences that every individual brings to our team. Applicants are evaluated based on job qualification - not race, color, *** / gender, religion, caste, national origin, age, disability, marital status, citizenship status, ****** orientation, gender identity or any other status. We are committed to taking affirmative action to employ and advance minorities, women, and qualified disabled individuals. We ensure a safe, productive, and harassment-free workplace for all. We are committed to providing reasonable accommodations for individuals with disabilities throughout the application and employment process. Please contact us to discuss any accommodations you may need. Culture and Values Our culture is welcoming, energetic, innovative, and deeply committed to fostering an environment of belonging, where diverse perspectives and backgrounds are valued. There’s an overall synergy that flows throughout the company, creating a sense of connect, belonging and unity in knowing that we’re all working to achieve the same overall goal. Our core values which we live by every day are integrity, People, Customer, and Innovation. https://www.transunion.com/privacy/global-job-applicant What is excitement and passion for us? We define it as a blend of curiosity, ability to unlearn and yet continuously learn, able to connect with meaning and finally the drive to execute ideas till the last mile is achieved. This passion helps us focus on continuous improvement, creative problem solving and collaboration which ensures delivery excellence. One world Center, 19th Floor, T2A-T2B, Senapati Bapat Marg, Lower Parel, Mumbai –400 013. Dynamics of the Role The incumbent will play a vital role in setting up data analytics technology practice in TransUnion UK. This practice will be having functions like Data Engineering, Data Research, Data Visualization, Analytics Systems and Innovation Culture. This role will be a bridge between GT and other verticals like ISG and DSA. Accelerating and transforming the Data Analytics Technology by helping TransUnion build more trust using advanced data analytics Roles &
Responsibilities Data Engineering & Scalable Pipelines Design and build distributed, scalable data pipelines for batch and real-time processing. Develop reusable ingestion, transformation, and processing frameworks using Spark and modern lakehouse patterns. Implement best practices for schema management, data versioning, partitioning, and performance tuning. Work on data modeling (star schema, data vault, wide tables) and optimize storage formats (Parquet, Delta/Iceberg/Hudi).
Advanced Apache Spark Develop
Spark applications using Spark SQL, DataFrames, and Structured Streaming.
Perform Spark performance optimization including: o Executor & memory tuning o Partitioning, file sizing, shuffle optimization o Adaptive Query Execution, caching & broadcast strategies Build Spark-based reusable modules for ingestion, ETL/ELT, and data quality.
Cloud Data
Engineering (AWS / Azure / GCP) Lead data engineering initiatives on public cloud platforms (AWS / Azure / GCP). Implement infrastructure-as-code for data platforms using Terraform or cloud-native tooling. Design secure, multi-tenant data environments with role-based access control, encryption, and auditability. Data Quality, Testing & Observability Build data quality, reconciliation, and validation frameworks embedded into pipelines. Implement observability for data pipelines, including: o SLA monitoring One world Center, 19th Floor, T2A-T2B, Senapati Bapat Marg, Lower Parel, Mumbai –400 013. o Pipeline health checks o Failure detection and recovery mechanisms Ensure data systems are designed for resilience, idempotency, and reprocessing. Analytics & ML Enablement Support the creation of data sets for analytics and machine learning, ensuring consistency and availability. Contribute to ML pip
About Us
We are India’s leading credit information company and have established ourselves as a key anchor in the credit ecosystem of the country over the past 25 years. As an information and analytics provider, we aim to bridge the gap between stakeholders across multiple ecosystems. We use information for good by analyzing credit information to generate solutions for institutions and consumers. Our solutions facilitate easy access to credit for consumers and provide valuable insights to lenders when evaluating potential borrowers. We are in the business of building trust between organizations and consumers. It is our core belief that trust can help organizations and consumers transact with ease and help them achieve great things. We are committed to not just being an equal opportunity employer but also actively cultivating a culture of inclusion and belonging. We celebrate and respect the unique backgrounds and experiences that every individual brings to our team. Applicants are evaluated based on job qualification - not race, color, *** / gender, religion, caste, national origin, age, disability, marital status, citizenship status, ****** orientation, gender identity or any other status. We are committed to taking affirmative action to employ and advance minorities, women, and qualified disabled individuals. We ensure a safe, productive, and harassment-free workplace for all. We are committed to providing reasonable accommodations for individuals with disabilities throughout the application and employment process. Please contact us to discuss any accommodations you may need. Culture and Values Our culture is welcoming, energetic, innovative, and deeply committed to fostering an environment of belonging, where diverse perspectives and backgrounds are valued. There’s an overall synergy that flows throughout the company, creating a sense of connect, belonging and unity in knowing that we’re all working to achieve the same overall goal. Our core values which we live by every day are integrity, People, Customer, and Innovation. https://www.transunion.com/privacy/global-job-applicant What is excitement and passion for us? We define it as a blend of curiosity, ability to unlearn and yet continuously learn, able to connect with meaning and finally the drive to execute ideas till the last mile is achieved. This passion helps us focus on continuous improvement, creative problem solving and collaboration which ensures delivery excellence. One world Center, 19th Floor, T2A-T2B, Senapati Bapat Marg, Lower Parel, Mumbai –400 013. Dynamics of the Role The incumbent will play a vital role in setting up data analytics technology practice in TransUnion UK. This practice will be having functions like Data Engineering, Data Research, Data Visualization, Analytics Systems and Innovation Culture. This role will be a bridge between GT and other verticals like ISG and DSA. Accelerating and transforming the Data Analytics Technology by helping TransUnion build more trust using advanced data analytics Roles &
Responsibilities Data Engineering & Scalable Pipelines Design and build distributed, scalable data pipelines for batch and real-time processing. Develop reusable ingestion, transformation, and processing frameworks using Spark and modern lakehouse patterns. Implement best practices for schema management, data versioning, partitioning, and performance tuning. Work on data modeling (star schema, data vault, wide tables) and optimize storage formats (Parquet, Delta/Iceberg/Hudi).
Advanced Apache Spark Develop
Spark applications using Spark SQL, DataFrames, and Structured Streaming.
Perform Spark performance optimization including: o Executor & memory tuning o Partitioning, file sizing, shuffle optimization o Adaptive Query Execution, caching & broadcast strategies Build Spark-based reusable modules for ingestion, ETL/ELT, and data quality.
Cloud Data
Engineering (AWS / Azure / GCP) Lead data engineering initiatives on public cloud platforms (AWS / Azure / GCP). Implement infrastructure-as-code for data platforms using Terraform or cloud-native tooling. Design secure, multi-tenant data environments with role-based access control, encryption, and auditability. Data Quality, Testing & Observability Build data quality, reconciliation, and validation frameworks embedded into pipelines. Implement observability for data pipelines, including: o SLA monitoring One world Center, 19th Floor, T2A-T2B, Senapati Bapat Marg, Lower Parel, Mumbai –400 013. o Pipeline health checks o Failure detection and recovery mechanisms Ensure data systems are designed for resilience, idempotency, and reprocessing. Analytics & ML Enablement Support the creation of data sets for analytics and machine learning, ensuring consistency and availability. Contribute to ML pip
