Sr. Manager - Data Engineering

Leadsquared is seeking a hands-on result-oriented Senior Manager - Data Engineering to lead and scale our data engineering. This critical leadership role will be responsible for defining and executing the data strategy, building and managing high-performing data engineering teams, and ensuring the availability, quality, and governance of data that fuels our cutting-edge data models and applications. You will partner closely with data Engineers, Data Science, ML Engineering, Product, and other stakeholders to drive innovation and deliver significant business impact through data-driven AI/ML solutions

Responsibilities
• Strategic Leadership: Develop and champion the overall data engineering strategy and roadmap aligned with the company's data lake vision, ML AI objectives and business goals.
• Team Building and Management: Recruit, mentor, and lead multiple teams of data engineers, fostering a culture of technical excellence, collaboration, and continuous improvement.
• Organizational Scaling: Design and implement the organizational structure and processes necessary to scale the data engineering function to meet the growing demands of our AI/ML initiatives.
• AI/ML Data Exposure: Understanding of data requirements for various AI/ML techniques (e.g., supervised learning, unsupervised learning, deep learning, NLP) and design data pipelines accordingly.
• Technology Selection: Evaluate and recommend appropriate data technologies and tools (e.g., cloud platforms, data warehouses, data lakes, ETL/ELT tools, streaming platforms, feature stores) based on project requirements and industry best practices.
• Pipeline Development: Oversee the development and implementation of scalable and reliable data pipelines using tools like Spark, Flink, Kafka, Airflow, or similar.
• Data Quality and Governance: Establish and enforce data quality standards, implement data governance policies, and ensure data security and compliance.
• Performance Optimization: Identify and resolve performance bottlenecks in data pipelines and infrastructure to ensure efficient data processing.
• Cloud Expertise: Leverage cloud platforms (e.g., AWS,..) and their data services to build and deploy scalable and cost-effective data solutions.
• Documentation: Create and maintain comprehensive documentation for data pipelines, data models, and data infrastructure.
• Innovation: Stay up-to-date with the latest trends and technologies in data engineering and AI/ML, and proactively propose and implement innovative solutions.

Qualifications
• Bachelor's or master's degree in computer science, Engineering, or a related field.
• 10+ years of experience in data engineering, with a significant focus on building data solutions for applications.
• Proven experience leading and mentoring data engineering teams.
• Deep understanding of data warehousing concepts, data modelling techniques, and database systems (both SQL and NoSQL).
• Strong proficiency in programming languages such as Java, Go, Python and SQL.
• Hands-on experience with big data processing frameworks like Spark or Flink.
• Experience with ETL/ELT tools and data integration technologies.
• Experience with data streaming platforms like Kafka or Kinesis.
• Solid understanding of cloud platforms (AWS, Azure, or GCP) and their data engineering services.
• Experience with feature stores and feature engineering pipelines is highly desirable.
• Knowledge of data governance principles and data quality management.
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and collaboratively in a fast-paced environment. Read more