Verified Job
Other Jobs
Data engineering
Other Jobs
#800250
Remote / WFH
Randtad
Job Description
Roles & Responsibilities:
- Expected to be an SME, collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5+ years of overall IT experience, which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design, build and deployment of Python-based applications.
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR, API Gateway, RDS instance, and Lambda.
- Understand complex data sets and ETL processes, and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
- Expected to be an SME, collaborate and manage the team to perform.
- Responsible for team decisions.
- Engage with multiple teams and contribute on key decisions.
- Provide solutions to problems for their immediate team and across multiple teams.
- Lead the coordination of project activities to ensure timely delivery of software components.
- Mentor junior team members to support their professional growth and skill development.
- Facilitate effective communication between stakeholders to align technical solutions with business objectives.
- Collaborate with other teams to integrate Spark jobs into the overall data pipeline.
- Monitoring and tuning data loads and queries
Professional & Technical Skills:
- Must To Have Skills: Proficiency in Data Engineering.
- Strong knowledge of data pipeline development and data integration techniques.
- 5+ years of overall IT experience, which includes building & deployment of Bigdata applications using PySpark and AWS cloud.
- Extensive experience in design, build and deployment of Python-based applications.
- Hands-on experience in relational databases preferably Oracle and PostgreSQL and writing complex SQL queries.
- Experience in various AWS services such as EMR, API Gateway, RDS instance, and Lambda.
- Understand complex data sets and ETL processes, and how they can be optimized using Spark.
- Knowledge of Java and Snowflake would be an added advantage
