Senior Engineer Big Data

We are in search of an experienced Senior Engineer who possesses advanced skills in Big Data technologies and Amazon Web Services (AWS). If you are an accomplished Senior Software Engineer with a robust background in Big Data technologies and an expert in AWS we encourage you to apply. Join our team and lead the design, development, and optimization of data solutions that power our cross-functional endeavors and drive excellence.

Responsibilities
• Design, implement, and optimize distributed computing systems for big data processing.
• Develop and maintain Apache Spark-based solutions for data processing and analytics.
• Write efficient and scalable code using programming languages such as Python, Java, or Scala.
• Utilize Hadoop v2, MapReduce, HDFS, and Sqoop for data storage, retrieval, and processing.
• Build stream-processing systems using technologies like Apache Storm or Spark Streaming.
• Implement messaging systems such as Kafka or RabbitMQ for real-time data processing.
• Utilize Big Data querying tools like Hive and Impala for data analysis and retrieval.
• Integrate data from multiple sources, including RDBMS, ERP systems, and files.
• Design and optimize SQL queries, joins, stored procedures, and relational schemas.
• Work with NoSQL databases like HBase, Cassandra, or MongoDB for specific use cases.
• Apply ETL techniques and frameworks for data transformation and integration.
• Perform performance tuning of Spark jobs to optimize data processing and analytics.
• Have experience with native Cloud data services, preferably AWS, Azure, or GCP.
• Demonstrate leadership skills in efficiently managing and leading a team.
• Design and implement end-to-end big data solutions that meet business requirements.
• Practice Agile methodology for project management and delivery.

Requirements
• 5 to 8 years of experience in Big Data and related technologies.
• Expert-level understanding of distributed computing principles.
• Extensive knowledge and experience with Apache Spark.
• Proficiency in programming languages such as Python, Java, or Scala.
• Strong familiarity with Hadoop v2, MapReduce, HDFS, and Sqoop.
• Experience building stream-processing systems using Apache Storm or Spark Streaming.
• Knowledge of messaging systems like Kafka or RabbitMQ.
• Good understanding of Big Data querying tools like Hive and Impala.
• Experience integrating data from multiple sources, including RDBMS and ERP systems.
• Proficiency in SQL queries, joins, stored procedures, and relational schemas.
• Familiarity with NoSQL databases like HBase, Cassandra, or MongoDB.
• Knowledge of ETL techniques and frameworks.
• Experience with performance tuning of Spark jobs.
• Experience with native Cloud data services, preferably AWS, Azure, or GCP.
• Ability to lead a team efficiently.
• Experience in designing and implementing Big Data solutions.
• Practitioner of Agile methodology for project management and delivery.

Technologies
• Data Engineering
• AWS

We offer
• Insurance Coverage
• Paid Leaves – including maternity, bereavement, paternity, and special COVID-1+ leaves.
• Financial assistance for medical crisis
• Retiral Benefits – VPF and NPS
• Customized Mindfulness and Wellness programs
• EPAM Hobby Clubs
• Hybrid Work Model
• Soft loans to set up workspace at home
• Stable workload
• Relocation opportunities with ‘EPAM without Borders’ program
• Certification trainings for technical and soft skills
• Access to unlimited LinkedIn Learning platform
• Access to internal learning programs set up by world class trainers
• Community networking and idea creation platforms
• Mentorship programs
• Self-driven career progression tool