Required Technical Skill Set:-
• Knowledge of Spark.
1. Write code in Scala to do ETL with Spark data frames.
2. Read and write to hive tables.
3. Read other dev’s code and update/augment based on requirements.
• Use Intelli-J for coding and GIT as a code repo. Peer review code and maintain clean branches ,autosys
Must-Have:-
• Knowledge of Spark.
1. Write code in Scala to do ETL with Spark data frames.
2. Read and write to hive tables.
3. Read other dev’s code and update/augment based on requirements.
Use Intelli-J for coding and GIT as a code repo. Peer review code and maintain clean branches, Autosys
Good-to-Have:-
• Knowledge of Hadoop:
1. Hive, HDFS, Yarn.
2. Able to query Hive tables and validate counts/logic.