Verified Job
IT / Software / Data Analyst
Senior Software Engineer-Python programming, pipeline and SQL-
IT / Software / Data Analyst
#568745
Remote / WFH
Job Description
Position: Senior Software Engineer (Python)
Skills: Python Programming, Data structure and algorithms, Data pipeline (mandate skills), SQL
Experience: minimum 5 years of experience is mandatory
About the Role
Reporting to the Software Engineering Manager as a member of the Development team at IDC, the Principal Software Engineer is responsible for delivering quality and performant software and design to handle the vast array of use cases that our customers have today. This role is responsible for developing software solutions by learning information needs, discussing with managers, studying systems flow, data usage, finding problem areas, coming up with solutions & following the software development lifecycle.
Responsibilities • Develop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipeline. • Recommend and implement ways to improve data reliability, efficiency, and quality. • Identify performant features and make them universally accessible to our teams across EE. • Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business-critical decisions. • Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build. • Define and evangelise data engineering best standards and practices to ensure engineering excellence at every stage of the development cycle. • Act as a resident expert for data engineering, feature engineering, and exploratory data analysis.
Qualifications, Skills and Experiences
• 5-8 years of experience in Python programming and data structures and algorithms.
• Experience in developing data pipelines for large-scale, complex datasets from varieties of data sources.
• Data engineering expertise with strong experience working with Python, Beautiful Soup, and web scraping.
• Knowledge of cloud-based data technologies such as Azure Data Lake, Azure Data Factory, and Azure Data Bricks is optionally desirable.
• Moderate coding skills. SQL or similar is required. C# or other languages are strongly preferred.
• Outstanding communication and collaboration skills. You can learn from and teach others.
• Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services.
Skills: Python Programming, Data structure and algorithms, Data pipeline (mandate skills), SQL
Experience: minimum 5 years of experience is mandatory
About the Role
Reporting to the Software Engineering Manager as a member of the Development team at IDC, the Principal Software Engineer is responsible for delivering quality and performant software and design to handle the vast array of use cases that our customers have today. This role is responsible for developing software solutions by learning information needs, discussing with managers, studying systems flow, data usage, finding problem areas, coming up with solutions & following the software development lifecycle.
Responsibilities • Develop, test and maintain architectures, such as databases and large-scale processing systems using high-performance data pipeline. • Recommend and implement ways to improve data reliability, efficiency, and quality. • Identify performant features and make them universally accessible to our teams across EE. • Work together with data analysts and data scientists to wrangle the data and provide quality datasets and insights to business-critical decisions. • Take end-to-end responsibility for the development, quality, testing, and production readiness of the services you build. • Define and evangelise data engineering best standards and practices to ensure engineering excellence at every stage of the development cycle. • Act as a resident expert for data engineering, feature engineering, and exploratory data analysis.
Qualifications, Skills and Experiences
• 5-8 years of experience in Python programming and data structures and algorithms.
• Experience in developing data pipelines for large-scale, complex datasets from varieties of data sources.
• Data engineering expertise with strong experience working with Python, Beautiful Soup, and web scraping.
• Knowledge of cloud-based data technologies such as Azure Data Lake, Azure Data Factory, and Azure Data Bricks is optionally desirable.
• Moderate coding skills. SQL or similar is required. C# or other languages are strongly preferred.
• Outstanding communication and collaboration skills. You can learn from and teach others.
• Strong drive for results. You have a proven record of shepherding experiments to create successful shipping products/services.
