Data Engineer (ETL)
- India
Posted yesterday
About the role
Data Engineer (ETL)
Experience: 7–8 Years
Location: Remote
Shift: 11:00 AM – 9:00 PM IST
Openings: 1
Job Summary
We are looking for an experienced Data Engineer with 7–8 years of strong hands-on experience in ETL/ELT, Python, PySpark, Azure Databricks, Snowflake, and SQL.
The ideal candidate will have a strong background in cloud data engineering and should also have experience with Terraform, CI/CD, Azure Functions, Service Bus, and AI/ML/LLM technologies.
Key Responsibilities
- Design, develop, and optimize scalable ETL/ELT data pipelines.
- Build robust data solutions using Python, PySpark, Azure Databricks, and Snowflake.
- Develop, troubleshoot, and optimize complex SQL queries and data transformations.
- Work with Azure Databricks and Unity Catalog for data processing, governance, and security.
- Implement cloud infrastructure using Terraform and Infrastructure as Code practices.
- Build and maintain CI/CD pipelines using GitHub Actions, Git, Artifactory, Sonar, and related tools.
- Integrate Azure Functions, Azure Service Bus, and other Azure cloud services.
- Monitor and optimize pipeline performance, scalability, reliability, and cost.
- Implement data quality, security, and governance best practices.
- Explore and leverage AI/ML and LLM capabilities within data engineering solutions.
- Collaborate with cross-functional teams in an Agile environment.
Required Skills
- 7–8 years of professional Data Engineering experience.
- Strong ETL/ELT development experience.
- Excellent Python programming skills.
- Strong experience with Apache Spark, PySpark, and Spark SQL.
- Hands-on experience with Azure Databricks and Unity Catalog.
- Strong experience with Snowflake.
- Advanced SQL development and query optimization skills.
- Experience with Terraform / Infrastructure as Code.
- Experience with CI/CD tools and practices, particularly:
- GitHub Actions
- Git
- Artifactory
- Sonar
- Hands-on experience with:
- Azure Functions
- Azure Service Bus
- Other Azure services
- Practical experience with AI/ML and LLM technologies.
Preferred Qualifications
- Experience working with Azure cloud-native data platforms.
- Strong understanding of data architecture and modern data engineering patterns.
- Experience with data orchestration, monitoring, performance tuning, and optimization.
- Experience working in Agile/Scrum environments.
Pay: ₹130,000.00 - ₹160,000.00 per month
Work Location: Remote