Lead II - Data Engineering
UST · ust.com
ThiruvananthapuramLead / Manager6+ yrs
Straight from the company’s own careers site.
The job in short
AWS Glue data pipeline engineering with PySpark, Python and SQL, mentoring junior engineers.
PythonSQLSnowflakeAWS
Details
- Found
- 8 d ago · posted 2026-10-14
- Location
- Trivandrum
Full description
Role: Sr Backend Developer – Glue, Python, SQL Key Responsibilities • Design, develop, and maintain ETL/ELT pipelines using AWS Glue (PySpark and Python shell jobs), Glue Data Catalog, crawlers, and workflows • Write complex, optimized SQL for data transformation, validation, and analysis across platforms such as Amazon Redshift, Athena, and relational databases • Build reusable Python libraries, frameworks, and utilities for data ingestion, transformation, and quality checks • Model data for analytical and operational use cases (star/snowflake schemas, data lakes/lakehouse patterns) • Implement data quality, lineage, monitoring, and ing for production pipelines • Optimize job performance and cost (partitioning, file formats such as Parquet, job bookmarks, DPU tuning) • Collaborate with cross-functional teams to gather requirements and translate them into technical designs • Participate in code reviews, enforce engineering best practices, and mentor junior engineers • Support CI/CD, infrastructure-as-code, and release processes for data workloads • Troubleshoot production issues and drive root-cause analysis • 6+ years of software or data engineering experience, with 3+ years building pipelines on AWS • Strong hands-on expertise with AWS Glue (ETL jobs, Data Catalog, crawlers, triggers, workflows) and PySpark • Advanced SQL skills, including query optimization, window functions, CTEs, and performance tuning on large datasets • Proficiency in Python, including packaging, testing (pytest), and writing maintainable, well-documented code • Experience with related AWS services: S3, Lambda, Athena, Redshift, Step Functions, IAM, CloudWatch • Solid understanding of data warehousing concepts, data modeling, and lakehouse architectures • Experience with version control (Git) and CI/CD pipelines • Strong problem-solving skills and ability to work independently in an agile environment
Data Engineer jobs in Thiruvananthapuram · Data Engineer jobs in India · All UST jobs · Senior Data Engineer jobs in Thiruvananthapuram · Senior Data Engineer jobs in India · Senior and manager-level jobs in Thiruvananthapuram