Job Title: Sr. DATA Engineer Work Location : Tampa FL Job Description: • Seeking a Senior professional with 3 to 5 years of experience in Python Scala and Spark SQL to design and optimize scalable big data processing solutions within the Databricks Spark SQL Scala Python Java ecosystem • Develop and maintain scalable data processing pipelines using Apache Spark with Python and Scala • Utilize Spark SQL to analyze large datasets and extract meaningful insights supporting data driven decision making • Collaborate with cross functional teams to integrate big data solutions into existing systems • Optimize Spark jobs for performance and efficiency in distributed computing environments • Ensure data quality and consistency across various data sources • Stay updated with the latest advancements in Spark Scala Python and related big data technologies to continuously improve processing capabilities Roles and Responsibilities • Design develop and deploy complex data processing workflows leveraging Apache Spark Scala Python and SparkSQL • Write efficient reusable and maintainable code for Spark applications • Troubleshoot and resolve performance bottlenecks in Spark jobs and data pipelines • Collaborate with data engineers’ data scientists and stakeholders to translate business requirements into technical solutions • Conduct code reviews and provide mentorship to junior team members on best practices and coding standards • Participate in architectural discussions and contribute to the evolution of big data platforms • Monitor and maintain the health availability and reliability of Spark clusters and data services Karat Interview mandate