Senior Data Engineer with Databricks Experience

At SunnyData, our mission is to help our customers in their data journey to power their business. We believe a solid foundation is the key to achieving outstanding data-driven results: highly scalable architectures, robust data engineering pipelines, simple data consumption layers, and accurate ML and AI applications. As a Senior Data Engineer with Databricks experience, you will play a critical role during this customer journey.

You will directly work with internal teams and customers to design, build, and deploy data solutions that capture, explore, transform, and utilize data to support Artificial Intelligence, Machine Learning, and Business Intelligence insights.

The Impact You Will Have

• Be part of a team responsible for supporting new and existing customers in their data engineering needs.

• Guide customers to make the best technical decisions to achieve their goals.

• Actively work across multiple customer accounts which you would need to track and report on their progress.

• Design, build, and operationalize complex data solutions, fix issues, apply transformations, and recommend data cleansing/quality solutions.

• Analyze data sources to determine their value and recommend them to include in analytical processes.

• Incorporate core data management competencies, including data governance, data security, and data quality.

• Collaborate within and across teams to support delivery and educate end users on data products/analytic environments.

• Perform data and system analysis, assessment, and resolution for defects and incidents of moderate complexity and correct as appropriate.

• Test data movement, transformation code, and data components.

What We Look For

• 7+ years of related experience in data engineering and data product development.

• 3+ years of experience in a consulting environment.

• Strong knowledge of the Databricks platform.

• Experience with legacy-to-Databricks migration projects.

• Hands-on experience with DAB, clustering techniques, and performance optimization.

• Familiarity with other data platforms and technologies.

• Experience working with Azure, AWS, or GCP, as well as CI/CD and GenAI.

• Strong SQL and PySpark skills.

• Excellent communication, collaboration, and leadership skills.

• Experience with Databricks Genie and Databricks Apps is a strong plus.

• Experience delivering 5+ projects using Databricks is a strong plus.

Education:

• Bachelor’s Degree in Computer Science or Engineering-related field.