Principal Solutions Architect (Databricks) - SunnyData

The Impact You Will Have

SunnyData is a leading Databricks technology partner, helping customers build scalable architectures, robust data engineering pipelines, and advanced ML and AI applications. As a Principal Solutions Architect, you'll drive customer success on two fronts: hands-on technical execution and presales support, splitting your time evenly between the two.

Key Responsibilities:

• Customer Engagement: Serve as a trusted advisor to customers, guiding their data engineering and architecture strategy with a focus on Databricks solutions.

• Technical Delivery: Design, build, and deploy comprehensive data solutions that capture, transform, and leverage data for AI, ML, and business intelligence initiatives, while driving innovation through thought leadership on data trends.

• Presales Support: Partner with sales teams to present technical solutions to prospective clients, demonstrating the value of SunnyData's offerings.

• Project Oversight: Manage multiple customer accounts, ensuring timely delivery and clear reporting on progress and outcomes.

• Solution Design: Architect data solutions incorporating best practices in governance, security, and data quality.

• Data Analysis: Evaluate data sources for their value and recommend inclusion strategies that strengthen analytics.

• Cross-Functional Collaboration & Mentorship: Guide internal teams, provide mentorship to project teams, and educate end users on data products and analytic environments.

• Problem Resolution: Perform system analysis, diagnose data and system defects, and apply the right fix.

• Quality Assurance: Test data movement, transformation code, and data components for accuracy and reliability.

Required Experience:

• Experience: 10+ years as a hands-on Solutions Architect and/or Senior Data Engineer, with recent experience implementing data solutions on Databricks.

• Technical Proficiency: Expertise in data engineering technologies (Spark, Hadoop, Kafka), the Databricks platform, and data science/ML tooling (pandas, scikit-learn).

• Architecture & Solution Design: Deep understanding of the end-to-end data analytics workflow (data modeling, ETL, data integration) using modern engineering techniques. Able to run complex architecture discovery, solution design sessions, and build implementation blueprints.

• Programming Skills: Proficiency in Java, Python, and/or Scala.

• Cloud Platforms: AWS, Azure, and/or GCP.

• SQL Expertise: Ability to write, debug, and optimize SQL queries.

• Client Facing Skills: Strong written and verbal communication, with experience in client-facing roles.

• Presentation Skills: Ability to build and deliver detailed presentations to clients and stakeholders.

• Documentation: Experience producing detailed solution documentation, including POCs, roadmaps, sequence diagrams, class hierarchies, and logical system views.

• End-to-End Solutions: Ability to take technical solutions into production, ensuring performance, security, scalability, and robust data integration.

• Team Leadership: Experience managing, mentoring, and growing a team of engineers.

Preferred Experience:

• Distributed Storage: Familiarity with cloud and distributed data storage systems such as S3, ADLS, HDFS, GCS, Kudu, ElasticSearch/Solr, Cassandra, or other NoSQL storage systems.

• Data Integration: Experience with data integration technologies like Spark, Kafka, StreamSets, Matillion, Fivetran, NiFi, AWS Data Migration Services, Azure Data Factory, and Informatica Intelligent Cloud Services (IICS).

• Automated Pipelines: Expertise in automated data transformation and curation using tools like dbt, Spark, Spark Streaming, and automated pipelines.

• Workflow Management: Experience with workflow management and orchestration tools like Airflow, AWS Managed Airflow, Luigi, and NiFi.

• Domain Knowledge: Background working in the Financial Services industry, preferably in a banking environment.

• Regulatory Compliance: Understanding of industry compliance requirements and standards, specifically in banking IT landscapes.

• Certifications: At least 2 of the following: Associate Developer for Apache Spark, Data Engineer Associate, Professional Data Engineer, Machine Learning Associate, Professional ML Engineer.

Education:

• Degree: 4-year Bachelor's degree in Computer Science or a related field.

Benefits:

• Health Insurance: Employees and their eligible family members including spouses, domestic partners, and children are eligible for coverage from the first day of employment.

• Paid Time Off: Start your career at SunnyData with a minimum of 20 days Paid Time Off annually, plus nine paid company holidays.

• An opportunity to grow your technical and people skills, lead teams on complex customer projects that are highly innovative and cutting-edge. Great opportunity to grow in your career with the right level of focus, innovation, and customer-centricity with a high-growth consulting company dedicated to Databricks.