We’re Hiring: Senior Databricks Solution Architect
22 hours ago
Norwich
We are seeking a Senior Databricks Architect to design, implement, and optimize enterprise-scale data platforms using theDatabricks Lakehouse Platform. The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. • Design and architectDatabricks Lakehouse solutionsfor large-scaleenterprise data platforms., • Lead end-to-end Databricks implementation, including architecture, development, deployment, and optimization., • Define Databricks workspace architecture, clusterstrategies, security models, and operational standards., • Design and developscalable data pipelinesusingDatabricks, Apache Spark, PySpark, SQL, andDelta Lake., • Implement data ingestionframeworks supporting batchand real-timedata processing., • Design Delta Lake architectures including:, • Delta tables, • Medallion architecture (Bronze/Silver/Gold layers), • Data quality frameworks, • Schema evolution andoptimization, • Perform Databricks performancetuning:, • Cluster optimization, • Spark job optimization, • Query tuning, • Cost optimization, • Implement Databricks governanceand security using:, • Unity Catalog, • Access controls, • Data lineage, • Audit capabilities, • Lead migration projectsfrom legacy dataplatforms to DatabricksLakehouse., • Define best practicesfor Databricks CI/CD, DevOps, andenvironment management., • Collaborate with dataengineers, data scientists, analysts, and enterprisearchitects., • Provide technical guidance, architecture documentation, and design reviews., • Strong experience in dataengineering, big data, or solution architecture., • Strong hands-onexperience withDatabricks Platform., • Expert knowledge of:, • Apache Spark, • PySpark / Scala, • Delta Lake, • SQL, • Data Lakehouse architecture, • ETL/ELT development, • Experience designing anddelivering enterprise Databrickssolutions., • Strong experience withcloud platforms:, • Azure Databricks (preferred), • AWS Databricks, • Google Cloud Databricks, • Experience with:, • Databricks Workflows, • Jobs orchestration, • Notebooks, • Repos, • Cluster management, • Unity Catalog, • Databricks Certified DataEngineer Professional /Databricks Certified Architect., • Experience with:, • MLflow, • Databricks Machine Learning, • Feature Store, • Model deployment, • MLOps, • Knowledge of streamingtechnologies:, • Structured Streaming, • Kafka, • Experience with datagovernance, compliance,and enterprise securityframeworks., • Strong Databricks architectureand implementation expertise., • Ability to designscalableand cost-efficientLakehouse solutions., • Strong troubleshooting andperformance optimizationskills., • Ability to leadtechnical discussions withenterprise stakeholders. #J-18808-Ljbffr