Senior Data Engineer (Fully In-Office)
PharmaLink, LLC
Summary
The Senior Data Engineer – Databricks is responsible for building and evolving PharmaLink's enterprise data, analytics, and AI platform. This hands-on role spans the modern Databricks ecosystem, including data engineering, lakehouse architecture, analytics and dashboards, AI, Databricks Apps, and Lakebase. The role requires strong engineering fundamentals, independent problem-solving, technical ownership, and the ability to design, build, and operate reliable, scalable production solutions.
Job description
Job Summary
The Senior Data Engineer – Databricks is responsible for building and evolving PharmaLink's enterprise data, analytics, and AI platform. This hands-on role spans the modern Databricks ecosystem, including data engineering, lakehouse architecture, analytics and dashboards, AI, Databricks Apps, and Lakebase. The role requires strong engineering fundamentals, independent problem-solving, technical ownership, and the ability to design, build, and operate reliable, scalable production solutions.
A hands-on builder approach is essential, with a focus on developing solutions rather than simply administering platforms. The ideal candidate can work across data engineering, platform architecture, analytics, AI, and data applications; independently investigate complex problems; and take solutions from initial design through production.
Key Responsibilities
- Design, build, and operate enterprise data solutions on Databricks.
- Develop scalable ETL/ELT pipelines using Python, PySpark, Spark SQL, and Delta Lake.
- Design curated data models and reusable data products for analytics, applications, and AI.
- Implement Unity Catalog, security, lineage, access controls, and data-governance capabilities.
- Build and optimize Databricks SQL Warehouses and AI/BI Dashboards.
- Develop data and AI applications using Databricks Apps.
- Design and implement appropriate solutions using Databricks Lakebase.
- Enable AI/ML and Generative AI use cases using Databricks capabilities.
- Build automated data-quality, reconciliation, anomaly-detection, and monitoring processes.
- Integrate data from databases, SaaS platforms, APIs, files, and other enterprise systems.
- Optimize Spark workloads, SQL queries, pipelines, performance, and platform cost.
- Establish engineering standards for Git, CI/CD, testing, monitoring, and production deployment.
- Troubleshoot complex production and data issues and perform root-cause analysis.
- Partner with Data Governance, BI, application, and business teams to deliver trusted data products.
Qualifications
- 8+ years of experience in data engineering, data platforms, or related engineering roles.
- 4+ years of hands-on Databricks experience in production environments.
- Advanced SQL, Python, PySpark, and Apache Spark skills.
- Strong experience with Delta Lake and Unity Catalog.
- Strong ETL/ELT, data modeling, and enterprise data architecture experience.
- Experience with Databricks SQL and AI/BI Dashboards.
- Familiarity with Databricks capabilities across AI/ML, Generative AI, Databricks Apps, and Lakebase.
- Strong relational database and data-engineering fundamentals.
- Experience with Git, CI/CD, automated testing, monitoring, and production support.
- Experience working with cloud-based enterprise data platforms.
- Must maintain a positive attitude in a high-performance team environment.
- Must pass background check and random drug screenings.
- Must be adaptable and flexible in dealing with a diverse work force
Preferred Qualifications
- Databricks on Microsoft Azure.
- Azure Data Factory.
- Microsoft SQL Server.
- Fivetran or similar CDC/ELT technologies.
- Power BI.
- Databricks Workflows.
- MLflow and AI/model lifecycle capabilities.
- API and application development experience.
- Azure DevOps or GitHub-based CI/CD.
- Databricks professional certification.
- Experience in healthcare, life sciences, or another regulated industry.
Requirements
- 8+ years of experience in data engineering, data platforms, or related engineering roles
- 4+ years of hands-on Databricks experience in production environments
- Advanced SQL, Python, PySpark, and Apache Spark skills
- Strong experience with Delta Lake and Unity Catalog
- Strong ETL/ELT, data modeling, and enterprise data architecture experience
- Experience with Databricks SQL and AI/BI Dashboards
- Familiarity with Databricks capabilities across AI/ML, Generative AI, Databricks Apps, and Lakebase
- Strong relational database and data-engineering fundamentals
- Experience with Git, CI/CD, automated testing, monitoring, and production support
- Experience working with cloud-based enterprise data platforms
- Must maintain a positive attitude in a high-performance team environment
- Must pass background check and random drug screenings
- Must be adaptable and flexible in dealing with a diverse work force
Company overview
PharmaLink, Inc. is the nation's premier Pharmaceutical Reverse Distribution provider, specializing in comprehensive reverse logistics and disposal solutions for the pharmaceutical supply chain. Founded in 2000 and headquartered in Largo, Florida, the company offers a complete spectrum of services including product inspections, destruction, handling, disposal, reporting, and assessment solutions. PharmaLink provides a 360-degree view of pharmaceutical return processing needs for all businesses involved in healthcare delivery, delivering impactful solutions that reduce cycle times while safely removing nonsalable products from the marketplace. With a powerful enterprise infrastructure, the company produces accurate processing and analytics that optimize business decisions while adhering to stringent state and federal regulatory guidelines. PharmaLink serves as the choice provider for pharmaceutical return and disposal solutions, offering customizable services for clients of all sizes across the United States.