Job Title:
Azure Databricks Engineer (Technical Specialist - Junior)
Location:
Hybrid - Triangle Region, North Carolina
(NCDOT Century Center, Building B, 1020 Birch Ridge Drive, Raleigh, NC 27610)
Work Arrangement:
Hybrid (onsite 1-2 days per month for meetings)
Note: The candidate must come onsite on the first day to collect equipment.
Employment Type:
Contract (10/26/2024 - 10/25/2025)
Agency Interview Type:
Webcam Interview Only
Job Description:
NCDOT's Transportation Web Systems Team is seeking a skilled Azure Databricks Engineer to design and implement ETL pipelines and product solutions using Azure Databricks. The ideal candidate will collaborate with team members and business stakeholders to develop efficient, resilient data solutions that ensure data quality and support organizational goals. You will work closely with the development team and technical lead to design, plan, and execute data engineering processes.
Key Responsibilities:
- Design and implement repeatable, scalable ETL workflows using Databricks notebooks and Delta Live Tables for both batch and stream processing.
- Collaborate with business stakeholders to develop data products aligned with business needs.
- Work with DBAs to ingest data from both cloud and on-prem transactional databases.
- Ensure adherence to data architecture standards, focusing on sensitive data security and streamlined development of data products.
- Document data engineering processes and ensure data quality through thorough testing and validation.
- Share insights and best practices with both stakeholders and fellow engineers across the organization.
Required Skills and Experience:
- Excellent interpersonal, written, and communication skills
- Required: 5 years
- Databricks notebook development (clean, easy-to-follow code)
- Required: 2 years
- Deep knowledge of data engineering best practices, data warehouses, data lakes, and Delta Lake architecture
- Required: 2 years
- Spark and Databricks SQL/PySpark
- Required: 2 years
- Technical experience with Azure Databricks and cloud providers (AWS, Google Cloud, Azure)
- Required: 2 years
- In-depth knowledge of OLTP and OLAP systems, Apache Spark, and streaming products like Azure Service Bus
- Required: 2 years
- Practical experience with Databricks Delta Live Tables
- Required: 2 years
- Object-oriented programming languages (C#, Java, Python)
- Desired: 7 years
Why Apply:
- Competitive pay rate
- Hybrid work arrangement with only 1-2 days per month onsite for meetings.
- Opportunity to contribute to innovative projects in data engineering and architecture within NCDOT.
Application Deadline:
Submissions must be received by 10/28/2024.
Apply Now!
Job Type: Contract
Pay: $65.02 - $70.00 per hour
Expected hours: 40 per week
Benefits:
- 401(k)
- Dental insurance
- Health insurance
Compensation Package:
Schedule:
People with a criminal record are encouraged to apply
Experience:
- Object-oriented programming languages: 7 years (Preferred)
- Databricks Delta Live Tables: 2 years (Required)
- OLTP and OLAP Systems: 2 years (Required)
- Spark: 2 years (Required)
- Azure Service Bus: 2 years (Preferred)
- Azure Databricks: 2 years (Preferred)
- Cloud Providers: 2 years (Preferred)
- Adobe Spark: 2 years (Preferred)
- PySpark: 2 years (Preferred)
- Databricks notebook development: 2 years (Preferred)
- DataBricks: 2 years (Preferred)
Ability to Commute:
- Raleigh, NC 27610 (Required)
Ability to Relocate:
- Raleigh, NC 27610: Relocate before starting work (Required)
Work Location: In person