Job Title:
Azure Databricks Engineer (Technical Specialist - Junior)

Location:
Hybrid - Triangle Region, North Carolina
(NCDOT Century Center, Building B, 1020 Birch Ridge Drive, Raleigh, NC 27610)

Work Arrangement:
Hybrid (onsite 1-2 days per month for meetings)
Note: The candidate must come onsite on the first day to collect equipment.

Employment Type:
Contract (10/26/2024 - 10/25/2025)

Agency Interview Type:
Webcam Interview Only

Job Description:

NCDOT's Transportation Web Systems Team is seeking a skilled Azure Databricks Engineer to design and implement ETL pipelines and product solutions using Azure Databricks. The ideal candidate will collaborate with team members and business stakeholders to develop efficient, resilient data solutions that ensure data quality and support organizational goals. You will work closely with the development team and technical lead to design, plan, and execute data engineering processes.

Key Responsibilities:

Design and implement repeatable, scalable ETL workflows using Databricks notebooks and Delta Live Tables for both batch and stream processing.
Collaborate with business stakeholders to develop data products aligned with business needs.
Work with DBAs to ingest data from both cloud and on-prem transactional databases.
Ensure adherence to data architecture standards, focusing on sensitive data security and streamlined development of data products.
Document data engineering processes and ensure data quality through thorough testing and validation.
Share insights and best practices with both stakeholders and fellow engineers across the organization.

Required Skills and Experience:

Excellent interpersonal, written, and communication skills
Required: 5 years
Databricks notebook development (clean, easy-to-follow code)
Required: 2 years
Deep knowledge of data engineering best practices, data warehouses, data lakes, and Delta Lake architecture
Required: 2 years
Spark and Databricks SQL/PySpark
Required: 2 years
Technical experience with Azure Databricks and cloud providers (AWS, Google Cloud, Azure)
Required: 2 years
In-depth knowledge of OLTP and OLAP systems, Apache Spark, and streaming products like Azure Service Bus
Required: 2 years
Practical experience with Databricks Delta Live Tables
Required: 2 years
Object-oriented programming languages (C#, Java, Python)
Desired: 7 years

Why Apply:

Competitive pay rate
Hybrid work arrangement with only 1-2 days per month onsite for meetings.
Opportunity to contribute to innovative projects in data engineering and architecture within NCDOT.

Application Deadline:
Submissions must be received by 10/28/2024.

Apply Now!

Job Type: Contract

Pay: $65.02 - $70.00 per hour

Expected hours: 40 per week

Benefits:

401(k)
Dental insurance
Health insurance

Compensation Package:

Hourly pay

Schedule:

8 hour shift

People with a criminal record are encouraged to apply

Experience:

Object-oriented programming languages: 7 years (Preferred)
Databricks Delta Live Tables: 2 years (Required)
OLTP and OLAP Systems: 2 years (Required)
Spark: 2 years (Required)
Azure Service Bus: 2 years (Preferred)
Azure Databricks: 2 years (Preferred)
Cloud Providers: 2 years (Preferred)
Adobe Spark: 2 years (Preferred)
PySpark: 2 years (Preferred)
Databricks notebook development: 2 years (Preferred)
DataBricks: 2 years (Preferred)

Ability to Commute:

Raleigh, NC 27610 (Required)

Ability to Relocate:

Raleigh, NC 27610: Relocate before starting work (Required)

Work Location: In person

Omega Solutions Inc

Latest Jobs

Data Engineer