We use cookies. Find out more about it here. By continuing to browse this site you are agreeing to our use of cookies.
#alert
Back to search results
New

Senior Data Engineer - ETL/Scala

Spectraforce Technologies
United States, Virginia, Richmond
Nov 19, 2026
Job Title: Senior Data Engineer - ETL/Scala

Duration: 6 months

Location: Richmond, VA 23238 (Hybrid)

Experience: 8+ years in Data Engineering

Number of positions: 2


Role Summary

We're looking for an experienced data engineer to join our team building and maintaining large-scale ETL pipelines on AWS EMR, primarily in Scala. This person will operate independently on complex, ambiguous problems, contribute to engineering standards across the team, and be equally comfortable in Python and SQL when the job calls for it. Familiarity with Databricks is a strong plus as we continue to evolve our data platform.

What You'll Do

- Design, build, and optimize ETL pipelines on AWS EMR using Scala/Spark, processing data at scale with attention to performance, cost, and reliability

- Write and maintain Python scripts and utilities to support data workflows, automation, and tooling where Scala isn't the right tool for the job

- Develop and troubleshoot complex SQL for data validation, transformation logic, and downstream analytics use cases

- Work with Databricks notebooks/jobs as our data platform usage grows, contributing to migration and modernization efforts where relevant

- Operate with autonomy on tasks with limited detail - scope ambiguous problems, propose solutions, and drive them to completion without heavy oversight

- Identify and implement opportunities for automation, reuse, and simplification across existing pipelines, reducing team toil

- Conduct peer code reviews, flag areas for improvement, and help maintain team coding/design standards and documentation

- Participate in production support - troubleshoot pipeline failures, perform root-cause analysis, and implement fixes that avoid technical debt

- Communicate progress, risks, and blockers proactively to the team and stakeholders

Required Qualifications

- 8+ years of professional experience in data engineering or related software engineering roles

- Strong hands-on experience with Scala and Apache Spark, specifically on AWS EMR

- Working proficiency in Python for scripting, tooling, or pipeline components

- Strong SQL skills - complex queries, performance tuning, and data validation

- Demonstrated experience owning ETL pipelines end-to-end: design, build, test, deploy, monitor

- Ability to work independently in ambiguous situations, exploring multiple approaches before settling on a solution

- Experience with version control (Git), CI/CD practices, and production-grade engineering hygiene

Preferred Qualifications

- Experience with Databricks (notebooks, jobs, Delta Lake, cluster management)

- Experience with workflow orchestration tools (Airflow, Step Functions, or similar)

- Familiarity with AWS data ecosystem beyond EMR (S3, Glue, Redshift, Athena)

- Experience mentoring other engineers or leading technical design discussions

- Background in financial services or another regulated industry

Core Competencies (aligned to Principal Associate expectations)

- Technically Skilled & Versatile: Builds breadth across the stack (Scala/EMR as primary, Python/SQL/Databricks as needed); quickly ramps on new tools

- Independent / Manages Ambiguity: Operates with minimal guidance, scopes ambiguous problems, and drives them to resolution

- Automation Focused: Actively looks for ways to simplify, automate, and reduce repeated manual work

- Results Focus: Delivers quality work on committed timelines, owns outcomes beyond initial delivery

- Problem Solving: Explores multiple design options (e.g., build vs. reuse) before committing to a solution; avoids accruing technical debt
Applied = 0

(web-7dc44f465b-pmrrn)