New

ETL, Ab Inito, Teradata, Spark and DevOps

Genesis10
life insurance, 401(k)
United States, Texas, Addison
May 06, 2025
Genesis10 is currently seeking a Ab Initio Developer with our client in the financial Industry located in Addison, TX or Charlotte, NC. This is a 12+ month contract position. Senior ETL Data/DevOps Engineer - Ab Initio \| Teradata \| PySpark and DevOps Length - 12 months We are looking for an innovative Senior ETL Data Engineer with strong experience in Ab Initio, Teradata, and PySpark to help modernize and optimize our data integration platforms. This role combines traditional ETL expertise with hands-on cloud-based, distributed computing using Spark frameworks. You will architect hybrid ETL solutions, bridging enterprise batch systems with real-time big data platforms, enabling next-generation analytics and AI-driven business insights. This role isn't just a traditional ETL developer - it's a transformation agent moving legacy systems toward modern, scalable, cloud-based ETL solutionswhile still leveraging existing enterprise strengths. Dev Ops is the focus to automate software development, deployment and infrastructure management. Responsibilities: Design, develop, and support scalable ETL pipelines using Ab Initio and Teradata for critical enterprise data systems. Develop PySpark applications for modern ETL workflows targeting distributed platforms (Hadoop, Databricks, EMR, GCP Dataproc, etc.). Architect data migration and transformation pipelines moving from traditional EDW (Teradata) to cloud-native or data lake environments. Implement hybrid workflows integrating batch ETL (Ab Initio) with distributed Spark-based real-time processing. Optimize Teradata SQL queries and Spark jobs to ensure low-latency, high-throughput data pipelines. Lead ETL modernization initiatives, identifying where Ab Initio can be refactored to Spark or cloud-based services. Collaborate with Data Architects, Cloud Engineers, and Analytics teams to design unified data ingestion frameworks. Develop and enforce best practices for ETL coding standards, error handling, data quality, and metadata management. Support orchestration using traditional schedulers (Autosys, Control-M) and modern orchestration tools (Airflow, Argo, Cloud Composer). Participate in Agile ceremonies, proactively suggesting ways to improve the team's velocity and technical stack. Required Qualifications: 5+ years of ETL development experience with Ab Initio. 5+ years of experience working with Teradata (deep SQL expertise, performance tuning). 2+ years of hands-on experience building scalable ETL pipelines using PySpark. Strong experience with distributed computing frameworks (e.g., Hadoop, Databricks, AWS EMR, GCP Dataproc). Deep understanding of data warehouse architectures (star schema, snowflake, normalization/denormalization strategies). Proficiency with Unix/Linux scripting for ETL automation. Solid knowledge of ETL performance tuning, especially around large volume data ingestion and transformation. Familiarity with cloud platforms (AWS, Azure, or GCP) and cloud-native data services. Strong problem-solving, analytical, and debugging skills. Excellent verbal and written communication skills; able to bridge between legacy teams and modern cloud/data engineering teams. Experience modernizing ETL pipelines from Ab Initio to Spark or cloud-native ETL frameworks. Knowledge of orchestration and workflow tools (Apache Airflow, Argo Workflows, GCP Cloud Composer). Understanding of containerization (Docker, Kubernetes) for Spark applications is a plus. Hands-on experience with streaming data processing (Kafka, Spark Structured Streaming) is a plus. Pay Range: $ 44 - 54 Only candidates available and ready to work directly as Genesis10 employees will be considered for this position. If you have the described qualifications and are interested in this exciting opportunity, please apply! About Genesis10: Ranked a Top Staffing Firm in the U.S. by Staffing Industry Analysts for six consecutive years, Genesis10 puts thousands of consultants and employees to work across the United States every year in contract, contract-for-hire, and permanent placement roles. With more than 300 active clients, Genesis10 provides access to many of the Fortune 100 firms and a variety of mid-market organizations across the full spectrum of industry verticals. For contract roles, Genesis10 offers the benefits listed below. If this is a perm-placement opportunity, our recruiter can talk you through the unique benefits offered for that particular client. Benefits of Working with Genesis10: Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years. The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years. Access to an experienced, caring recruiting team (more than 7 years of experience, on average.) Behavioral Health Platform Medical, Dental, Vision Health Savings Account Voluntary Hospital Indemnity (Critical Illness & Accident) Voluntary Term Life Insurance 401K Sick Pay (for applicable states/municipalities) Commuter Benefits (Dallas, NYC, SF) Remote opportunities available For multiple years running, Genesis10 has been recognized as a Top Staffing Firm in the U.S., as a Best Company for Work-Life Balance, as a Best Company for Career Growth, for Diversity, and for Leadership, amongst others. To learn more and to view all our available career opportunities, please visit us at our website. Genesis10 is an Equal Opportunity Employer. Candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran. #DIG10-ST