Data Engineer (Python / AWS / Data Pipelines) (1372) at Softgic

Job Description

This is a remote position.

We are looking for a skilled Data Engineer to design, build, and maintain scalable data systems supporting analytics and machine learning initiatives. This role works in hybrid environments—both SaaS-hosted and client-managed (on-premises)—and focuses on enhancing the Elastic Hierarchy framework, unifying diverse financial datasets across multiple systems.

Key Responsibilities:

Design and maintain scalable ETL/ELT pipelines in Python, integrating structured and semi-structured data (JSON, CSV, XML) across Snowflake, MongoDB, Postgres, and AWS services (S3, Glue, Lambda, EC2, EMR, Redshift, RDS).
Build and optimize transformation and orchestration workflows using DBT, Airflow, Prefect, or Dagster.
Implement data governance, quality checks, and security best practices throughout data pipelines.
Extend and optimize the Elastic Hierarchy framework to harmonize financial data from various systems.
Collaborate with analysts, ML engineers, and product teams to deliver business-ready datasets and data solutions.

Requisitos

Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field.
Strong Python programming skills with experience in data processing and automation.
Proven experience with the AWS data ecosystem, including S3, Glue, Lambda, EMR, EC2, Redshift, and RDS.
Hands-on experience with Snowflake, MongoDB, and Postgres databases.
Proficiency with DBT or similar data transformation tools, and with orchestration frameworks such as Airflow, Prefect, or Dagster.
Knowledge of data mapping, attribution, or reconciliation (experience in financial services is a strong plus).
Understanding of hybrid/on-premise deployment models within enterprise environments.
Excellent English communication skills and ability to collaborate effectively within distributed teams.

Apply