Senior Data Engineer at Black Duck Software

📋 Description

• Design and build data pipelines and datasets within a governed data platform.
• Implement ingestion patterns for batch, streaming, and event data.
• Build reliable data pipelines with data quality checks, lineage, monitoring.
• Develop curated datasets and views for analytics and reporting.
• Develop data models for operational and analytics stores per conventions.
• Support ML/analytics by producing trusted training datasets and features.
• Collaborate with PMs, engineers, analysts, and security to deliver data solutions.
• Participate in on-call rotations and operational support for data services.
• Contribute to technical discussions, code reviews, and data engineering practices.

🎯 Requirements

• Solid experience building and operating production data pipelines in cloud environments.
• Strong SQL skills and Python experience for pipelines, transformations, and automation.
• Hands-on with cloud platforms (AWS or Google Cloud) incl. core compute, storage, IAM.
• Experience with operational databases (RDS-style) and analytics/columnar stores.
• Ability to design data models for operational and analytics stores with standard conventions.
• Build curated datasets with data quality checks, lineage, monitoring.
• Collaborate with product managers, engineers, analysts, and security to meet roadmap needs.
• Participate in on-call rotations and operational support for data services.
• Contribute to technical discussions and continuous improvement of data engineering practices.