Data Engineer (Full-time, Remote) at Kata.ai

Design, build, and maintain scalable data pipelines, streaming infrastructure, and AI/ML data workflows that power data-driven products and enterprise AI solutions — ensuring reliable, timely, and high-quality data is available across the organization — so that AI Engineers, Product teams, and enterprise clients can make accurate, insight-driven decisions and deliver intelligent customer experiences through Kata's AI and voice platforms.

Qualifications & Education :

• Bachelor's degree in Computer Science, Information Systems, Data Engineering, Statistics, or related field
• Relevant certifications (GCP Professional Data Engineer, Databricks, Airflow/Astronomer, etc.) are a plus

Technical Skills :

• Streaming: Apache Kafka — topic design, consumer groups, partitioning strategy, and real-time event processing
• Batch Orchestration: Apache Airflow — DAG design, scheduling, dependency management, and failure handling
• Distributed Processing: Apache Spark — batch and micro-batch transformations, DataFrame API, optimization
• Data Warehousing: Google BigQuery (primary); Apache Hive for large-scale batch analytics
• NoSQL / Wide-Column: Apache Cassandra — data modeling for high-write, time-series, and event-driven workloads
• Languages: Python (required); SQL (required); Scala is a plus
• Cloud: GCP — BigQuery, Dataflow, Cloud Storage, Pub/Sub, Vertex AI Pipelines; Azure is a plus
• Containerization: Docker; basic Kubernetes for deploying data services
• CI/CD: GitLab CI, GitHub Actions, or equivalent for pipeline deployment automation
• Data Quality: Great Expectations, dbt tests, or custom validation frameworks
• Monitoring: Prometheus, Grafana, or GCP Monitoring for pipeline observability; alerting on SLA breaches
• Version Control: Git with feature branching and pull request workflow

Experience : 

Associate Level (1–2 years)

• 1–2 years of professional experience in data engineering, software engineering with data focus, or a related technical role
• Hands-on experience building or maintaining data pipelines in a production environment
• Practical exposure to at least one streaming or batch processing technology (Kafka, Spark, or Airflow)
• Familiarity with SQL and relational or columnar databases (BigQuery, PostgreSQL, Hive, or equivalent)
• Exposure to cloud data services on GCP or Azure
• Experience working in Agile/Scrum teams with sprint-based delivery

Mid Level (3–5 years)

• 3–5 years of professional experience in data engineering, with at least 2 years building and operating production-grade pipelines
• Proven hands-on experience with Apache Kafka for real-time event streaming — including topic design, consumer group management, and at-least-once/exactly-once delivery patterns
• Demonstrated experience designing and maintaining batch workflows using Apache Airflow and large-scale data transformations with Apache Spark
• Experience working with BigQuery and/or Hive for large-scale analytics workloads, including query optimization and partitioning strategies
• Hands-on experience with Cassandra or similar NoSQL wide-column stores for high-write or time-series data use cases
• Experience supporting AI/ML data pipelines — feature engineering, training dataset preparation, or model inference data feeds
• Experience with data quality frameworks and implementing data observability practices in production environments

We value a flexible working hour for our employees.

The most important is we provide a learning experience in Conversational AI Industry.