Senior Data Engineer

Senior Data Engineer

  • Permanent
  • India

AWS, Spark Scala, Hive, Glue, Stream Processing, Airflow, EMR

  • Design and Build Data pipelines in collaboration with Product Owners / BSA, Data Analysts and Business stake holders
  • Work with Lead Data Engineers and Junior Data Engineers to ensure quality solutions are implemented with adherence to engineering best practices
  • Build Reusable utilities, frameworks and user defined functions
  • Evaluate and utilize new technologies/tools/frameworks centered around Orchestration Pipelines
  • Provide technical leadership on Orchestration Pipelines to the project teams
  • Overall 5+ Years of Experience in Data Modelling and ETL processing projects
  • 3+ Years of Experience in designing and implementing large data pipelines using Spark, Hive on AWS with Python and pySpark
  • 1+ Years of Experience in building end to end pipelines using Airflow DAG
  • Hands on Experience in Data Ingestion, Data Cleansing, Data Transformation tools and techniques
  • Hands on Experience with Data Quality and Data Lineage tools and frameworks
  • Hands on Experience in Spark Batch and Streaming Data pipelines with Kafka
  • Experience of working in Agile model
  • Expert level knowledge on Coding and Design Best Practices
  • Expert level experience in SQL
  • Expert level Experience in Trouble shooting
  • Experience supporting and working with cross-functional teams in a dynamic environment
  • Excellent verbal and written communication skills in English
  • Ability to communicate effectively with team members and business stakeholders

Nice to have:

  • General knowledge/experience of AWS snowflake is an added advantage

Kloud9 provides a robust compensation package and a forward-looking opportunity for growth in emerging fields.

Kloud9 is an equal opportunity employer and will not discriminate against any employee or applicant on the basis of age, color, disability, gender, national origin, race, religion, sexual orientation, veteran status, or any classification protected by federal, state, or local law.

Resumes to be sent  torecruitment@kloud9.nyc

Apply Online

Position : Senior Data Engineer