Senior Data Engineer

Senior Data Engineer

  • Long term contract
  • India

Pyspark, Snowflake, Scala, Hives, Kafka, Python, AWS

  • Leveraging your deep knowledge to provide technical leadership to take projects from zero to completion.
  • Architect, build and maintain scalable data pipelines and access patterns related to permissions and security.
  • Research, evaluate and utilize new technologies/tools/frameworks centered around high-volume data processing
  • Drive a culture of collaborative reviews of designs, code and test plans
  • Work with the architecture engineering team to ensure quality solutions are implemented and engineering best practices adhered to
  • Develop set process for Data mining, Data modeling, and Data production
  • A data engineer with strong programming experience in Python
  • Extensive experience related to processing frameworks such as Spark, Spark Streaming, Airflow, Hive, Sqoop, Kafka etc.
  • Experience with big data processing within cloud environments such AWS S3.
  • Passion for Data Privacy and Security, Data Management.
  • Experience building and shipping data production pipelines sourcing data from a diverse array of sources.
  • Deep understanding of measuring and ensuring data quality at scale and the required tooling to monitor and optimize the performance of our data pipelines
  • Ability to influence and communicate effectively with team members and business stakeholders
    Nice to have:
  • Knowledge of the retail and eCommerce sector and its use cases.
  • Scala
  •  Oracle/MySQL
  • Airflow
  • Spark
  • AWS Cloud (EMR, S3, SNS,SQS)
  • 3 Data Lake
  • Kafka

Resumes to be sent  torecruitment@kloud9.nyc

Apply Online

Position : Senior Data Engineer