View All Jobs

Staff Data Engineer

R&D - Engineering / Product - Data Platform
Bangalore
Job Description
Key Responsibilities:
  1. Data Pipeline Development: Design, develop, and maintain robust data pipelines to ingest, process, and analyze large volumes of data.
  2. Data Modeling and Architecture: Collaborate with cross-functional teams to design scalable and efficient data models and architecture that meet the requirements of our evolving product ecosystem and align with our business goals.
  3. Data Integration: Integrate data from diverse sources, including APIs and third-party services, to create comprehensive data sets for analysis and visualization.
  4. Data Quality Assurance: Implement processes and tools to ensure the quality, accuracy, and reliability of data, including data validation, cleansing, and monitoring, while driving the adoption of best practices in data modeling, data quality, data governance, and data security across all data platforms.
  5. Performance Optimization: Optimize data processing and querying performance to ensure timely and efficient access to insights for internal stakeholders and customers.
  6. Monitoring & Troubleshooting: Oversee data pipeline execution, troubleshoot issues, and serve as the leading point for data engineering incidents. This role demands vigilance and problem-solving skills to ensure data systems' reliability and efficiency.
  7. Data Analysis and Insights: Work closely with other technical teams such as software engineering and product management to translate business requirements into data-driven solutions, providing actionable insights and recommendations.
  8. Technical Leadership: Serve as a technical lead and mentor junior team members, fostering a culture of collaboration, innovation, and continuous learning.
  9. Evaluation of Emerging Technologies: Evaluating and recommending new and emerging data technologies and tools to enhance the capabilities of the data platform, ensuring it stays current with industry advancements and best practices.
  10. Documentation and Communication: Document technical designs, implementation details, and best practices, and effectively communicate complex concepts and solutions to both technical and non-technical stakeholders.
Qualifications:
  1. Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field. Advanced certifications in data engineering, cloud-based data platforms (eg. Databricks, Snowflake) or cloud platforms (e.g., AWS, Azure, GCP) are a plus.
  2. Experience: Minimum of 6 years of experience in data engineering, with a proven track record of designing and implementing data pipelines and systems at scale.
  3. Technical Skills:
    • Proficiency in programming languages such as Python, Java, or Scala.
    • Strong SQL skills and experience with relational and NoSQL databases (e.g., PostgreSQL, MongoDB).
    • Hands-on experience with big data technologies such as Hadoop, Spark, Kafka, and distributed computing frameworks.
    • Familiarity with cloud platforms and services for data storage, processing, and analytics (e.g., AWS S3, Redshift, EMR, Glue).
    • Expertise in Data Engineering Platforms and Tools: Demonstrated experience with advanced data engineering platforms such as Databricks and Snowflake, alongside proficiency in ELT and data modeling tools like Fivetran, dbt, and Debezium etc.
    • Knowledge of Druid or other similar technologies for real-time analytics and data exploration.
    • Proficiency in Infrastructure as Code (IaC) tools such as Terraform and AWS CloudFormation for automating and managing infrastructure deployments.
    • Familiarity with AI and machine learning algorithms is a plus.
  4. Problem-Solving Ability: Ability to analyze complex problems, propose innovative solutions, and drive projects to completion in a fast-paced startup environment.
  5. Communication Skills: Excellent verbal and written communication skills, with the ability to collaborate effectively with cross-functional teams and present technical concepts to diverse audiences.
  6. Adaptability: Willingness to learn new technologies and methodologies, and adapt to evolving business requirements and industry trends.
  7. Team Player: Strong interpersonal skills and a collaborative mindset, with a passion for mentoring and supporting the growth of junior team members.
Job Requirement
Python, Java, or Scala
Hadoop, Spark, Kafka
SQL, Nosql
AWS S3, Redshift, EMR, Glu