Hadoop Development Lead

Job Title: Hadoop Development Lead

Primary Skill: PySpark, Hadoop, Hive, SQL, must have worked as team lead for minimum 2-3 years.

Location: Hyderabad/Pune/Coimbatore

Mode of Work: Hybrid

Experience: 7-10 years

About the Job :

We are seeking an experienced Hadoop Data Engineer to join our team- a team of high-energy individuals who thrive in a rapid-pace and agile product development environment. The ideal candidate will have extensive experience in Hadoop development, strong knowledge of PySpark programming, and expertise in Hadoop, Hive framework. You will be responsible for writing complex SQL queries.

Know your team:

At ValueMomentum’s Engineering Center, we are a team of passionate engineers who thrive on tackling complex business challenges with innovative solutions while transforming the P&C insurance value chain. We achieve this through strong engineering foundation and continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Our core expertise lies in six key areas: Cloud Engineering, Application Engineering, Data Engineering, Core Engineering, Quality Engineering, and Domain expertise.

Join a team that invests in your growth. Our Infinity Program empowers you to build your career with role-specific skill development leveraging immersive learning platforms. You’ll have the opportunity to showcase your talents by contributing to impactful projects.


  • Batch Processing Optimization: Analyse and optimize existing batch workflows to ensure efficient data processing.
  • Development & Scripting: Write and maintain batch scripts using Python, Hive, and Shell to automate data workflows.
  • Data Analysis: Utilize Hadoop and related technologies to gather, analyse, and report on data performance and processing metrics.
  • Troubleshooting: Identify and resolve data processing bottlenecks and workflow issues to minimize downtime and ensure smooth operations.
  • Collaboration: Work closely with data engineering, data science, and business intelligence teams to understand requirements and improve the data pipeline.
  • Documentation: Develop and maintain technical documentation for workflows, troubleshooting guides, and best practices.
  • Continuous Improvement: Propose and implement improvements for existing batch processes and pipelines.


  • 5-7 years of experience in batch processing optimization and data workflows, with a strong focus on Hadoop, Python, Hive, and Shell scripting.
  • Proficient in Hadoop ecosystem tools (HDFS, YARN, etc.).
  • Strong knowledge of Python for data processing and automation.
  • Advanced proficiency in Hive query language.
  • Expertise in writing Shell scripts for Linux/Unix environments.
  • Strong troubleshooting and performance analysis skills to optimize workflows.
  • Excellent communication and collaboration skills with a problem-solving mindset.
  • Familiarity with Agile methodologies.
  • Experience working in cross-functional teams.

About ValueMomentum:

ValueMomentum is a leading solutions provider for the global property & casualty insurance industry, supported by deep domain and technology capabilities. We offer a comprehensive suite of advisory, development, implementation, and maintenance services across the entire P&C insurance value chain. This includes Underwriting, Claims, Distribution, and more, empowering insurers to stay ahead with sustained growth, high performance, and enhanced stakeholder value.  Trusted by over 75 insurers, ValueMomentum is one of the largest standalone insurance-focused solutions providers to the US insurance industry.

Our culture – Our fuel

At ValueMomentum, we believe in making employees win by nurturing them from within, collaborating and looking out for each other.

  • People first – Empower employees to succeed.
  • Nurture leaders – Nurture from within.
  • Enjoy wins – Recognize and celebrate wins.
  • Collaboration – Foster a culture of collaboration and people-centricity.
  • Diversity – Committed to diversity, equity, and inclusion.
  • Fun – Create a fun and engaging work environment.
  • Warm welcome – Provide a personalized onboarding experience.

Company Benefits:

  • Compensation – Competitive compensation package comparable to the best in the industry.
  • Career advancement – Career development, coaching and mentoring programs for professional and leadership skill development. Comprehensive training and certification programs.
  • Benefits: Comprehensive health benefits, wellness, and fitness programs.

    Apply for this Job