Senior Software Engineer-JAVA Bigdata
- Experience: 3 - 5 Years
- Location: Indore
Requisition Description
We are looking for a high-performing Java Engineer to join our team managing large-scale data pipelines processing billions of records. You will be responsible for building and optimizing high-throughput services that interface with Kafka, Solr, and HDFS within a Kubernetes environment. The ideal candidate has an 'under the hood' understanding of how Java interacts with Linux and distributed storage.
Roles and Responsibilities Design & Development: Build robust, scalable, and maintainable Java applications using Spring Boot or Microservices architecture and Spark.o Data Orchestration: Develop and optimize Kafka producers/consumers, ensuring efficient data serialization and compression (Snappy/LZ4).o Big Data Integration: Manage data lifecycle operations within HDFS and perform high-speed indexing and querying in Solr.o Performance Tuning: Diagnose and resolve bottlenecks in JVM memory management, Garbage Collection, and JNI-based native library interactions.o Cloud Native Deployment: Containerize applications using Docker and manage deployments in Kubernetes (K8s) across Oracle Linux environments.o Troubleshooting: Solve complex issues related to distributed locking, file system consistency, and network latency at scale.o Technical Requirements: 1. Core Java: Strong expertise in Java 8+ (Collections, Multithreading, Concurrency, and Stream API). 2. Messaging: Deep understanding of Apache Kafka (partitioning strategies, consumer groups, and offset management).
