We are seeking a skilled and ambitious Big Data Engineer to join our dynamic team. As a Big Data Engineer, you will be responsible for building and maintaining the infrastructure that supports the processing, storage, and analysis of large-scale data. You will collaborate with multi-disciplinary teams to develop robust data pipelines and architectures that enable seamless data integration, while ensuring data quality and security.
Responsibilities:
- Design and Develop Data Solutions: Collaborate with cross-functional teams to understand business requirements and design scalable and efficient big data solutions. Develop and implement data pipelines, ETL processes, and data warehousing solutions.
- Data Ingestion and Integration: Develop and maintain data ingestion pipelines from various sources, ensuring smooth and reliable data flow into our systems. Integrate real-time and batch data to support near-real-time analytics.
- Data Storage and Management: Build and maintain data storage systems, data lakes, and data warehouses to provide a consolidated and easily accessible repository for structured and unstructured data.
- Performance Optimization: Fine-tune and optimize data processing and storage systems to achieve high performance and low latency data access. Monitor system performance and take proactive measures to ensure optimal data processing.
- Data Quality and Governance: Implement data quality checks, validation procedures, and data governance practices to maintain data accuracy, consistency, and security.
- Emerging Technologies: Stay updated with the latest trends and advancements in big data technologies and cloud computing. Evaluate and recommend suitable technologies to enhance our data engineering capabilities.
- Collaboration and Support: Collaborate closely with data scientists, analysts, and other engineering teams to support their data needs and provide the necessary infrastructure for their analytics projects. Troubleshoot and resolve data-related issues in a timely manner.
- Documentation and Communication: Document data engineering processes, architecture, and best practices. Communicate complex technical concepts to non-technical stakeholders effectively.
Qualifications and Requirements:
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related field.
- Proven experience as a Big Data Engineer or a similar role, with a strong background in building and maintaining large-scale data processing systems.
- Proficiency in programming languages such as Python, Java, Scala, or similar, with experience in implementing data processing algorithms and data manipulation.
- Familiarity with big data processing frameworks like Apache Hadoop, Apache Spark, or similar technologies.
- Experience with cloud-based data platforms, such as AWS, Azure, or GCP, and knowledge of cloud-native data services.
- Understanding of data modeling, data warehousing concepts, and database design principles.
- Strong problem-solving skills and an analytical mindset to handle complex data-related challenges.
- Excellent communication skills with the ability to work collaboratively in a team-oriented environment.
Join our innovative team, where your expertise in big data engineering will play a crucial role in transforming how businesses harness data-driven insights for success. At [Company Name], we value diversity and inclusivity, and we encourage all qualified candidates to apply.
At TIGI HR, we are an equal opportunity employer that values diversity and is dedicated to fostering an inclusive environment for all employees.
Our services are available in all major cities including Bangalore, Pune, Mumbai, Gurgaon, Noida, Hyderabad, Ahmedabad, Kolkata, Chennai, Gandhinagar, Indore etc. We have also expanded our wings toward Australia and Canada too.
Also, we do have many jobs for Work From Home (WFH) and on a Remote basis as well.
