Senior Data Engineer - Data Insights

You’ll Succeed With: 

• BS or MS in computer science or a related technical field.

• Proven experience as a big data engineer or a similar role, with a deep understanding of big data technologies, frameworks, and best practices.

• Designing and implementing large-scale, distributed data processing systems using technologies such as Apache Hadoop, Apache Spark, or Apache Flink.

• Developing and optimizing data pipelines and workflows for ingesting, storing, processing, and analyzing large volumes of structured and unstructured data.

• Building and maintaining data infrastructure, including data lakes, data warehouses, and real-time streaming platforms.

• Designing and implementing data models and schemas for efficient data storage and retrieval.

• Ensuring the scalability, availability, and fault-tolerance of big data systems through proper configuration, monitoring, and performance tuning.

• Identifying and evaluating new technologies, tools, and frameworks to improve the efficiency and effectiveness of big data processing.

• Implementing data security and privacy measures to protect sensitive information throughout the data lifecycle.

• Knowledge of distributed computing principles and technologies, such as HDFS, YARN, and containerization (e.g., Docker, Kubernetes).

• Understanding of real-time streaming technologies and frameworks, such as Kafka (Confluent) or Apache Flink.

• Strong problem-solving skills and ability to optimize and tune big data processing systems for performance and scalability.

• Excellent communication and teamwork skills to collaborate with cross-functional teams and stakeholders.

Nice to Haves:

• Leadership skills: guiding and mentoring junior members, coordinating projects and collaborating with other teams.

• Problem solving: Dealing with complex challenges, optimizing data pipelines and ensuring data quality and troubleshooting issues.

• Domain Knowledge:Having understanding of the hospitality industry can greatly enhance your ability to design and build effective data pipelines.

• Data Architecture Expertise: Contributing to designing data architecture and system and having deep understanding of data storage and ability to design a robust, scalable and maintainable architecture is a valuable skill.

• Performance Optimization: Being able to optimize the performance of data pipelines is a crucial skill such as query optimization, indexing, caching and data partitioning and sharding.

• Data Governance and compliance: having an understanding of different data governance principles and compliance regulations such as GDPR and HIPAA is very important.

• Programming knowledge: Knowledge of Python and SQL is essential, familiarity with Java and other languages is a plus.

• Experience working with a remote-first and globally distributed team

• Experience with CI/CD tooling, including GitHub Actions and Build Workflows

• Knowledge of Confluent and AWS

#LI-IK1