Description

Vi manages a petabyte scale data lakehouse that drives data and ML pipelines across all our products. Vi is looking for an engineer with deep expertise building and scaling big-data and ML pipelines. The role will be a hybrid between an FDE that owns integrations of customer data with Vi’s data lakehouse and a platform engineer responsible for packaging Vi’s products into repeatable, scalable, automated pipelines.

Tech Stack

• Apache PySpark, Apache Iceberg, AWS EMR, Glue, S3

• Python, PyTorch, sklearn, xgboost, catboost, mlflow

• AWS SageMaker, Airflow

Key Responsibilities

• Own the design and implementation of data and ML pipelines based on a combination of customer first party data and Vi’s internal data lakehouse. 

• Prototype predictive insight products and run rapid feasibility studies to assess new commercial opportunities for Vi’s capabilities built on petabyte scale data lakehouse.

• Identify commonalities between customer engagements to synthesize reusable data capabilities.

What We’re Looking For

• Deep expertise building large-scale data analytics pipelines with Apache Spark.

• Comfortable using Python and AWS technologies for data engineering and ML. 

• Strong communication skills to interact with technical counterparts across customer accounts to coordinate data integrations.

Nice to Have

• Experience in the healthcare and life sciences domain.

• Familiarity with ML and statistical modeling methodologies and experimental design.