Senior Software Engineer (Data Platform)

Who You Are

We are looking for an experienced Software Engineer with a focus on Distributed System tech leadership to be a part of our growing Data Platform team. You are bold and creative, and have deep empathy for customers. You will design and implement significant parts of the code base and will have the opportunity to make an immediate impact with your work and guide the product and team as we grow.

The Data Platform team at Phaidra is responsible for -

• Building streaming & batch components that power both realtime & historical data ingestion & processing

• Building and optimising the data and feature stores

• Building high-throughput and highly-available multi-modal APIs used throughout the organization, from agent training, to realtime inference to user facing applications.

• Building the central orchestration & metadata layer, and APIs of the core Platform.

• Building services to fetch and store external third party data (ex. weather forecasts) required for AI training.

**We are seeking a team member located in one of the following areas: the UK, Norway, Italy, Spain, Portugal, or Ireland.

Responsibilities

The Data Platform team that you are joining is responsible for building the core data platform that powers data ingestion, processing, storage and serving for all customers. We focus on multi-tenancy as a core construct while building all the layers and interact with various internal teams to provide automated tooling to utilise the raw/processed data. This is a Software Engineering role at the core, and you will join as an experienced engineer to scale our service across batch and stream processing as we grow.

As an organization, we strongly believe in expertise across the stack. As such, you will experience flavors of Software Engineering, Distributed Systems, Machine Learning, MLOps and DevOps. As a Software Engineer(Tech Lead, Data Platform) you will play a Tech Lead role in the Data platform team and in particular:

• Build and design scalable components for the Data platform to allow high throughput data ingestion & data processing, which feeds the data into agents doing realtime inference for autonomous control of industrial systems.

• Design and develop systems to store and serve batch data for agent training or analytics.

• Contribute to the design and implementation of the product backend by writing REST & gRPC API services and scalable event-driven backend applications.

• Design clear, extensible software interfaces for the team's customers and maintain a high release quality bar.

• Design and optimize data storage & retrieval mechanisms for high throughput, security & ease of access.

• Perform DevOps duties of CI/CD, Release & Deployment management.

• Be a part of our global production oncall team and, own & operate your services in production, meeting Phaidra’s high bar for operational excellence.

• Lead cross-functional initiatives collaborating with engineers, product managers and TPM across teams.

• Mentor your peers and be a technical role-model in the team.

Onboarding

In your first 30 days...

• You will be immersed in an onboarding program that introduces you to Phaidra and our product.

• You will spend time in the Engineering org, learning how the teams operate, interact, and approach problems.

• You will read various parts of our handbook and familiarize yourself with the documentation culture at Phaidra.

• You will set up your development environment and start working on an onboarding exercise that will introduce you to various parts of our code base.

• You will learn about how we use agile and be able to navigate our sprint boards and backlogs.

• You will learn about various team standards and development & release processes.

• You will start to learn about our system architecture and infrastructure.

• You will start picking up few good “first-tasks” to get yourself accustomed to the end to end release flow.

By your first 60 days...

• You will have a solid understanding of what Phaidra does and how we do it.

• You will have met with team members across Phaidra and started building relationships that will help you be successful at your job.

• You will have completed the onboarding exercise and will be on your way to completing your first production task.

• You will have identified areas of improvement in the data platform and started thinking towards them while working with the team.

By your first 90 days...

• You will have been fully integrated in the team and with team members across the company.

• You will get a more in-depth understanding of our system architecture and infrastructure.

• You will have completed your first on-call experience helping monitor and improve our production environments.

• You will have become an expert with our tooling.

• Be a distributed systems expert in the company, drive discussions on important topics and present them in the learning sessions.

Key Qualifications

• 7+ years of work experience.

• Strong experience on designing and implementing distributed systems for data ingestion, storage and processing

• Extensive experience with platform Software Engineering with the ability to contribute on all levels as an individual contributor

• Strong expertise on building, operating and monitoring large scale multi-tenant systems with high availability, fault tolerance, performance tuning, monitoring, and metrics collection

• Ability to take ownership of realtime production systems - aligning technical with business requirements, raising the bar for operational excellence and on-call incident handling

• Strong expertise in Cloud environments and any of Python, Go or Java

• Ability to collaborate and communicate effectively in an all-remote setting

• Doing your work with curiosity, ownership, transparency & directness, outcome orientation, and customer empathy.

• Experience with latest batch & realtime data processing, storage & service technologies.

Bonus

• Experience with containerization and orchestration technologies like Docker & Kubernetes.

• Experience with building applications that can be deployed in cloud, as well as in hybrid or on-prem environment

• Experience with building self-serve Data Platform and/or high scale DataLake/Lakehouse.

Our Stack

• Python, Go

• Ray.io, Prefect

• REST & gRPC micro-services

• Docker, Kubernetes, Terraform & Kapitan

• GCP - GKE, PubSub, CloudSQL, BigTable, Postgres, MongoDB, BigQuery etc.

• Grafana Cloud, Prometheus

• Poetry, Pants

• Gitlab CI, ArgoCD, Atlantis

General Interview Process

All of our interviews are held via Google Meet, and an active camera connection is required.

• Hiring manager interview (30 minutes): The purpose of this meeting is for you to get to know the manager for the role. This chat will mainly focus on your previous experience and technical background. You can expect to talk about projects that you have worked on in the past and ask any questions about the team & role.

• Meeting with Operations (30 minutes) The purpose of this interview is to meet you, learn more about your background, discuss what you are looking for in a new position and cover formalities around your application.

• Technical Interview 1 (75 minutes) The purpose of this interview is to assess your skills in data structures, algorithms and problem solving. This interview will involve live coding exercises.

• Technical Interview 2 (90 minutes) In this interview, we will go over a real world distributed systems problem. You can expect to draw architecture diagrams using boxes & arrows in your browser. We will talk about system design, scalability and monitoring.

• Tech Lead + Product interview (45 minutes): This interview is a combination of technical and cultural fit assessment. We will cover your technical experience and the skills as an engineer and a tech lead while discussing projects that you have worked on in the past.You will meet with a member of the Product team as well as our VP of Engineering, with the opportunity to ask any questions about the team, role and engineering at Phaidra

• Culture fit interview with Phaidra’s co-founders (30 minutes): This interview focuses on alignment with Phaidra’s values and the mutual cultural fit.

Base Salary

UK Residents: £94,400-£141,600/year

Salary ranges for other countries (Italy, Spain, Portugal, Norway, Ireland) will vary based on the market rate for the location.

This position will also include equity.

These are best faith estimates of the base salary range for this position. Multiple factors such as experience, education, level, and location are taken into account when determining compensation.