About the company Our client is building the next-generation data platform designed for AI/ML workloads. Our open-source distributed data engine, Daft, is already deployed at scale, running on over 800K CPU cores daily. Backed by Y Combinator, Caffeinated Capital, and leading Silicon Valley investors, we are redefining how AI engineers and data scientists work with large-scale data. Our founding team comes from top tech firms such as Tesla, Anyscale, and Lyft, bringing deep expertise in AI, distributed computing, and data infrastructure.
Roles and Responsibilities
Core Infrastructure Development: Architect and optimize the AI/ML infrastructure layer of Daft to support high-throughput, distributed data processing.
High-Performance Computing: Implement efficient scheduling, execution engines, and memory management for AI workloads.
Cloud & Distributed Systems: Design and scale cloud-native distributed systems that integrate seamlessly with AWS, Ray, and modern data lakes.
AI/ML Optimization: Collaborate with research teams to fine-tune AI data pipelines, model training, and inference workloads.
Systems-Level Engineering: Work with Rust, C++, or similar low-level programming languages to optimize performance-critical components.
Early-Stage Startup Execution: Drive technical initiatives in a fast-paced, high-autonomy environment, shaping the product roadmap.
Job Requirements
Experience: 4+ years in high-performance computing, ML/AI infrastructure, or distributed systems development.
Deep Technical Expertise:
Strong background in data-intensive applications, parallel computing, or GPU/accelerator optimization.
Proficiency in Python (NumPy, PyTorch, TensorFlow) and familiarity with Rust, C++, or Go.
Experience with cloud platforms (AWS, GCP) and distributed computing frameworks (Ray, Spark, Dask, or Kubernetes).
AI/ML Background: Prior work on data pipelines, model serving, or optimizing training/inference workflows.
Startup & Product Experience: Hands-on experience shipping production software at a startup or early-stage company (<50 employees preferred).
Educational Background: CS degree from a top-30 university (MIT, Stanford, Berkeley, CMU, UIUC, Caltech, Georgia Tech, Princeton, etc.) is a strong plus.
Bonus Skills:
Open-source contributions, especially in ML/AI/data infrastructure.
Technical blogging, public speaking, or community engagement in AI/ML.
Entrepreneurial experience or a strong ownership mindset.