R

Software Engineer, Product

RainesDev
Full-time
On-site
San Francisco, California, United States
Software / Technology / IT
About the company Our client is building the next-generation data platform designed for AI/ML workloads. Our open-source distributed data engine, Daft, is already deployed at scale, running on over 800K CPU cores daily. Backed by Y Combinator, Caffeinated Capital, and leading Silicon Valley investors, we are redefining how AI engineers and data scientists work with large-scale data. Our founding team comes from top tech firms such as Tesla, Anyscale, and Lyft, bringing deep expertise in AI, distributed computing, and data infrastructure.

Roles and Responsibilities
  • Core Infrastructure Development: Architect and optimize the AI/ML infrastructure layer of Daft to support high-throughput, distributed data processing.
  • High-Performance Computing: Implement efficient scheduling, execution engines, and memory management for AI workloads.
  • Cloud & Distributed Systems: Design and scale cloud-native distributed systems that integrate seamlessly with AWS, Ray, and modern data lakes.
  • AI/ML Optimization: Collaborate with research teams to fine-tune AI data pipelines, model training, and inference workloads.
  • Systems-Level Engineering: Work with Rust, C++, or similar low-level programming languages to optimize performance-critical components.
  • Early-Stage Startup Execution: Drive technical initiatives in a fast-paced, high-autonomy environment, shaping the product roadmap.

Job Requirements
  • Experience: 4+ years in high-performance computing, ML/AI infrastructure, or distributed systems development.
  • Deep Technical Expertise:
    • Strong background in data-intensive applications, parallel computing, or GPU/accelerator optimization.
    • Proficiency in Python (NumPy, PyTorch, TensorFlow) and familiarity with Rust, C++, or Go.
    • Experience with cloud platforms (AWS, GCP) and distributed computing frameworks (Ray, Spark, Dask, or Kubernetes).
  • AI/ML Background: Prior work on data pipelines, model serving, or optimizing training/inference workflows.
  • Startup & Product Experience: Hands-on experience shipping production software at a startup or early-stage company (<50 employees preferred).
  • Educational Background: CS degree from a top-30 university (MIT, Stanford, Berkeley, CMU, UIUC, Caltech, Georgia Tech, Princeton, etc.) is a strong plus.
  • Bonus Skills:
    • Open-source contributions, especially in ML/AI/data infrastructure.
    • Technical blogging, public speaking, or community engagement in AI/ML.
    • Entrepreneurial experience or a strong ownership mindset.