Software Engineer (AI Performance)

Gimlet Labs is building the foundation for the next generation of AI applications. As generative AI workloads rapidly scale, inference efficiency is becoming the critical bottleneck. Gimlet is redefining AI inference from the ground up, combining cutting-edge research with an integrated hardware-software stack that delivers breakthrough performance, efficiency, and model quality. Gimlet pairs its inference stack with a seamless developer experience, allowing users to deploy, manage, and monitor AI workloads from frameworks like PyTorch and LangChain at production scale in seconds. 

Gimlet is spun out of a Stanford research project under Professors Zain Asgar and Sachin Katti. The founding team has deep experience across AI, distributed systems, and hardware with previous successful exits.

Gimlet Labs is seeking a Software Engineer focused on AI Performance. You will be researching and implementing techniques to drive performance and quality optimizations across the latest AI models. You will implement techniques such as quantization, KV caching, and FlashAttention to enable inference efficiency. You will design parallelism strategies to distribute data and workloads across compute nodes at production scale. You will dive deep into GPU code and kernel optimizations to accelerate AI workloads.

Responsibilities:

• Evaluating and implementing cutting-edge AI research for model performance and efficiency 

• Architecting infrastructure for distributed AI workloads across both the software stack and GPU kernel layers

• Profiling, benchmarking, and analyzing system performance, identifying bottlenecks and optimization opportunities in execution runtimes targeting various hardware systems

Qualifications:

• Bachelor’s degree in computer science, engineering, applied mathematics or comparable area of study

• Experience with performance optimization

Preferred Qualifications:

• Graduate degree in computer science, engineering, applied mathematics or comparable area of study

• Familiarity with compilers and compiler frameworks such as MLIR

• Experience with PyTorch, TensorFlow, vLLM, ONNX and other AI frameworks

• Software development experience with Python, C++, and CUDA