Note: The job is a remote job and is open to candidates in USA. Deepgram is the leading platform for Voice AI, providing real-time APIs for speech-to-text and text-to-speech. The Embedded AI Engineer will optimize and deploy Deepgram's models on low-power consumer hardware, ensuring efficient performance on resource-constrained devices.
Responsibilities
- Take Deepgram's Speech and Conversational models and get them running on embedded and low-power consumer hardware — defining the architecture for on-device, real-time inference across a diverse range of processors and accelerators
- Optimize models for constrained targets through quantization, pruning, distillation, operator fusion, and architecture-specific compilation to meet strict latency, memory, power, and thermal budgets
- Write and optimize performance-critical runtime code (C, C++, and/or Rust) for embedded environments, including bare-metal and real-time operating systems such as FreeRTOS and Zephyr
- Integrate with industry-standard edge inference runtimes and vendor NPU/DSP toolchains, mapping model graphs efficiently onto on-device accelerators and CPU/GPU/NPU heterogeneity
- Build the on-device runtime plumbing: model packaging, deployment pipelines, over-the-air update mechanisms, and lightweight telemetry for devices operating with limited or intermittent connectivity
- Establish repeatable benchmarking and validation across target hardware — measuring latency, accuracy, power consumption, memory footprint, and resource utilization — and catch regressions before they ship
- Partner with silicon and device vendors on SDK integration and performance tuning, getting our models to run efficiently on new chipsets and reference platforms
- Collaborate with Research and Engine teams to influence model architectures toward edge-friendly designs from the start, reducing the optimization burden at deployment time
Skills
- Experience delivering production systems on resource-constrained hardware — embedded systems, mobile, edge AI, or small low-power devices
- Strong proficiency in C, C++, and/or Rust, with experience writing performance-critical code for constrained environments
- Hands-on experience with model optimization for on-device deployment, including quantization, pruning, knowledge distillation, or architecture-specific compilation
- Familiarity with edge inference runtimes (e.g., ONNX Runtime, TensorRT, TFLite, ExecuTorch) and/or vendor-specific NPU/DSP toolchains
- A strong understanding of hardware-software interaction — CPU/GPU/NPU/DSP architectures, memory hierarchies, fixed-point/integer arithmetic, and power management — and how they affect inference performance
- Experience working close to the metal: bare-metal or RTOS environments (e.g., FreeRTOS, Zephyr), embedded Linux, or microcontroller and edge SoC development
- Strong communication skills and a builder mindset — you can scope an ambiguous optimization problem, drive it to a measurable result, and explain the tradeoffs clearly
- Experience with real-time audio processing on embedded platforms — DSP pipelines, audio codec optimization, wake-word or always-on listening, or streaming inference on microcontrollers and edge SoCs
- Depth in ML optimization techniques — custom quantization schemes, mixed-precision inference, or neural architecture search for edge targets
- Background in hardware evaluation and benchmarking — systematically comparing accelerators, SoCs, or GPUs for specific workload profiles
- Experience shipping AI features in consumer products at scale, and the instinct for what 'production quality' means on a battery-powered device
- Familiarity with model compilation and optimization toolchains and their tradeoffs across hardware targets
- Experience with secure, robust on-device deployment practices — code signing, encrypted model storage, and safe update mechanisms
Company Overview
Company H1B Sponsorship