Note: The job is a remote job and is open to candidates in USA. ITRex Group is a pioneering AI company that builds real-world systems and solutions. They are seeking an AI Inference Engineer with strong C++ expertise to deploy and optimize AI models for production, focusing on integrating and enhancing existing products with machine learning capabilities.
Responsibilities
- Work on deploying machine learning models to edge devices using the frameworks: llama.cpp, ggml
- Collaborate closely with researchers to assist in coding, training and transitioning models from research to production environments
- Integrate AI features into existing products, enriching them with the latest advancements in machine learning
Skills
- 4+ years of professional experience in Modern C++ (C++17/20)
- Strong knowledge of memory management, multithreading, profiling and performance optimization
- Experience debugging low-level issues (memory leaks, fragmentation, OOM, concurrency)
- Experience working with Linux development environments
- Experience integrating machine learning models into production applications
- Experience deploying and optimizing AI inference pipelines
- Hands-on experience with AI inference frameworks such as: llama.cpp (strong plus), ggml (strong plus), ONNX Runtime, TensorRT / TensorRT-LLM, OpenVINO, MLC LLM, ExecuTorch, TVM
- Experience profiling inference performance and optimizing memory usage and latency
- Strong understanding of modern AI model architectures, including: Transformer architecture, Large Language Models (LLMs), Diffusion Models, Tokenization, Attention mechanisms, KV Cache, Quantization techniques, Model conversion and deployment
- Experience working with one or more of the following: LLM deployment, Computer Vision models, OCR models, Multimodal models, Speech models, Image generation models
- Experience evaluating new models and integrating them into existing products is highly desirable
- CUDA
- Vulkan Compute
- Metal
- OpenCL
- Typescript
- Python
- Experience contributing to open-source AI infrastructure projects
Benefits
- Remote flexibility: Work where and how you work best - we trust you to deliver
- Fair compensation: Competitive salary + benefits that matter (medical, learning)
- Ownership opportunities: See a problem worth solving? Own it. We back smart risks over bureaucratic safety
- AI enhancement: We leverage AI to make you faster and stronger - complementing your abilities, not replacing them
- Learning investment: English classes, professional development
- Career progression: Real paths up, not just sideways shuffling
- Responsive teammates: No ignored Slacks, no "not my problem" attitudes
- Supportive culture: When you're stuck, people help. When things break, we fix them together
- Human connections: Regular meetups, tech talks, and actual relationships beyond work
Company Overview