I had mentioned in my last post that I would add AI to my project next. Here is an update on this project. The video below has the details. Under that is a summary of the changes done. All of these are running on NVIDIA Jetson Orin Nano, which is the cheapest in the Jetson family. 1K CUDA cores, 8GB shared RAM, and a powerhouse for such a small footprint.
- Added 4 AI models for inference. All pre-trained, no training involved yet.
- Piper for text-to-speech, Whisper for speech-to-text. Although Jetson keeps falling back on eSpeak.
- YOLO for vision processing.
- TinyLlama for language processing. It's an SLM, a small language model rather than an LLM, with about 1.1 billion parameters. To contrast, ChatGPT 5 is rumored to have trillions of parameters.
- My UI does have a training mode for learning from actual mowing sessions, but I am hesitant to turn my backyard into a training ground.
- The agentic flow yields about 4 seconds of latency, which is good for an SLM running on a small machine. It started at about 20 seconds and took quite some effort to tune it down to 4 seconds.
- It does hallucinate when asked complex questions. But I intend to use it only for very simple navigational commands, augmented by deterministic code and safeguards before it executes navigation.
- Some of the challenges that can slow you down on an edge AI device like this are drivers and port assignment.
- By default NVIDIA ships the unit with no OS, and all GPIO ports disabled. They are pre-assigned for specific protocols but still need to be enabled via jetson-io. It will look something like this.
- Most aftermarket peripherals come with drivers that need to be installed separately. Relay board, Arduino, NRF transceiver etc had to be enabled using CH340/341 drivers because they use proprietary Chinese chips.