Speaker
Description
Enabling robots to operate autonomously in dynamic environments over extended periods requires robust memory, reasoning, and decision-making capabilities. Although large language models (LLMs) have demonstrated significant potential, their limited context size constrains their ability to manage long-term data effectively. Autonomous systems must be able to provide detailed information about a robot’s past actions, locations, and observations—critical for supporting human decision-making and ensuring reliable long-term operation.
To address this challenge, we introduce a Spatio-Temporal Memory System integrated into the RAI framework. This system enables robots to collect, organize, and reason over data accumulated over hours, weeks, or even months. By capturing and structuring spatial, temporal, and visual information, it allows robots to efficiently retrieve relevant data and respond to complex queries with high accuracy and speed.
The system operates through two parallel processes: memory construction and query execution. This design ensures continuous data collection while enabling real-time user queries. To facilitate scalable and efficient data retrieval, the architecture integrates vector databases and NoSQL storage. In addition, it supports multimodal data processing, location- and time-based indexing, and semantic search, significantly improving the ability of a robot to recall and reason about past experiences.
Real-world evaluations demonstrate the effectiveness of the system in improving autonomous decision making and long-term adaptability. This work presents a scalable and practical solution for robots that navigate complex and evolving environments. The implementation is open source and available at: https://github.com/RobotecAI/rai
