Uploaded May 2018 | Updated September 2026, 2 weeks ago
ICRA 2018 Spotlight Video
Interactive Session Wed PM Pod I.4
Authors: Gong, Ze; Zhang, Yu
Title: Temporal Spatial Inverse Semantics for Robots Communicating with Humans
Abstract:
Effective communication between humans often embeds temporal and spatial context. While spatial context captures geographic settings of objects in the environment, temporal context describes the process of their changes over time. In this paper, we propose temporal spatial inverse semantics (TeSIS) to extend the inverse semantics approach to also consider the temporal context for robots communicating with humans. Inverse semantics generates natural language requests while taking into account how well the humans would interpret those requests given the current spatial context. Compared to inverse semantics, our approach incorporates also spatial context information in the past. To achieve this, we extend the sentence structure in inverse semantics to generate sentences that can refer to not only the current but also previous states of the environment. A new metric based on the extended sentence structure is developed by breaking a single sentence into multiple independent sentences that refer to environment states at different times. To evaluate our approach, we randomly generate scenarios in an experimental domain. Each scenario includes the description of the current and several immediate previous states. Natural language sentences are generated for these scenarios using both inverse semantics and our approach. Amazon MTurk is used to compare the sentences generated and results show that TeSIS achieves better accuracy, sometimes by a significant margin, than the baseline.
ICRA 2018 Spotlight Video
Interactive Session Wed PM Pod I.4
Authors: Gong, Ze; Zhang, Yu
Title: Temporal Spatial Inverse Semantics for Robots Communicating with Humans
Abstract:
Effective communication between humans often embeds temporal and spatial context. While spatial context captures geographic settings of objects in the environment, temporal context describes the process of their changes over time. In this paper, we propose temporal spatial inverse semantics (TeSIS) to extend the inverse semantics approach to also consider the temporal context for robots communicating with humans. Inverse semantics generates natural language requests while taking into account how well the humans would interpret those requests given the current spatial context. Compared to inverse semantics, our approach incorporates also spatial context information in the past. To achieve this, we extend the sentence structure in inverse semantics to generate sentences that can refer to not only the current but also previous states of the environment. A new metric based on the extended sentence structure is developed by breaking a single sentence into multiple independent sentences that refer to environment states at different times. To evaluate our approach, we randomly generate scenarios in an experimental domain. Each scenario includes the description of the current and several immediate previous states. Natural language sentences are generated for these scenarios using both inverse semantics and our approach. Amazon MTurk is used to compare the sentences generated and results show that TeSIS achieves better accuracy, sometimes by a significant margin, than the baseline.
