Latest demo of HanS, the Health and Safety Robot Inspector (June 2022)

HanS is an experiment in service robotics and has been designed to be able to patrol KMi to try and identify possible violations to Health and Safety rules. For instance, typical rules check that all fire extinguishers are in place and that emergency exits are free of obstacles and clearly labelled through the correct signs.

This video shows HanS’ risk assessment of the KMi space. The platform relies on standard Robot Operating System (ROS) modules for robot navigation and localisation, while also introducing novel customised components for visual sensemaking and commonsense reasoning. Specifically, HanS was trained to recognise 60 object classes that are commonly found in KMi from only a few training examples. An Edge TPU accelerator is mounted on the robot, to ensure fast learning times. An important innovation developed for HanS is that the robot is also aware of the typical sizes and spatial relations between objects, as well as the common-sense facts about these objects. For example, it knows that paper is flammable and that flammable objects may cause a fire, if placed near electrical devices. These properties were autonomously extracted and summarised from a number of Knowledge Bases, including WordNet [1], ShapeNet [2], Visual Genome [3], Amazon product information, and Quasimodo [4].  These capabilities improve HanS’ sensemaking ability and also allow it to maintain a semantic map of the KMi space, where observations that refer to the same object are aggregated and updated over time, to reflect changes in the environment.

[1] Miller, G. A. (1995). WordNet: a lexical database for English. CACM, 38(11), 39-41.
[2] Savva, M., Chang, A. X., & Hanrahan, P. (2015). Semantically-enriched 3D models for common-sense knowledge. IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 24-31).
[3] Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., … & Fei-Fei, L. (2017). Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123(1), 32-73.
[4] Romero, J., Razniewski, S., Pal, K., Z. Pan, J., Sakhadeo, A., & Weikum, G. (2019, November). Commonsense properties from query logs and question answering forums. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 1411-1420).