Summary
It has been said that falls and tumbles account for more than half of all accidents at nursing care facilities. If accidents can be reduced, the burden on these facilities can be significantly reduced. For the device to work in the facilities, it must work in sunlight and the dark at night. In order not to impose a burden on care receivers, sensing technology that takes privacy into account is necessary to achieve this.
Since OMRON has the technology to develop algorithms using AI technology with OKAO Vision*, we can build the necessary algorithms for pose estimation based on this technology. The pose estimation technology uses AI technology to estimate human skeletal points from distance data output from the 3D TOF sensor module B5L. With this technology, it has become possible to sense a person's condition that is less affected by sunlight, operates in the dark, and is privacy-conscious. Therefore, we believe that developing algorithms from skeletal point information can quantitatively recognize a person's posture and that analyzing skeletal points can be used to estimate a person's falls and tumbles.
What is Pose Estimation?
Pose estimation is a technology that uses distance measurement data output from 3D TOF sensor B5L to estimate human skeletal points and output their coordinates using OMRON's proprietary algorithm that utilizes AI technology. From these skeletal points, it is possible to quantitatively recognize what kind of posture a person is in and to understand a person's posture, which is generally called pose estimation or skeletal estimation. As an application of pose estimation, we believe that customers can use the algorithm to analyze these skeletal points to estimate the possibility of a person falling and tumbling.
Background of Development
A growing number of people being cared for has led to a chronic labor shortage in the nursing field due to the aging population. It is expected that the labor shortage will worsen in the future.
It is said that falls and tumbles account for more than half of all accidents at nursing care facilities. Accident response and post-accident handling place a heavy burden on the nursing care field. To reduce the burden, when it is presumed that an accident has occurred, the system first remotely checks the condition of the person being cared for, and then only if necessary, a caregiver rushes to the person's room, thus economizing on manpower. However, installing cameras in the room of a person being cared for is often avoided for privacy reasons.
We have devised a solution to grasp a person's condition while considering privacy by developing a pose estimation technology algorithm that estimates human skeletal points from distance data output from the 3D TOF sensor B5L.
Since OMRON has the technology to develop algorithms using AI technology with OKAO Vision*, we can build the necessary algorithms for pose estimation based on this technology through deep learning.
AI technology generates human skeletal points using the output of privacy-conscious 3D TOF sensor
Features of Pose Estimation
Extracting human skeletal point information with algorithm developed by OMRON using AI technology
Algorithm with advantages of small size and high speed enables GPU-less operation
Stable operation under sunlight and at night without lighting
Based on the "OKAO Vision"* technology, we have developed a highly accurate algorithm to optimize the human skeletal points through AI technology from the output of a distance to a person measured by B5L.
Deep learning has reduced the risk of the skeletal points of one person being cared for becoming transferred to another person or the legs of a chair or table.
The video accurately displays skeletal points based on pose estimation following one to three persons' movements, such as sitting and bending.
OMRON Pose Estimation Software - Privacy-Conscious Software --
By applying the compact and high-speed technology developed over many years in the development of OKAO Vision*, an image-sensing product that can be embedded in PCs and mobile devices, to this product, it can be operated on a PC without the need for the host to provide a device such as a GPU.
Due to the living environment of the person being cared for, sunlight may enter the room through windows. However, B5L's light resistance allows it to output distance data to the person cared for, even in sunlight or at night without lighting. From this distance data, the skeletal points are estimated using the algorithms developed by OMRON.
Note: B5L is equipped with an LED for distance measurement and can measure distance even at night when there is no light.
The video shows that skeletal point estimation for pose estimation is affected little, even in an environment with sunlight through opened curtains.
OMRON Pose Estimation Software - Output in sunlight or at night without lighting –
General Specifications
| Item | Specifications |
|---|---|
| B5L installation angle *1 | Roll direction: Horizontal to ground Pitch direction: Horizontal or oblique to the human body (0 to -60°) |
| B5L installation height *1 | 1.8 to 2.5 m |
| Sensing distance *1 | 0.5 to 3.0 m |
| Detection condition *1 | Human body with at least 7 of the 13 skeletal points visible |
| Max. number of detections *2 | 3 |
| Output points *3 | Position coordinates (x, y) and confidence level for the 13 skeletal points in the figure above |
| Human body detection | Included |
| Target device | Laptop *No GPU required |
| Detectable posture | Sitting, standing, walking, bending, lying down * Skeletal points can be detected in the above postures |
| Processing speed *4 | ca. 5FPS |
*1 Depending on the installation height, camera angle, and distance between the B5L and the human body, a human body that does not show 7 or more skeletal points within the angle of view due to being too close or other reasons may not be detected or the detected coordinates may not be accurate.
*2 If four or more people are in the image, the confidence level of the human skeletal points is comprehensively judged internally, and the top three people with the highest results are output.
*3 Skeletal points that are hidden by something and not in the image tend to be output with a low confidence level.
*4 Values for typical images in OMRON's DB for Intel Core i7 CPU @ 3GHz.
System Configuration Example
The 3D TOF sensor B5L and the pose estimation software must be used as a set, and a host PC is required to install the pose estimation software.
Shown below is an example of a system in which the output data from the 3D TOF sensor B5L is used for estimating skeletal points using pose estimation software installed in the host PC, outputting XY coordinate data, and judging whether the person being cared for has fallen or not, as well as sending the judgment results to the caregiver.
Example Application
Extract human skeletal point information
Care and monitor
Application
From skeletal point data for pose estimation, falls can be estimated.
Software for Trial
If you would like to try and evaluate our pose estimation software, please contact us via our contact form, a distributor who handles OMRON products, or an OMRON salesperson.
