BYOD; the smartphone as a machine controller using gestures

July 16, 2013 // By Graham Prophet
A software team within the industry application department of the development service provider, Berner & Mattner has carried out a feasibility study on the subject of Human Machine Interface (HMI): ‘Smart Multi-Axis Control’ implements control of a multi-axis grabbing arm using intuitive gestures made on a commercially available smartphone.

The smartphone’s sensors transmit movement data to the MESSINA real-time platform using a Java server. This data is then converted into control commands for the robotic grabbing arm. The rapid spread of powerful and inexpensive sensors on mobile devices and the ‘bring your own device’ (BYOD) trend are opening up new possibilities in the industrial user interface sector. Instead of using buttons, switches, or other classical interfaces to enter data on machines or devices, manufacturers are looking for opportunities to implement cost-effective and flexible user concepts based on tablet PCs.

“Our teams have already realised modern HMI concepts on a variety of mobile devices for numerous manufacturers,” explains Dr. Michael Sturm, spokesperson for the current proof-of-concept project group for ‘Smart Multi-Axis Control.’

“Our new study shows that tablets and smartphones are no longer just being used as classic input devices. We use the smartphone’s gyroscope and accelerometer to achieve a new method of controlling multi-axis systems. The added bonus: Most humans are capable of intuitive and precise hand-eye coordination, which means it is possible to learn to properly control the device in just a few minutes. The test we carried out with visitors at a trade fair substantiates this.”

The small robotic arm controlled by a smartphone app appeared at the Berner & Mattner booth at the Embedded World event. All components were developed by Berner & Mattner, from the smartphone app to the Java-based server, from the controller of the robotic arm to MESSINA, the platform for the model-based testing of control devices and HiL real-time applications. Berner & Mattner Systemtechnik specialises in systems engineering, development and testing of complex electronic and mechanical systems, offering services ranging from consulting, conceptual design, software and systems development to the setup and operation of entire test and integration systems. Berner & Mattner: