The teaching
Teaching is probably the most important part of a successful recognition process. However, many people only think of the one image that our customers take on site, but far from it. We enable “teaching with just one image” by showing the visioncheckout over 6 million images of dishes and retail articles during the development phase. These were then analyzed by 7 million neurons and divided into their individual features. These could be size, shape or color, for example. The visioncheckout is now an absolute pro at recognizing and distinguishing the individual features. What happens now during on-site training is just the final touch: this basic knowledge is now specialized precisely to the dishes served that day. This is how we achieve an accuracy of over 99%!
The scanning
Scan. Pay. Enjoy. All without a single click. Sounds obvious, doesn’t it? What sounds like a no-brainer is actually the result of a sophisticated development process. The key is that the scanning process
- is triggered automatically
- is completed automatically (and thus payment is started)
- and is recognized when a guest leaves.
These three steps must work for every possible payment process, no matter what special cases may arise. Some guests, for example, only push their tray halfway under the visioncheckout. Others leave no gap to their predecessor’s tray, so you can’t rely on that either. Another very common occurrence is that someone briefly takes the key from the tray. If they scan at this exact moment, it can happen that something is covered by the hand.
By cleverly analyzing the movements under the visioncheckout, we were able to identify all these special cases so well that the “checkout without a single click” is not just an empty shell for the best-case scenario, but a reality for our customers.
The blurring
The visioncheckout is trained to extract background objects and make them unrecognizable before recognition. This means that no personal data is stored and the recognition process is not distorted by unknown objects.
This step takes just a few tenths of a second. This ensures that it does not slow down the checkout and still ensures a secure payment process.
The recognition
As soon as the tray has been scanned and the background objects have been extracted, the core of the visioncheckout starts: the recognition.
This step raises the most questions and at the same time causes the greatest astonishment. We are not surprised, because with an accuracy of over 99%, the recognition works with a lower error rate than at self-tipping checkouts and is also many times faster. The visioncheckout has to answer two questions in the shortest possible time:
Where are the items located?
Even before the items are assigned to the created articles, the visioncheckout must determine where they are located. Thanks to our pixel-perfect segmentation, it can determine in great detail where the boundary between the items is and what shape they have. This is the first feature needed to answer the second question.
Which articles are there?
In addition to the shape, the visioncheckout also uses properties such as the color or size of the items. All these properties are now analyzed and evaluated in the second step in order to assign the dishes on the image to the items stored in the menu.
The smart correction
The last step only occurs in the rarest of cases, but even the most secure system is well advised to have a false bottom!
We have done just that with our Smart Corrector. If the visioncheckout is wrong when assigning the meals, the most similar articles are suggested to the guest. The recognition can now be corrected and the checkout process continues to run smoothly. Corrected trays are also checked out faster than at the self-tipping checkout.
For our existing customers, only 0.35% of all trays checked in with visioncheckout need to be corrected, but we have nevertheless focused on making this process as intuitive and fast as possible.
To give our customers even more flexibility in their menu design, we have developed various additional features, such as our Weigh & Pay solution. If you are interested in one of these extensions or would like to get to know visioncheckout in general, please book your personal live demo.
