Samsung Gesture-Controlled Scratch App at a Sarajevo Shopping Mall

Samsung needed a way to introduce its latest smartphones to shoppers in a busy mall in a way that felt like discovery rather than another advertisement competing for attention.

A phone launch in a mall corridor has a specific problem: the product is small, the screen showing it is large, and a looping product video says everything at once. Once the whole device is on screen, there is nothing left for the shopper to do. The brief was to make the reveal itself the reason to stop.

TapGest, in partnership with Samsung, built an Interactive Scratch App on digital signage that turned the reveal of new smartphones into a playful game: each scratch uncovered a glimpse of the phone's design and features.

Instead of showing the device, the application hides it under a covering layer that the shopper removes with their own hand movement. The information is the same product material Samsung supplied; what changed is that the shopper uncovers it at their own pace rather than waiting for a loop to reach the right frame.

Shoppers stepped up to the digital display and swiped their hand in the air in front of it — the camera tracked the movement and scratched away the layer covering the content. No touchscreen and no app were involved. With each reveal, excitement grew as participants anticipated the moment the new phones would be fully unveiled.

Step by step: a shopper walks up to the screen and is picked up by the camera, raises a hand, and moves it across the air in front of the display. The scratch follows the hand in real time, so the connection between the movement and the screen is obvious within the first second. Swiping continues until the device underneath is fully exposed, at which point the shopper sees the new phone and its features and can simply walk on — there is no form, no registration and no staff step to finish.

The gesture-controlled scratch mechanic runs on camera-based interaction on the mall's digital signage, with custom campaign content built around Samsung's product materials for each device being launched.

The installation is the mall's digital signage display, a camera facing the space in front of it, and the TapGest scratch application. Hand position is derived from the camera image alone: there is no touch foil on the screen, no controller, no wearable and no depth sensor to calibrate. Because the interaction is entirely touchless, the display stays a display — nothing on it is handled by the public.

The campaign layer is content, not code. The hidden artwork, the reveal underneath and the feature callouts come from Samsung's product materials for each device being launched, which is what allows the same mechanic to be pointed at a new phone without rebuilding the application.

The consideration that decides whether an installation like this works is the space in front of the screen. The camera defines an interaction zone, and a shopper only becomes a player once they stand inside it — so where the screen sits in the corridor, and how much room a person has to raise an arm without stepping into passing traffic, matters as much as the tracking itself.

The app quickly became a popular attraction in the mall, sparking curiosity and encouraging shoppers to learn more about Samsung's new devices, while the partnership with a trusted global brand added credibility and reach to the activation.