Visual Check-In: a Large-Scale Visual Check-In System for TV Content-Aware Web with Cient-Side Video Analysis Offloading
Proceedings of the 18th International Conference on Web Information Systems Engineering (WISE 2017), pages 159–174, Moscow, Russia, October 2017.
The intuitive linkage between TV and the web brings about new opportunities to motivate people to watch video content or visit websites. A check-in system that recognizes which specific programs are being watched by users is highly effective in promoting TV content. However, such a check-in system faces two technical problems: the temporal characteristics of broadcasting media, resulting in a massive number of simultaneous check-in requests, and the wide variation of audience environments, such as lighting, cameras, and TV devices. We propose a visual check-in system for linking websites and TV programs. The system identifies what program a user is watching by analyzing the visual features of a video captured with a smartphone. The key technology is a real-time video analysis framework that achieves both scalability to an enormous number of simultaneous requests and practical robustness in terms of content identification. We have constructed a special color scheme consisting of 120 (non-neutral) colors to absorb differences in the illumination levels of user environments. This color scheme plays an important role in offloading video analysis tasks onto the client-side in a tamper-proof way. Our system assigns a unique color scheme to each user and verifies a check-in request using the corresponding color scheme, thus preventing malicious users from sharing the analysis results with others. Experimental results using a real dataset demonstrate the accuracy and efficiency of the proposed method. We have applied the system to actual TV programs and clarified its scalability and precision in a production environment.
Citation: .bib format