Architectures for Distributed High Performance Video Processing in the Cloud.

Session Date & Time: Day 2, March 24, 18:00 EST (45 min)
Keywords: Video, Map Reduce, Amazon EC2, HaaS, PaaS.
Authors: Karin Breitman and Markus Endler, Depto. Informática, Pontifícia Universidade Católica do Rio de Janeiro; Marcello Azambuja and Rafael Pereira, Globo.com.
Abstract: TV is dead! at least, as we know it. The traditional model, with clear separation between roles (producers, broadcasters and consumers) is giving way to a Cloud based one, where users produce, distribute, combine and watch video content anywhere, anytime, and using a multitude of different devices. In our presentation we’ll discuss some of its challenges, and demonstrate the solutions we’re working whith:
o Production challenges – The exponential growth of User-Generated-Content (UGG) makes it virtually impossible to estimate the volume of resources needed to run open submission systems, in particular those whose usage is seasonal. We introduce a cloud-based architecture that addresses this problem, and demonstrate an instance application that runs the registration system for candidates who wish to participate in the Brazilian Big Brother reality TV show (1).

• Distribution challenges – The proliferation of mobile device types is pushing the demand for processing services to unprecedented levels. Every object is processed several times, to secure encoding standard compatibility with different devices (PC's, mobile_phones, media_centers, game_consoles,etc), which support different codecs (H.264_Baseline, Main and High_Profiles, H.263,etc), as well as compression rates, to adapt to local storage and bandwidth restrictions. We introduce a generalization of the Map-Reduce architecture to tackle this problem, and demonstrate a running private-cloud implementation that reduces HD video encoding times dramatically – to pre-fixed values, independently of content duration (2).
• Consuming challenges - Internet allows the mix and match of multi-source content. It’s commonplace for networks to mix their own footage with UGC to provide: Haiti earthquake coverage featured as much mobile phone videos than professional footage. We discuss our recent research, applying Semantic Web and machine-learning techniques to improve indexing, querying and retrieval of data in the Cloud.

(1) This application is required to receive and process large numbers of personal, homemade videos. Videos are accepted in any format, but need to be encoded in the MPEG standard, to facilitate visualization by the jurors. There are few, but steep peaks of usage, as about 60% of the total submissions (circa 200,000 videos) is expected on the last weekend.
(2) This application was conceived to quickly encode and distribute sports videos, soccer in particular, processed from very large HD original footage (petabyte scale), over the web. The demand varies greatly over the year, as national championships evolve, with periods where there is a need to process up to ten games simultaneously. Seasonal events, e.g. World Cup, pose similar challenges.

Karin Breitman is a professor at PUC-Rio where she coordinates the Computer Engineering Program. Her research focuses on Software Engineering and its applications in Knowledge Representation, Requirements Engineering, Semantic Web and Cloud Computing. She is the Publications Director of the Brazilian Computer Society, and IFIP’s TC-1 Secretary. In 2009 she was the PC Chair of two IEEE conferences (ICECCS’09 & SEW’09), 2 workshops (IEEE ECBS-LARC'09 and WIT), as well as FME’s sponsored ICFEM’09. Author of over 70 research papers in conference proceedings and journals received numerous research grants from public institutions and private industry, including NASA, CNRS, IBM and more recently Hewlett Packard. Her next book, “Analogy and Metaphor in Information Technology” is due Spring 2010, by Springer- Verlag. She actively collaborates with SBC in their gender equity initiative, and recently received the Google Brazil Women in Information Technology award for her contribution in promoting the importance of gender related issues in her country.

Markus Endler obtained his doctoral degree in Computer Science from the Technical University of Berlin, and the Professor Livre-docente title from the University of São Paulo. From 1989 to 1993 he worked as a researcher at the GMD Research Institute Karlsruhe (Germany), and from 1994 to 2000 as an Assistant Professor at the Institute of Mathematics and Statistics of the University of São Paulo. In 2001 he joined the Department of Informatics of the Pontifícia Universidade Católica in Rio de Janeiro (PUC-Rio), where he is currently Associate Professor. His main research interests include Mobile, Ubiquitous and Cloud Computing. He has supervised 7 doctoral and 17 master theses, and has published 90+ scientific papers. He is member of the ACM, the Brazilian Computer Society (SBC) and of the IFIP Working Group 6.1.