The Barcelona Supercomputing Center (BSC)-Microsoft Research Center is pleased to announce the launch of Project Aloja, a joint research project with the goal of providing automated optimization for the performance of Hadoop infrastructure deployments. Hadoop is an open source software system which is experiencing rapid adoption growth and is used for the processing of extremely large data sets.
The initial focus of the project is to carry out a systematic study of Hadoop execution performance across a broad range of different hardware and software configurations deployed to both on premise physical servers and cloud based infrastructures. The performance results from these tests, as well as the testing methodology and test infrastructure will be made available to the public through a portal hosted by the Barcelona Supercomputing Center. Our intent is that companies, individuals and organizations investigating, deploying and evangelizing the Hadoop solution stack will benefit from this growing database of performance results and configuration guidance.
The Aloja project started with a preliminary 8 month planning and incubation cycle and now has first year funding of greater than $1.1 million. The team of researchers, based in Barcelona (Spain) and Redmond Washington (US) are led by Rob Reinauer (Microsoft) and David Carrera (BSC-CNS).
The BSC-Microsoft Research Center has been actively involved in investigations into software interactions across both multi-core processors and massively parallel processing architectures since 2008. This center had been established following a first successful joint project between the Microsoft Technical Computing Initiative in Redmond, U.S. and BSC from 2006 to 2008 on applications of Transactional Memories to highly parallel multicore and many core processors.
