Ernesto Damiani gave a keynote at the 3rd International Winter School on Big Data (BigDat 2017), titled “Model-driven Development of Big Data Applications”. The school has been organized from the 13th to the 17th of February, 2017, at the University of Bari.

The advent of Big Data has highlighted a number of problems in the design, development and deployment of applications. Big data applications involve multiple components (collection and cleaning, data lake creation and management, analytics parallelization etc.) that have different features and lifecycles, and correspond to a rich panoply of software tools. Experience has shown that model-based approaches leading from technology independent to technology dependent models and finally to deployment support well the design of data-intensive applications. However, while classic MDA transformations introduce each technology-dependent feature at a pre-set stage of the model refinement chain, Big Data computations Deliver their best in term of scalability and efficiency by making binding architectural and data modeling decisions at the last possible moment, i.e. when information on Big Data distribution, volume and variety becomes available.
This talk discusses how Big-Data-as-a-Service, where data features can be different for each deployment of the analytics, can benefit from a Software Product-Line (SPL) parametric approach to keep multiple alternative models alive and postpone binding modeling decisions via variation points, in order to make them at the right (i.e., the last possible) moment. We discuss how to delay and parameterize modeling decisions for all key aspects of a Big Data application, including data preparation, representation and storage, analytics parallelization and visualization.