Abstract
Big data solutions are designed to cope with data of huge Volume and wide Variety, that need to be ingested at high Velocity and have potential Veracity issues, challenging characteristics that are usually referred to as the "4Vs of Big Data". In order to evaluate possibly complex big data solutions, stress tests require to assess a large number of combinations of sub-components jointly with the possible big data variations. A formalization of the Design of Experiments (DoE) on big data solutions is aimed at ensuring the reproducibility of the experiments, facilitating their partitioning in sub-experiments and guaranteeing the consistency of their outcomes in a global assessment. In this paper, an ontology-based approach is proposed to support the evaluation of a big data system in two ways. Firstly, the approach formalizes a decomposition and recombination of the big data solution, allowing for the aggregation of component evaluation results at inter-component level. Secondly, existing work on DoE is translated into an ontology for supporting the selection of experiments. The proposed ontology-based approach offers the possibility to combine knowledge from the evaluation domain and the application domain. It exploits domain and inter-domain specific restrictions on the factor combinations in order to reduce the number of experiments. Contrary to existing approaches, the proposed use of ontologies is not limited to the assertional description and exploitation of past experiments but offers richer terminological descriptions for the development of a DoE from scratch. As an application example, a maritime big data solution to the problem of detecting and predicting vessel suspicious behaviour through mobility analysis is selected. The article is concluded with a sketch of future works.
Abstract (translated)
大数据解决方案的设计是为了应对海量、种类繁多的数据，这些数据需要以高速吞食，并且具有潜在的准确性问题，具有挑战性的特点，通常被称为“大数据的4V”。为了评估可能复杂的大数据解决方案，压力测试需要评估大量的子组件组合以及可能的大数据变化。大数据解决方案实验设计（DOE）的形式化旨在确保实验的再现性，促进子实验的划分，并确保其结果在全球评估中的一致性。本文提出了一种基于本体的方法，以两种方式支持大数据系统的评估。首先，该方法将大数据解决方案的分解和重组形式化，允许在组件间级别聚合组件评估结果。其次，将现有的DOE工作转化为支持实验选择的本体论。这种基于本体的方法提供了将评估领域和应用领域的知识相结合的可能性。它利用域和域间对因子组合的特定限制来减少实验次数。与现有的方法相反，本体论的提议使用不仅限于对过去实验的断言描述和利用，而且为从头开始开发DOE提供了更丰富的术语描述。作为一个应用实例，选择了一种通过机动性分析来检测和预测船舶可疑行为的海上大数据解决方案。文章最后是对未来作品的概述。
URL
https://arxiv.org/abs/1904.08626
