A semantically-based big data processing system using hadoop and map-reduce

Wanting，Wang1; Zheng，Qin1,2

2016

1868-4238

1868-422X

IFIP Advances in Information and Communication Technology

477

246-247

In financial industry, a wide range of financial systems generate vast amount of data in different structures, which change with compliance rules change and hard to manage due to their heterogeneity. This paper introduces a semantically-based big data processing system to integrate the data from different sources, which realizes the query and computation in semantic layer. The system provides a new data management way for the financial industry. With Semantic Web, the information can be managed, integrated, and collaborated in a more fluent way than it in traditional ETL. In order to clear the complex logical relationship among data, the system uses SPARQL to query. Through Map-Reduce, this system, based on Hadoop and Hbase can improve the processing speed for big data.

Big data Data integration Distributed computation Semantics

其他

英语

2-s2.0-84979643017

Scopus

1.School of Information Management and Engineering,Shanghai University of Finance and Economics,Shanghai,China
2.South University of Science and Technology of China,Shenzhen,China

Wanting，Wang,Zheng，Qin. A semantically-based big data processing system using hadoop and map-reduce[C],2016:246-247.