Divide-and-Iterate Approach to Big Data Systems

Publication Date

1-1-2022

Document Type

Article

Publication Title

IEEE Transactions on Services Computing

Volume

15

Issue

4

DOI

10.1109/TSC.2020.3027580

First Page

1967

Last Page

1979

Abstract

Matrix calculations are often required for the analysis of any big-data cloud computing system. It is quite common to process big-data associated matrices possessing the sparsity and low-rank properties. In order to efficiently deal with big-data matrices, we propose a new divide-and-iterate framework, which can be invoked to solve an enormously large linear system of equations by taking advantage of factored matrices. The Kaczmarz algorithm (KA) is utilized here to design the parallel iterative algorithms which are capable of solving a large system of equations by iteratively updating the solution through the reduction into the factorized subsystems in parallel. The convergences of our proposed new iterative algorithms are justified by the rigorous proofs. Besides, the time- and memory-complexities are studied to demonstrate the resource efficiency of the proposed algorithms. Numerical experiments are also presented to illustrate the effectiveness of this proposed new framework.

Keywords

big data, Kaczmarz algorithm, matrix factorization, random iterations, tremendous linear systems

Department

Applied Data Science

Link to Full Text

Share

COinS