Divide-and-Iterate Approach to Big Data Systems
IEEE Transactions on Services Computing
Matrix calculations are often required for the analysis of any big-data cloud computing system. It is quite common to process big-data associated matrices possessing the sparsity and low-rank properties. In order to efficiently deal with big-data matrices, we propose a new divide-and-iterate framework, which can be invoked to solve an enormously large linear system of equations by taking advantage of factored matrices. The Kaczmarz algorithm (KA) is utilized here to design the parallel iterative algorithms which are capable of solving a large system of equations by iteratively updating the solution through the reduction into the factorized subsystems in parallel. The convergences of our proposed new iterative algorithms are justified by the rigorous proofs. Besides, the time- and memory-complexities are studied to demonstrate the resource efficiency of the proposed algorithms. Numerical experiments are also presented to illustrate the effectiveness of this proposed new framework.
big data, Kaczmarz algorithm, matrix factorization, random iterations, tremendous linear systems
Applied Data Science
Shih Yu Chang and Hsiao Chun Wu. "Divide-and-Iterate Approach to Big Data Systems" IEEE Transactions on Services Computing (2022): 1967-1979. https://doi.org/10.1109/TSC.2020.3027580