中国科学院重庆绿色智能技术研究院机构知识库

QQ客服 官方微博 反馈留言

	Adaptively-Accelerated Parallel Stochastic Gradient Descent for High-Dimensional and Incomplete Data Representation Learning
	Qin, Wen1,2; Luo, Xin3; Zhou, Mengchu4,5,6
	2024-02-01
摘要	High-dimensional and incomplete (HDI) interactions among numerous nodes are commonly encountered in a Big Data-related application, like user-item interactions in a recommender system. Owing to its high efficiency and flexibility, a stochastic gradient descent (SGD) algorithm can enable efficient latent feature analysis (LFA) of HDI data for its precise representation, thereby enabling efficient solutions to knowledge acquisition issues like missing data estimation. However, LFA on HDI data involves a bilinear issue, making SGD-based LFA a sequential process, i.e., the update on a feature can impact the results on the others. Intervening the sequence of SGD-based LFA on HDI data can affect the training results. Therefore, a parallel SGD algorithm to LFA should be designed with care. Existing parallel SGD-based LFA models suffer from a) low parallelization degree, and b) slow convergence, which significantly restrict their scalability. Aiming at addressing these vital issues, this paper presents an Adaptively-accelerated Parallel Stochastic Gradient Descent (AP-SGD) algorithm to LFA by: a) establishing a novel local minimum-based data splitting and scheduling scheme to reduce the scheduling cost among threads, thereby achieving high parallelization degree; and b) incorporating the adaptive momentum method into the learning scheme, thereby accelerating the convergence rate by making the learning rate and acceleration coefficient self-adaptive. The convergence of the achieved AP-SGD-based LFA model is theoretically proved. Experimental results on three HDI matrices generated by real industrial applications demonstrate that the AP-SGD-based LFA model outperforms state-of-the-art parallel SGD-based LFA models in both estimation accuracy for missing data and parallelization degree. Hence, it has the potential for efficient representation of HDI data in industrial scenes.
关键词	Parallel algorithm industrial application latent feature analysis high-dimensional and incomplete data stochastic gradient descent parallelization shared-memory data science
DOI	10.1109/TBDATA.2023.3326304
发表期刊	IEEE TRANSACTIONS ON BIG DATA
ISSN	2332-7790
卷号	10期号:1页码:92-107
通讯作者	Luo, Xin(luoxin21@gmail.com)
收录类别	SCI
WOS记录号	WOS:001165410200007
语种	en