Adaptive Optimization of Resource Allocation in Parallel Processing of Large Language Models Using Reinforcement Learning Algorithms

Abstract

Given the increasing demand for efficient and rapid execution of large language models (LLMs) within variable and resource-constrained infrastructures, the use of reinforcement learning (RL) algorithms as intelligent decision-making tools for resource allocation is of critical importance. This article, based on real CPU usage data and simulated values for other influential factors such as latency, energy consumption, and bandwidth, constructs a more realistic environment for evaluating resource allocation policies. In the core algorithmic section, three methods have been implemented and compared: Q-Learning as the primary reinforcement approach, SARSA as a similar method more sensitive to the decision sequence, and a Fixed-Policy method as the baseline for comparison. The state space is composed of normalized CPU data and other attributes, while the action space includes combinations of GPU count and data/model/hybrid processing types. The designed reward function is multi-objective, incorporating a balanced mix of factors such as low CPU and memory usage, reduced latency, lower energy consumption, and high bandwidth. Simulation results revealed that Q-Learning achieved the best average performance among the three algorithms. Numerically, the values obtained for Q-Learning were reported as Accuracy = 0.85, Precision = 0.83, F1-Score = 0.84, and Mean Total Reward = 26.7. In comparison, SARSA recorded respective values of 0.79, 0.76, 0.77, and 22.4, while the Fixed-Policy approach yielded the weakest outcomes at 0.74, 0.71, 0.72, and 19.6. Additionally, Q-Learning also demonstrated superior energy efficiency and latency, which are operationally vital in cloud environments. This simulation confirmed that Q-Learning can adaptively and intelligently optimize resource allocation under complex and dynamic conditions, offering better performance than alternative methods.