Applied On-Chip Machine Learning for Dynamic Resource Control in Multithreaded Processors
Abstract
In this paper, we propose a machine learning algorithm to control instruction fetch bandwidth in a simultaneous multithreaded CPU. In a simultaneous multithreaded CPU, multiple threads occupy pools of hardware resources in the same clock cycle. Under some conditions, one or more threads may undergo a period of inefficiency, e.g., a cache miss, thereby inefficiently using shared resources and degrading the performance of other threads. If these inefficiencies can be identified at runtime, the offending thread can be temporarily blocked from fetching new instructions into the pipeline and given time to recover from its inefficiency, and prevent the shared system resources from being wasted on a stalled thread. In this paper, we propose a machine learning approach to determine when a thread should be blocked from fetching new instructions. The model is trained offline and the parameters embedded in a CPU, which can be queried with runtime statistics to determine if a thread is running inefficiently and should be temporarily blocked from fetching. We propose two models: a simple linear model and a higher-capacity neural network. We test each model in a simulation environment and show that system performance can increase by up to 19% on average with a feasible implementation of the proposed algorithm.
References
- 1. , Simultaneous multithreading: A platform for next-generation processors, IEEE micro 17(5) (1997) 12–19. Crossref, ISI, Google Scholar
- 2. , Converting thread-level parallelism to instruction-level parallelism via simultaneous multithreading, ACM Transactions on Computer Systems (TOCS) 15(3) (1997) 322–354. Crossref, ISI, Google Scholar
- 3. , Simultaneous multithreading: Maximizing on-chip parallelism, in 25 Years of the International Symposia on Computer Architecture (Selected Papers) (ACM, 1998), pp. 533–544. Crossref, Google Scholar
- 4. , Dynamically controlled resource allocation in SMT processors, in Proc. of the 37th Annual IEEE/ACM Int. Symp. on Microarchitecture (IEEE Computer Society, 2004), pp. 171–182. Crossref, Google Scholar
- 5. , Exploiting choice: Instruction fetch and issue on an implementable simultaneous multithreading processor, ACM SIGARCH Computer Architecture News 24 (1996) 191–202. Crossref, Google Scholar
- 6. , Improving memory latency aware fetch policies for SMT processors, in Int. Symp. on High Performance Computing (Springer, 2003), pp. 70–85. Crossref, Google Scholar
- 7. , A memory-level parallelism aware fetch policy for SMT processors, in IEEE 13th Int. Symp. on High Performance Computer Architecture (HPCA 2007),
2007 (IEEE, 2007), pp. 240–249. Crossref, Google Scholar - 8. , Optimising long-latency-load-aware fetch policies for SMT processors Int. J. High Performance Computing and Networking 2(1) (2004) 45–54. Crossref, Google Scholar
- 9. , Learning-based SMT processor resource distribution via hill-climbing, ACM SIGARCH Computer Architecture News 34(2) (2006) 239–251. Crossref, Google Scholar
- 10. , Coordinated management of multiple interacting resources in chip multiprocessors: A machine learning approach, in Proc. of the 41st Annual IEEE/ACM Int. Symp. on Microarchitecture (IEEE Computer Society, 2008), pp. 318–329. Crossref, Google Scholar
- 11. , Pattern recognition, Machine Learning 128 (2006) 1–58. Google Scholar
- 12. , Deep Learning (MIT Press, 2016). Google Scholar
- 13. J. Sharkey, D. Ponomarev and K. Ghose, M-SIM: A flexible, multithreaded architectural simulation environment, Technical Report CS-TR-05-DP01 (2005). Google Scholar
- 14. , The SimpleScalar tool set, version 2.0, ACM SIGARCH Computer Architecture News 25(3) (1993) 13–25. Crossref, Google Scholar
- 15. , Alpha AXP architecture, Communications of the ACM 36(2) (1993) 33–44. Crossref, ISI, Google Scholar
- 16. , SPEC CPU2006 benchmark descriptions, ACM SIGARCH Computer Architecture News 34(4) (2006) 1–17. Crossref, Google Scholar
- 17. ,
Cross-validation , in Encyclopedia of Database Systems (Springer, 2009), pp. 532–538. Crossref, Google Scholar - 18. M. Abadi et al., Tensorflow: Large-scale machine learning on heterogeneous distributed systems, arXiv preprint arXiv:1603.04467 (2016). Google Scholar


