A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM
In: IEEE Access, Vol 11, Pp 35007-35021 (2023, 2023
Online
academicJournal
Zugriff:
Extracting information of interest from continuous video streams is a strongly demanded computer vision task. For the realization of this task at the edge using the current de-facto standard approach, i.e., deep learning, it is critical to optimize key performance metrics such as throughput and energy consumption according to prescribed application requirements. This allows achieving timely decision-making while extending the battery lifetime as much as possible. In this context, we propose a method to boost neural-network performance based on a co-execution strategy that exploits hardware heterogeneity on edge platforms. The enabling tool is Apache TVM, a highly efficient machine-learning compiler compatible with a diversity of hardware back-ends. The proposed approach solves the problem of network partitioning and distributes the workloads to make concurrent use of all the processors available on the board following a pipeline scheme. We conducted experiments on various popular CNNs compiled with TVM on the Jetson TX2 platform. The experimental results based on measurements show a significant improvement in throughput with respect to a single-processor execution, ranging from 14% to 150% over all tested networks. Power-efficient configurations were also identified, accomplishing energy reductions above 10%.
Titel: |
A Pipelining-Based Heterogeneous Scheduling and Energy-Throughput Optimization Scheme for CNNs Leveraging Apache TVM
|
---|---|
Autor/in / Beteiligte Person: | Velasco-Montero, Delia ; Goossens, Bart ; Fernandez-Berni, Jorge ; Rodriguez-Vazquez, Angel ; Philips, Wilfried |
Link: | |
Zeitschrift: | IEEE Access, Vol 11, Pp 35007-35021 (2023, 2023 |
Veröffentlichung: | IEEE, 2023 |
Medientyp: | academicJournal |
ISSN: | 2169-3536 (print) |
DOI: | 10.1109/ACCESS.2023.3264828 |
Schlagwort: |
|
Sonstiges: |
|