Cristin-resultat-ID: 1894256
Sist endret: 9. mars 2023, 14:19
NVI-rapporteringsår: 2022
Resultat
Vitenskapelig artikkel
2022

Backtracking gradient descent method and some applications in Large scale optimisation. Part 1: Theory

Bidragsytere:
  • Tuyen Trung Truong og
  • Hang-Tuan Nguyen

Tidsskrift

Minimax Theory and its Applications
ISSN 2199-1413
e-ISSN 2199-1421
NVI-nivå 1

Om resultatet

Vitenskapelig artikkel
Publiseringsår: 2022
Volum: 7
Hefte: 1
Sider: 79 - 108

Importkilder

Scopus-ID: 2-s2.0-85148941410

Beskrivelse Beskrivelse

Tittel

Backtracking gradient descent method and some applications in Large scale optimisation. Part 1: Theory

Sammendrag

Deep Neural Networks (DNN) are essential in many realistic applications, including Data Science. At the core of DNN is numerical optimisation, in particular gradient descent methods (GD). The purpose of this paper is twofold. First, we prove some new results on the backtracking variant of GD under very general situations. Second, we present a comprehensive comparison of our new results to the previously known results in the literature, providing pros and cons of these methods. To illustrate the efficiency of Backtracking line search, we will present some experimental results (on validation accuracy, training time and so on) on CIFAR10, based on implemetations developed in another paper by the authors. Source codes for the experiments are available on GitHub.

Bidragsytere

Trung Tuyen Truong

Bidragsyterens navn vises på dette resultatet som Tuyen Trung Truong
  • Tilknyttet:
    Forfatter
    ved Flere komplekse variable, logikk og operatoralgebraer ved Universitetet i Oslo

Hang-Tuan Nguyen

  • Tilknyttet:
    Forfatter
    ved Vietnam
1 - 2 av 2