Cristin-resultat-ID: 1970452
Sist endret: 22. februar 2022, 12:50
NVI-rapporteringsår: 2021
Resultat
Vitenskapelig artikkel
2021

Big data workflows: Locality-aware orchestration using software containers

Bidragsytere:
  • Andrei-Alin Corodescu
  • Nikolay Nikolov
  • Akif Quddus Khan
  • Ahmet Soylu
  • Mihhail Matskin
  • Amir H. Payberah
  • mfl.

Tidsskrift

Sensors
ISSN 1424-8220
e-ISSN 1424-8220
NVI-nivå 1

Om resultatet

Vitenskapelig artikkel
Publiseringsår: 2021
Publisert online: 2021
Volum: 21
Hefte: 24
Sider: 1 - 27
Artikkelnummer: 8212
Open Access

Importkilder

Scopus-ID: 2-s2.0-85120809412

Beskrivelse Beskrivelse

Tittel

Big data workflows: Locality-aware orchestration using software containers

Sammendrag

The emergence of the Edge computing paradigm has shifted data processing from centralised infrastructures to heterogeneous and geographically distributed infrastructures. Therefore, data processing solutions must consider data locality to reduce the performance penalties from data transfers among remote data centres. Existing Big Data processing solutions provide limited support for handling data locality and are inefficient in processing small and frequent events specific to the Edge environments. This article proposes a novel architecture and a proof-of-concept implementation for software container-centric Big Data workflow orchestration that puts data locality at the forefront. The proposed solution considers the available data locality information, leverages long-lived containers to execute workflow steps, and handles the interaction with different data sources through containers. We compare the proposed solution with Argo Workflows and demonstrate a significant performance improvement in the execution speed for processing the same data units. Finally, we carry out experiments with the proposed solution under different configurations and analyze individual aspects affecting the performance of the overall solution.

Bidragsytere

Andrei-Alin Corodescu

  • Tilknyttet:
    Forfatter
    ved DIG Digitalisering ved Universitetet i Oslo

Nikolay Nikolov

  • Tilknyttet:
    Forfatter
    ved Sustainable Communication Technologies ved SINTEF AS

Akif Quddus Khan

  • Tilknyttet:
    Forfatter
    ved Institutt for datateknologi og informatikk ved Norges teknisk-naturvitenskapelige universitet

Ahmet Soylu

  • Tilknyttet:
    Forfatter
    ved Institutt for informasjonsteknologi ved OsloMet - storbyuniversitetet

Mihhail Matskin

  • Tilknyttet:
    Forfatter
    ved Kungliga Tekniska högskolan
1 - 5 av 7 | Neste | Siste »