Nadaljevalno učenje s superpozicijo v transformerjih. (Slovenian)

In many machine learning applications, new data is continuously collected, e.g., in healthcare, for weather forecasting etc. Researchers often want a system that allows for continuous learning of new information. This is extremely important even in the case when not all data can be stored indefinitely. The biggest challenge in continual machine learning is the tendency of neural models to forget previously learned information after a certain time. To reduce model forgetting, our continual learning method uses superposition with binary contexts, which require negligible additional memory. We focus on transformer-based neural networks, comparing our approach with several prominent continual learning methods on a set of natural language processing classification tasks. On average, we achieved the best results: 4.6% and 3.0% boost in AUROC (area under the receiver operating characteristic) and AUPRC (area under the precision-recall curve), respectively. [ABSTRACT FROM AUTHOR]

V mnogih aplikacijah strojnega učenja se novi podatki nenehno zbirajo, npr. v zdravstvenem varstvu, za vremenske napovedi itd. Raziskovalci si pogosto želijo sistem, ki bi omogočal nadaljevalno učenje novih informacij. To je izjemnega pomeni tudi v primeru, ko vseh podatkov ni mogoče shranjevati v nedogled. Največji izziv pri nadaljevalnem strojnem učenju je težnja nevronskih modelov, da po določenem času pozabijo prej naučene informacije. Da bi zmanjšali pozabljanje modela, naša metoda nadaljevalnega učenja uporablja superpozicijo z binarnimi konteksti, ki zavzemajo zanemarljiv dodaten pomnilnik. Osredotočamo se na nevronske mreže v obliki transformerjev, pri čemer smo naš pristop primerjali z več vidnimi metodami nadaljevalnega učenja na nizu klasifikacijskih nalog obdelave naravnega jezika. V povprečju smo dosegli najboljše rezultate: 4,6% izboljšavo pri ploščini pod krivuljo ROC (angl. AUROC - area under the receiver operating characteristic) in 3,0% izboljšavo pri ploščini pod krivuljo PRC (angl. AUPRC - area under the precision-recall curve). [ABSTRACT FROM AUTHOR]

Copyright of Uporabna Informatika is the property of Slovensko Drustvo Informatika and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)