PetroBERT: A Domain Adaptation Language Model for Oil and Gas Applications in Portuguese - Rodrigues, Rafael B. M. [UNESP]; Privatto, Pedro I. M. [UNESP]; de Sousa, Gustavo José [UNESP]; Murari, Rafael P. [UNESP]; Afonso, Luis C. S. [UNESP]; Papa, João P. [UNESP]; Pedronette, Daniel C. G. [UNESP]; Guilherme, Ivan R. [UNESP]; Perrout, Stephan R.; Riente, Aliel F. (Springer Nature, 2022)
This work proposes the PetroBERT, which is a BERT-based model adapted to the oil and gas exploration domain in Portuguese. PetroBERT was pre-trained using the Petrolês corpus and a private daily drilling report corpus over BERT multilingual and BERTimbau. The proposed model was evaluated in the NER and sentence classification tasks and achieved interesting results, which shows its potential for such a domain. To the best of our knowledge, this is the first BERT-based model to the oil and gas context.
Keywords: BERT; Domain adaption; Oil and gas