Impact of Quantization on Large Language Models for Portuguese Classification Tasks

Abstract

Large Language Models have emerged as transformative agents in the frequently evolving landscape of artificial intelligence, reshaping the world towards a disruptive and modern technological era. This paradigm stresses their crucial role in extending the generative capabilities in the context of natural language processing. Generative Artificial Intelligence, an innovative and cutting-edge research topic, is critical to unlocking remarkable opportunities in our era of unparalleled technological progress. Despite the remarkable progress made in language model architectures, their exponential growth still raises pertinent concerns regarding their deployment and the associated costs for retraining efforts tailored to specific tasks. We present a study achieving a detailed analysis of the impact resulting from the application of diverse quantization methodologies on an open-source large language model tailored for Portuguese classification tasks, aka Bode. Our research thoroughly evaluates the performance nuances introduced by various quantization strategies, thus providing valuable insights into the constant concerns surrounding the optimization of large language models, aiming for enhanced efficiency and effectiveness in growing applications for the Portuguese community.