IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017 1525

An Ensemble-Based Stacked Sequential Learning
Algorithm for Remote Sensing Imagery Classification

Danillo R. Pereira, Rodrigo J. Pisani, André N. de Souza, and João P. Papa, Member, IEEE

Abstract—Contextual-based image classification attempts at
considering spatial/temporal information during the learning pro-
cess in order to make the classification process smarter. Sequen-
tial learning techniques are one of the most used ones to perform
contextual classification, being based on a two-step classification
process, in which the traditional noncontextual learning process is
followed by one more step of classification based on an extended
feature vector. In this paper, we propose two ensemble-based ap-
proaches to make sequential learning techniques less prone to er-
rors, since their effectiveness is strongly dependent on the feature
extension process, which ends up adding the wrong predicted label
of the neighborhood samples as new features. The proposed ap-
proaches are validated in the context of land-cover classification,
being their results considerably better than some state-of-the-art
techniques in the literature.

Index Terms—Land-cover classification, optimum-path forest
(OPF), sequential learning.

I. INTRODUCTION

MACHINE-LEARNING techniques have been considered
a game-changing in the way one organizes and analyzes

data in different areas. Although traditional pattern recognition
techniques have been widely used in several research areas [1],
such approaches usually assume that the dataset samples are
independent and identically distributed. Such an assumption
means that the samples’ spatial/temporal dependence are not
considered during the learning process, which may be a serious
shortcoming when dealing with applications that require such
knowledge. Time-series prediction of finance-related problems
and meteorological observations are some examples that may
not fit well in models that do not employ contextual information.
In the context of image classification, a way to introduce prior
knowledge in the problem formulation is to employ smoothness

Manuscript received September 29, 2016; revised November 16, 2016 and
December 8, 2016; accepted December 14, 2016. Date of publication Jan-
uary 15, 2017; date of current version March 22, 2017. This work was
supported by FAPESP under Grant #2013/20387-7, Grant #2014/16250-9,
and Grant #2014/12236-1, and by CNPq under Grant #303182/2011-3, Grant
#470571/2013-6, Grant #487032/2012-8, and Grant #306166/2014-3. (Corre-
sponding author: João P. Papa.)

D. R. Pereira is with the University of the West São Paulo, Presidente Prudente
19050-920, Brazil (e-mail: dpereira@ic.unicamp.br).

R. J. Pisani is with the Nature Sciences Institute, Federal University of
Alfenas, Alfenas 37130-000, Brazil (e-mail: pisanigeo@gmail.com).

A. N. Souza is with the Department of Electrical Engineering, São Paulo
State University, Bauru 17033-360, Brazil (e-mail: andrejau@feb.unesp.br).

J. P. Papa is with the Department of Computing, São Paulo State University,
Bauru 17033-360, Brazil (e-mail: papa@fc.unesp.br).

Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/JSTARS.2016.2645820

constraints considering the spatial context of the data. When
looking at a picture or a video, for instance, one can realize
that the pixels vary smoothly with the exception of the high-
frequency regions (borders and discontinuities).

Pixel-based classification has been the foremost approach
for satellite image classification over the past decades, since the
assumption of independent and identically distributed pixels is
surmised and, hence, employed by a considerable number of
works [2]–[6]. However, the problem of modeling each pixel
as a sole entity without considering its neighborhood may lead
us to a poorly labeled image, since pixels within homogeneous
regions are likely to describe the similar content. A forthright
approach to deal with this problem is the so-called contextual-
based classification, in which a given pixel and its neighborhood
are then used to enhance the labeling process [7], [8]. Such
approaches, also referred to semantic-based classification [9],
make use of additional data provided by spatial/temporal
information to provide more accurate results. Among several
approaches for such purpose, one may highlight the stacked se-
quential learning (SSL) proposed by Cohen and Carvalho [10],
which aims at modeling contextual information by means of a
two-stage classification process: 1) in the first one, a naı̈ve clas-
sification step is performed, i.e., a training procedure followed
by the classification of test samples are carried out; and 2) soon
after, the original feature vectors of training and test samples are
extended with the labels of their neighborhood, followed by an
ordinary training and classification processes. The SSL uses a
unique classifier in the first step of the two-stage classification.

Later on, Gatta et al. [11] proposed a multiscale sequen-
tial learning approach, in which the contextual information is
obtained not only from the sample’s neighborhood, but also
from pixels far apart. The idea of multiple scales is driven by
several Gaussian-convolved labeled images, which are former
obtained by means of a traditional classification process. After-
ward, Puertas et al. [12] addressed the aforementioned work in
the context of multiclass-based classification problems. Sampe-
dro et al. [13] proposed a similar approach to the one introduced
by Puertas et al. [12], but now in a 3-D space and using error-
correcting output codes in the context of medical image classi-
fication, and Gonzaléz et al. [14] employed the SSL paradigm
for pedestrian detection. In 2015, Puertas et al. [15] presented
an improvement of the former multiscale sequential learning
approach proposed by Gatta et al. [11]. This new version now
supports multiclass problems, as well as the authors proposed to
compress the final extended feature vector with minimal effect
in the recognition rates.

1939-1404 © 2017 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.



1526 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

Roscher et al. [16] used an incremental learning algorithm
based on support vector machines (SVMs) to classify hyper-
spectral data, being the main idea to improve the quality of
the training set with new informative samples, but at the same
time removing noninformative ones. Apart from remote sensing-
based applications, but related to sequential learning, Subrama-
nian and Suresh [17] presented a neuro-fuzzy-based sequen-
tial learning algorithm that aimed at controlling the learning
process. Roughly speaking, the idea is to use the error over a
misclassified sample in order to learn the best training strategy
for that sample. Other works have employed online sequen-
tial learning techniques either, but using extreme learning ma-
chines and in the context of social networks [18] and time-series
prediction [19].

Since sequential learning also concerns contextual classifica-
tion, a number of related works can be referred in the literature.
Susa et al. [20] have recently used the Gaussian process to clas-
sify remote sensing images by means of a Bayesian-theoretic
framework, and Li et al. [21] highlighted the importance of em-
ploying spatiocontextual information in remote sensing image
classification. SVMs have been considered in contextual-based
learning either [22], [23], where Markov random fields (MRFs)
play the role of considering neighboring information during
the learning process. Recently, Osaku et al. [24] showed that
one can improve the classification of satellite images using a
contextual version of the optimum-path forest (OPF) classifier
[25]–[27], which makes use of MRFs, the spatial dependence
among nearby pixels either.

Although the aforementioned works obtained very promising
results, it is possible to improve their approaches by means of
ensemble of classifiers. The main drawback with respect to se-
quential learning is related to the misclassified samples during
the first classification step, which can lead the second classifier
to errors, since the feature vector of each sample is extended with
the predicted labels of its neighbors. Albeit all the aforemen-
tioned approaches for sequential learning may employ different
methodologies, all of them rely on the very same idea of ex-
tending the number of features for each sample. Therefore, this
paper proposes two ensemble-based approaches to sequential
learning, which are validated in the context of land-use clas-
sification. We have observed that a committee of learners can
improve the classification results when dealing with sequential
learning approaches, which are strongly dependent on the first
(initial) classification results.

The proposed approaches are validated over OPF and a Naı̈ve
Bayes classifier, since they are parameterless and quite fast for
training. However, it is important to shed light on that the pro-
posed approaches can be used with any other supervised pattern
recognition techniques. In regard to SSL-based implementations
considering the OPF classifier, the reader can refer to only one
very recent work conducted by Pereira et al. [28], which evalu-
ated OPF in the context of land-use classification in satellite and
radar images. Therefore, the main contributions of this paper are
threefold: 1) to present a voting-based and a 2) concatenation-
based approach to enhance sequential learning, and 3) to foster
the research on the OPF classifier regarding remote sensing

images. The OPF classifier has gained attention and popularity
in the past years [29], [30], since it has been consistently simi-
lar or even more accurate than SVMs, but faster for training. It
has a number of advantages over some state-of-the-art pattern
recognition techniques.

1) It is parameterless.
2) It does not make assumptions about separability of the

samples in the feature space.
3) It can be easily adapted to different situations by just

changing some of its main components.
Roughly speaking, OPF can be seen as a framework to the

design of pattern classification techniques based on the theory
about OPFs, which means one can design a new OPF classifier
by just designing some of its modules (more details about it are
given further).

The remainder of this paper is organized as follows.
Section II presents a brief theoretical background about the
OPF and Naı̈ve-Bayes, and Section III revisits the techniques of
SSL employed in this paper for comparison purposes. The novel
proposed approaches are described in Section IV. The method-
ology and experimental results are discussed in Sections V
and VI, respectively. Section VII states conclusions and future
works.

II. THEORETICAL BACKGROUND

In this section, a brief theoretical background concerning the
pattern recognition techniques used in this work is presented.

A. Optimum-Path Forest

The OPF framework is a recent highlight to the development
of pattern recognition techniques based on graph partitions. The
nodes are the data samples, which are represented by their corre-
sponding feature vectors, and are connected according to some
predefined adjacency relation (i.e., a complete or a k-NN graph).
Given some key nodes (prototypes), they will compete among
themselves aiming to conquer the remaining nodes. Thus, the
algorithm outputs an OPF, which is a collection of optimum-
path trees (OPTs) rooted at each prototype. This work employs
the OPF classifier proposed by Papa et al. [25], [26], which
uses a complete graph and a path-cost function that computes
the maximum arc-weight along a path. Additionally, the key
nodes are the nearest samples from different classes, which are
obtained by means of a minimum spanning tree (MST) com-
putation over the training set. Follow, below, a more detailed
explanation about the OPF mechanism.

Let D = D1 ∪ D2 be a labeled dataset, such that D1 and D2
stand for the training and test sets, respectively. Let P ⊂ D1
be a set of prototypes of all classes (i.e., key samples that best
represent the classes). Let (D1 , A) be a complete graph, whose
nodes are the samples in D1 and any pair of samples defines an
arc in A = D1 ×D1 , as displayed in Fig. 1(a). Additionally, let
πs be a path in (D1 , A) with terminus at sample s ∈ D1 .

The OPF algorithm proposed by Papa et al. [25], [26] employs
the path-cost function fmax due to its theoretical properties for
estimating prototypes (Section II-A1 gives further details about



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1527

Fig. 1. (a) Training set modeled as a complete graph. (b) MST computation over the training set (prototypes are highlighted). (c) OPF over the training set. (d)
Classification process of a “green” sample. (e) Test sample is finally classified.

this procedure):

fmax(〈s〉) =
{

0, if s ∈ P
+∞, otherwise

fmax(πs · 〈s, t〉) = max{fmax(πs), d(s, t)} (1)

where d(s, t) stands for a distance (e.g., Euclidean distance)
among nodes s and t, such that s, t ∈ D1 . Therefore, fmax(πs)
computes the maximum distance between adjacent samples in
πs , when πs is not a trivial path. Roughly speaking, the main
idea of OPF is to minimize fmax(πt), ∀t ∈ D1 . By minimizing
fmax , one can create an OPF rooted at S, which is essentially
a collection of OPTs that are composed of their most strongly
connected nodes.

1) Training: Let P∗ be an optimum set of prototypes when
the OPF algorithm minimizes the classification errors for every
s ∈ D1 . Notice that P ∗ can be found by exploiting the theoretical
relation between the MST and the OPT for fmax [31]. The
training essentially consists of finding P ∗ and an OPF classifier
rooted at P ∗.

By computing an MST in the complete graph (D1 , A) [see
Fig. 1(b)], one can obtain a connected acyclic graph whose nodes
are all samples ofD1 and the arcs are undirected and weighted by

the distances d between adjacent samples. The spanning tree is
optimum in the sense that the sum of its arc weights is minimum
as compared to any other spanning tree in the complete graph.
In the MST, every pair of samples is connected by a single path,
which is optimum according to fmax . Hence, the MST contains
one OPT for any selected root node.

The optimum prototypes are the closest elements of the MST
with different labels in D1 [i.e., elements that fall in the frontier
of the classes, as displayed in Fig. 1(c)]. By removing the arcs
between different classes, their adjacent samples become pro-
totypes in P ∗, and the OPF algorithm can define an OPF with
minimum classification errors in D1 [see Fig. 1(d)].

2) Classification: For any sample t ∈ D2 , one shall consider
all arcs connecting t with samples s ∈ D1 , as though t were part
of the training graph [see Fig. 1(d)]. Considering all possible
paths from P ∗ to t, one finds the optimum path F ∗(t) from P ∗

and label t with the class λ(R(t)) of its most strongly connected
prototype R(t) ∈ P ∗. This path can be identified incrementally,
by evaluating the optimum cost C(t) as

C(t) = min{max{C(s), d(s, t)}}, ∀s ∈ D1 . (2)



1528 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

Let the node s∗ ∈ D1 be the one that satisfies (2) (i.e.,
the predecessor F (t) in the optimum path F ∗(t)). Given that
L(s∗) = λ(R(t)), the classification simply assigns L(s∗) as the
class of t [see Fig. 1(e)]. An error occurs when L(s∗) 
= λ(t).

B. Bayesian Classifier

Although Naı̈ve-Bayes is a well-known pattern recognition
technique in the literature, we decided to leave some main back-
ground related to its formulation here. In this context, suppose
the elements of the feature vector of each sample are condi-
tionally independent of each other given the classification pro-
cess. Roughly speaking, this means that given a certain class
yi , i = 1, 2, . . . , c, the values of different features do not affect
others.

Let us consider sample x once again. Since Naiı̈ve-Bayes
is a conditional-dependent model, one needs to compute the
posterior distributions P (yi |x), ∀i, as follows:

P (yi |x) =
P (yi)P (x|yi)

p(x)
(3)

where P (ωi) is the so-called prior probability. Since the denom-
inator does not depend on ωi , one can leave it behind to obtain
a more compact representation of the posterior distribution as
follows:

P (yi |x) = P (yi)P (x|yi). (4)

By assuming the conditional independence of the features,
one has a new formulation for the posterior distribution:

P (yi |x1 , x2 , . . . , xn ) = P (yi)
n∏

i=1

P (xi |yi) (5)

where xi stands for the ith feature of sample x. Finally, Naı̈ve-
Bayes assigns the label yj that maximizes the above equation.
The term “naı̈ve” comes from the fact we are assuming the
features are independent of each other.

III. SEQUENTIAL LEARNING

As aforementioned in Section I, the main idea of sequential
learning-based techniques is to model the contextual informa-
tion by means of a two-step classification process, in which a
traditional learning is conducted in the first phase, followed by
the extension of the feature vectors by the predicted labels of
a given sample’s neighborhood. After that, a new classification
process is performed to refine the prior results. Roughly speak-
ing, the methods that rely on such a context differ primarily with
respect to the function that models each sample’s neighborhood.

Basically, the SSL [10] and the sliding window [32] are very
similar to each other: while the former extends the feature vec-
tor of a given sample with the labels of its neighborhood, the
latter approach employs the feature vectors of the neighboring
samples for such purpose. The multiscale sequential learning
proposed by Gatta et al. [11] considers two different roads to
compute a given sample’s neighborhood: multiresolution- and
pyramid-based decompositions. The first approach attempts at
convolving the first-step-labeled image into a series of Gaus-
sian kernels with different variances to simulate distinct scales;

thus, the feature vector is extended by concatenating the sam-
ple’s neighborhood at each scale. The pyramid decomposition
also takes the Gaussian-convoluted images, but now the image
is scaled down for further feature vector extension. The next
sections describe in more details such procedures.

A. Standard SSL

The SSL is a metalearning procedure that has two basic steps:
1) a base classifier and 2) the classification of the extend fea-
tures. Given a dataset S = {(x1 , y1), (x2 , y2) . . . , (xm , ym )}
composed of m pairs of samples (xi , yi), where xi ∈ Rn rep-
resents a feature vector and yi ∈ {1, 2, . . . , c} denotes the class
of xi , the first step employs a base classifier f trained over a
subset ST r ⊂ S, in which c stands for the number of classes.
The next step extends the feature vector xi to a new vector x̃i

containing more features. The new feature vector x̃i contains
all features of xi , the output of the classifier f(xi), and the
output of the base classifier of the neighborhood samples xj ,
∀xj ∈ Ωi , where Ωi represents the neighborhood samples of
xi , j 
= i. Thus, x̃i = xi ∪ ŷi ∪ ŷj1 ∪ ŷj2 ∪ . . . ∪ ŷju

, where u
denotes the neighborhood size of xi , ŷi is the output of the
classifier considering sample xi , i.e., ŷi = f(xi), and ŷju

is the
classification output considering xju

, i.e., ŷju
= f(xju

). The
second step aims at training a new classifier f̃(.) using the ex-
tended feature vectors x̃i . The final output for the classification
of xi is given by ỹi = f̃(x̃i). The first and second steps of the
standard SSL paradigm are presented in Fig. 2.

B. Multiscale SSL

The multiresolution stacked sequential learning (MR-
SSL) [11] is based on the multiresolution theory widely used
in image-processing-oriented applications. The basic difference
between SSL and MR-SSL is the neighborhood sampling pro-
cess: MR-SSL samples feature vectors using different scales on
the dataset S.

Let ρi(xj ) be the probability of the sampled position xj be-
long to the class i = {1, 2, . . . , c}. Therefore, a multiresolution
decomposition MR is defined as follows:

Mi
R (xj , s) = ρi(xj ) ∗ G(0, δs−1) (6)

where s = {1, 2, . . . , S} is the scale of the multiresolution de-
composition, ∗ denotes the convolution operator, G is a multi-
dimensional Gaussian distribution with zero mean and variance
σ = δs−1 , and δ is the step of the multiscale decomposition.1

After that, it is necessary to define the neighborhood sampling
methodology, i.e., Ω. First, one should set the displacement vec-
tors �d = {�d1 , �d2 , . . . , �d9} as follows:

�d1 = (−1,−1) �d2 = (0,−1) �d3 = (1,−1)

�d4 = (−1, 0) �d5 = (0, 0) �d6 = (1, 0)

�d7 = (1,−1) �d6 = (0, 1) �d9 = (1, 1).

1In hard classification techniques, i.e., the ones that output the label only,
ρi (xj ) ∈ {0, 1}.



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1529

Fig. 2. Schematic diagram for the (a) first and (b) second steps of the aforementioned SSL paradigm.

In regard to MR-SSL, x̃i is obtained as follows:

x̃i = {xi ,MR (xi + �d1 , 1), . . . ,MR (xi + �d9 , 1)

× MR (xi + δ�d1 , 2), . . . , MR (xi + δ�d9 , 2)

× . . . . . . MR (xi + δS−1 �d1 , S), . . . ,

× MR (xi + δS−1 �d9 , S)}. (7)

Finally, MR (xi , s) is defined as follows:

MR (xi , s) = arg
j

max{Mj
R (xi , s)}. (8)

C. Pyramidal Stacked Sequential Learning

The pyramidal stacked sequential learning (PY-SSL) is a sim-
ple modification of the MR-SSL, and it differs in the sampling
process only. The pyramidal decomposition can be obtained by

Mi
PYR(xj , s) = Mi

R (�kssxj�, s) (9)

where ks is the sampling step defined as ks = δs/2.
The PY-SSL approach to obtain the extended feature vector

x̃i from xi is given by

x̃i = {xi ,MPYR(xi + �d1 , 1), . . . , MPYR(xi + �d9 , 1)

× MPYR(�xi/δ� + �d1 , 2), . . . , MPYR(�xi/δ� + �d9 , 2)

× . . . . . . MPYR(�xi/δS−1� + �d1 , S), . . . ,

× MPYR(�xi/δS−1� + �d9 , S)}. (10)

In order to clarify the main differences between MR-SSL and
PY-SSL representations, Fig. 3 depicts the sampling process
used when creating the neighborhood of a given sample for
feature vector extension purposes. That pictorial representation
was based on the work by Gatta et al. [11].

IV. PROPOSED ENSEMBLE-BASED SSL

In this section, two modifications of the standard SSL
paradigm that can considerably improve the accuracy results
are presented. The main difference between our approach and
the classical SSL methods concerns the number of base clas-
sifiers. While the classical SSL uses a unique classifier in the
first step, the proposed approach employs a variable number k
of classifiers, each one trained on a partition S i

Tr of the training
set STr , such that STr = S1

Tr ∪ S2
Tr ∪ · · · ∪ Sk

Tr . Based on such
an assumption, two different approaches to extend the feature

Fig. 3. Sampling the neighborhood by means of MR-SSL and PY-SSL ap-
proaches at different scales s = {1, 2, 3}.

vector xi are proposed: 1) concatenated and 2) voting. The mo-
tivation for such ensemble-based approaches comes from the
following assumption: if the base classifier is prone to errors,
it will degrade the learning process of the second one, since
it propagates the wrong labels to the nearby samples. Conse-
quently, such misclassified samples will corrupt the extended
feature vector of a given sample, leading to even worse classifi-
cation results. The next sections present more details about the
proposed approaches.

A. Concatenated SSL

The concatenated stacked sequential learning (CN-SSL) ex-
tends the feature vector xi to a new vector x̃i using the output
of each classifier fz for both xi and also for each sample that
falls in its neighborhood Ωi , z ∈ {1, 2, . . . k}. Therefore, the
idea is to use the output of each classifier fz trained over the
partition Sz

Tr to augment the original feature vector xi from
n features to n + k + ku features, where n corresponds to the
number of features extracted from xi , and u stands for its neigh-
borhood size Ωi . In short, we have that x̃i = xi ∪ ŷ1

i ∪ ŷ2
i ∪

· · · ∪ ŷk
i ∪ ŷ1

j1 ∪ ŷ2
j1 ∪ · · · ∪ ŷk

j1 ∪ ŷ1
j2 ∪ ŷ2

j2 ∪ · · · ∪ ŷk
ju , where

ŷz
i = fz (xi), i.e., the output of classifier fz (·) with respect to

the sample xi . The next step (second classification) is performed
as usual. Fig. 4 depicts the pipeline of CN-SSL.

The main idea of CN-SSL is to alleviate possible misclassifi-
cations by increasing the dimensionality of the feature vector xi .
Suppose a situation in which a classifier fz assigns the wrong



1530 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

Fig. 4. Schematic diagram of CN-SSL.

Fig. 5. Schematic diagram of VO-SSL.

label to some or even all neighbors of a sample xi , and the re-
maining classifiers fv perform a correct classification, ∀v 
= z.
Since the misclassified neighbors will take part of u dimensions
only (x̃i ∈ Rn+k+ku ), they may not contribute a lot to corrupt
x̃i .

B. Voting SSL

The voting stacked sequential learning (VO-SSL) extends
the feature vector xi to a new vector x̃i using the majority of
the output considering the k base classifiers. In the case of a
draw, the algorithm always returns the class with lower index.
In this variant, the extended feature vector x̃i has n + 1 + u
dimensions (features), as follows: x̃i = xi ∪ ỹi ∪ ỹj1 ∪ ỹj2 ∪
· · · ∪ ỹju , where ỹi stands for the majority voting considering
the classifiers in {ŷ1

i , ŷ2
i , . . . , ŷk

i }. Fig. 5 depicts the working
mechanism of VO-SSL.

C. Variations

In order to evaluate the robustness of the proposed ap-
proaches, they were combined with standard SSL, MR-SSL,
and PY-SSL, thus resulting in six new methods, as detailed
below:

1) CN-SSL: Concatenated SSL.
2) VO-SSL: Voting SSL.
3) CN-MR-SSL: CN-SSL applied to MR-SSL.
4) VO-MR-SSL: VO-SSL applied to MR-SSL.
5) CN-PY-SSL: CN-SSL applied to PY-SSL.
6) VO-PY-SSL: VO-SSL applied to PY-SSL.
The rationale in combining the proposed approaches with

others is to show that one can enhance them by simply adding
the ensemble-based tool into their working mechanism.

V. METHODOLOGY

In this section, the methodology employed to validate the
proposed approaches in the context of land-cover classification
is presented. In regard to the experiments, images obtained from
CBERS-2B and Landsat 5 TM covering the area of Itatinga, SP,

Fig. 6. Satellite images used in the experiments: covering the area of Itatinga,
SP, Brazil by (a) CBERS-2B CCD (20 m) sensor (R2G3B4) and (b) Landsat 5
TM (30 m) sensor (R4G3B5), and covering the area of Duque de Caxias, RJ,
Brazil by (c) Ikonos-2 MS sensor (R4G3B2) and (d) Geoeye sensor (R5G4B3).
The CBERS-2B and Landsat 5 TM images have 526 × 492 pixels, and Ikonos-2
MS and Geoeye images have 258 × 250 and 268 × 250 pixels, respectively.
Notice that Ikonos-2 MS and Geoeye images were obtained through a fusion
process between the corresponding images from MS (4 m) and PAN (1 m) sen-
sors using the pan-sharpening method. The final image has a spatial resolution
of 1 m.

Brazil, and other images are obtained from Ikonos-2 MS and
Geoeye covering the area of Duque de Caxias, RJ, Brazil [29]
were used. Fig. 6 displays these images, being their respective
ground truth versions illustrated in Fig. 7. Additionally, Table I
presents the description of the land-cover classes for each im-
age, and Tables II and III present the number of samples per



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1531

Fig. 7. Labeled images used in the experiments: (a) and (b) refer to the
images displayed in Fig. 6(a) and (b), respectively, and (c) and (d) stand for
images displayed in Fig. 6(c) and (d), respectively.

TABLE I
LAND-COVER DESCRIPTION FOR EACH COVERING AREA

Covering area Sensor # land-cover
classes

Land use classes

Itatinga CBERS-2B 6 grass lands, reforesting,
cultures, roads,
dams and bushes

Itatinga Landsat 5 TM 6 grass lands, reforesting,
cultures, roads,
dams and bushes

Duque de Caxias Ikonos-2 MS 8 roads, grass lands,
bare soil moist, covering of tree,
covering of clear tonality,
covering of dark tonality,
bare soil clear and shadows

Duque de Caxias Geoeye 9 roads, grass lands
bare soil moist, covering of tree,
covering of clear tonality,
covering of average tonality,
covering of dark tonality,
bare soil clear and shadows

TABLE II
NUMBER OF SAMPLES FOR THE COVERING AREA OF ITATINGA

Label CBERS-2B Landsat 5 TM

culture 18 407 13 356
bushes 13 779 12 387
dams 142 123
grass lands 14 794 17 567
reforesting 18 825 22 581
roads 803 736

TABLE III
NUMBER OF SAMPLES FOR THE COVERING AREA OF DUQUE DE CAXIAS

Label Ikonos-2 MS Geoeye

covering of trees 5914 6132
shadows 6481 2822
grass lands 12 054 19 370
covering of dark tonality 3578 5073
roads 22 871 22 924
bare soil moist 4417 2380
bare soil clear 7400 4490
covering of clear tonality 1785 1026
covering of average tonality - 2783

TABLE IV
COLORS ASSOCIATED WITH EACH LAND-COVER CLASS

land-cover class.2 Finally, Table IV presents the color map used
to obtain the ground-truth images displayed in Fig. 7.

The standard OPF3 classifier was compared against nine
different sequential learning approaches: OPF with standard
stacked sequential learning [10] (OPF-SSL), OPF with multi-
scale sequential learning and multiresolution-based decompo-
sition (OPF-MR-SSL) [11], OPF with concatenated SSL (OPF-
CN-SSL) and voting SSL (OPF-VO-SSL), OPF with multi-scale
sequential learning and pyramid-based decomposition (OPF-
PY-SSL) [11], OPF-MR-SSL combined with concatenated SSL
(OPF-CN-MR-SSL) and voting SSL (OPF-VO-MR-SSL), and
OPF-PY-SSL combined with concatenated SSL (OPF-CN-PY-

2Images are available at http://wwwp.fc.unesp.br/˜papa/recogna/remote_
sensing.html

3We employed the LibOPF [33], which is an open-source library to the design
of OPF-based classifiers.



1532 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

Fig. 8. Experimental methodology adopted in the work.

TABLE V
EXPERIMENTAL RESULTS REGARDING CBERS-2B IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH THREE, FIVE, AND SEVEN

BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

OPF 66.8 ± 2.1 65.8 ± 2.2 65.2 ± 2.8
OPF-SSL 71.9 ± 0.6 71.6 ± 0.4 72.2 ± 0.4
OPF-VO-SSL 68.3 ± 0.1 68.7 ± 0.0 68.8 ± 0.0 67.9 ± 0.0 68.0 ± 0.0 67.9 ± 0.0 67.3 ± 0.1 66.9 ± 0.0 66.9 ± 0.0
OPF-CN-SSL 73.5 ± 0.0 74.3 ± 0.0 74.6 ± 0.0 74.0 ± 0.0 74.7 ± 0.0 74.9 ± 0.0 74.3 ± 0.0 75.0 ± 0.0 75.5 ± 0.0
OPF-MR-SSL 70.6 ± 0.5 70.2 ± 0.4 70.5 ± 0.6
OPF-VO-MR-SSL 73.0 ± 0.0 71.2 ± 0.0 71.4 ± 0.0 71.1 ± 0.1 70.6 ± 0.0 70.7 ± 0.0 72.4 ± 0.0 70.8 ± 0.0 70.8 ± 0.0
OPF-CN-MR-SSL 72.6 ± 0.0 72.8 ± 0.0 73.0 ± 0.0 73.2 ± 0.0 73.3 ± 0.0 73.7 ± 0.0 73.5 ± 0.0 74.0 ± 0.0 74.2 ± 0.1
OPF-PY-SSL 68.7 ± 0.5 68.9 ± 0.4 69.5 ± 0.4
OPF-VO-PY-SSL 69.1 ± 0.3 68.6 ± 0.3 68.8 ± 0.5 68.8 ± 0.5 67.6 ± 0.2 67.7 ± 0.5 66.6 ± 0.0 66.8 ± 0.2 67.4 ± 0.0
OPF-CN-PY-SSL 72.8 ± 0.1 73.6 ± 0.1 74.2 ± 0.2 73.6 ± 0.1 74.1 ± 0.2 74.6 ± 0.3 74.1 ± 0.1 74.6 ± 0.2 75.1 ± 0.1
Bayes 72.0 ± 0.0 72.2 ± 0.1 71.0 ± 0.0
Bayes-SSL 66.5 ± 0.0 66.9 ± 0.0 68.0 ± 0.0
Bayes-VO-SSL 72.5 ± 0.0 72.4 ± 0.0 72.2 ± 0.0 72.5 ± 0.0 72.5 ± 0.0 72.5 ± 0.0 72.5 ± 0.0 72.7 ± 0.0 72.7 ± 0.0
Bayes-CN-SSL 74.8 ± 0.1 75.2 ± 0.0 75.3 ± 0.1 75.4 ± 0.1 75.6 ± 0.1 75.8 ± 0.1 75.9 ± 0.1 76.2 ± 0.1 76.4 ± 0.1
Bayes-MR-SSL 64.7 ± 0.0 66.9 ± 0.1 68.0 ± 0.0
Bayes-VO-MR-SSL 73.0 ± 0.0 72.9 ± 0.0 72.9 ± 0.0 73.1 ± 0.0 73.1 ± 0.0 73.0 ± 0.0 73.4 ± 0.0 73.6 ± 0.0 73.6 ± 0.0
Bayes-CN-MR-SSL 73.5 ± 0.0 73.7 ± 0.0 73.5 ± 0.0 74.1 ± 0.0 74.2 ± 0.0 74.4 ± 0.0 75.0 ± 0.0 75.1 ± 0.0 75.3 ± 0.0
Bayes-PY-SSL 64.9 ± 0.1 66.8 ± 0.1 68.3 ± 0.2
Bayes-VO-PY-SSL 72.8 ± 0.1 72.6 ± 0.2 72.7 ± 0.0 72.6 ± 0.1 72.7 ± 0.0 72.7 ± 0.0 72.9 ± 0.2 73.0 ± 0.0 72.7 ± 0.0
Bayes-CN-PY-SSL 74.3 ± 0.1 74.7 ± 0.1 74.9 ± 0.2 75.5 ± 0.0 75.2 ± 0.2 75.4 ± 0.0 75.5 ± 0.0 75.9 ± 0.2 76.6 ± 0.1

SSL) and voting SSL (OPF-VO-PY-SSL). In addition, we imple-
mented the very same proposed approaches considering naı̈ve
Bayes classifier in order to show that the proposed approaches
can also obtain very good results when applied to other tech-
niques. Following the above nomenclature, one has Bayes (stan-
dard Bayesian classifier), Bayes-SSL (Bayesian classifier with
standard SSL), Bayes-MR-SSL (Bayes with multi-scale se-
quential learning and multi-resolution-based decomposition),
Bayes with concatenated SSL (Bayes-CN-SSL) and voting
SSL (Bayes-VO-SSL), Bayes with multi-scale sequential learn-
ing and pyramid-based decomposition (Bayes-PY-SSL) [11],
Bayes-MR-SSL combined with concatenated SSL (Bayes-CN-

MR-SSL) and voting SSL (Bayes-VO-MR-SSL), and Bayes-
PY-SSL combined with concatenated SSL (Bayes-CN-PY-SSL)
and voting SSL (Bayes-VO-PY-SSL). Although one can use any
pattern recognition technique, the proposed approach was vali-
dated over OPF and naı̈ve Bayes classifier, since both techniques
are parameterless and do not require a considerable computa-
tional load.4 Additionally, the main motivation for ensemble-
based SSL is related to OPF classifier, since such graph-based
approach has been consistently more accurate than SVM in
several applications, but being faster for training patterns. An

4In regard to Bayesian classifier, our own implementation has been employed.



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1533

Fig. 9. CBERS-2B images classified using: (a) OPF (10%), (b) Bayes (10%), (c) OPF-CN-SSL (5% and seven base classifiers), (d) Bayes-CN-SSL (5% and
seven base classifiers), (e) OPF-CN-SSL (10% and seven base classifiers), (f) Bayes-CN-SSL (10% and seven base classifiers), (g) OPF-CN-SSL (20% and seven
base classifiers), and (h) Bayes-CN-SSL (20% and seven base classifiers).

extra round of experiment was conducted in order to check the
robustness of the proposed approaches when one uses different
classifiers for each step (this last experiment is called “Hybrid”).
Fig. 8 displays the experimental evaluation procedure adopted
in this paper.

The influence of different training set sizes with 5%, 10%,
and 20% of the entire image was evaluated, being the remaining
pixels used to compose the test set. Additionally, the influence in
using a different number of base classifiers was also considered.
For such purpose, we use k ∈ {3, 5, 7} classifiers for each the
training set sizes. In order to allow a robust statistical evaluation,
a cross-validation with ten runnings for the further computation
of the Wilcoxon signed-rank test [34] over the accuracy rates
was performed.5 Additionally, each pixel has been described by
its RGB values to compose the dataset samples.6 Finally, the
experiments were conducted on a personal computer equipped
with an Intel Xeon CPU E5-2603 1.60-GHz processor, 16 GB
of RAM and Ubuntu 14.04 LTS as the operational system.

VI. EXPERIMENTS

In this section, the experimental results regarding sequen-
tial learning-based OPF classification using nine different ap-
proaches were presented, including the six new variations con-
sidered in this paper.

5We employed an accuracy measure proposed by Papa et al. [25] that con-
siders unbalanced datasets, which is often faced in land-cover classification.

6In this work, we used an eight-neighborhood system for OPF-SSL, and an
11- and a three-neighborhood systems for OPF-MSSL-MR and OPF-MSSL-PY,
respectively. We also employed seven scales of decomposition for OPF-MSSL-
MR, and five scales of decomposition for OPF-MSSL-PY. Such values have
been empirically set.

A. CBERS-2B Image

Table V presents the mean accuracy results with respect to
the CBERS-2B image displayed in Fig. 6(a). The most accurate
techniques considering the Wilcoxon signed-rank test are in
bold.7 Considering such results, one can draw some interesting
conclusions.

1) Standard OPF accuracy does not improve when one in-
creases the training set size, which means there is no
guarantee we are always adding good samples for train-
ing purposes (Bayes performance gets better slightly
with 10% for training, but it drops again when one
uses 20%).

2) Both OPF and Bayes techniques do not have substan-
tially more accurate results using SSL when one increases
the training set size, which corroborates our assumption
that misclassified samples do not help sequential learning
techniques, thus making them even worse.

3) The configuration with seven base classifiers worked bet-
ter for almost all pairs of classifiers and training set sizes,
which makes sense since we use more classifiers to com-
pose the ensemble, and thus, more specialists are consid-
ered into the decision-making process (obviously, there is
a tradeoff between the number of classifiers to compose
the ensemble and the training set size assigned to each of
them).

4) Both the proposed approaches, i.e., CN-SSL and VO-
SSL, improved the base techniques where they have been

7The statistical test is performed for each classifier and amount of training
set (e.g., OPF-CN-SSL with seven base classifiers obtained the best results
considering OPF and using 5% of the data for training purposes).



1534 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

TABLE VI
EXPERIMENTAL RESULTS REGARDING THE “HYBRID” EXPERIMENT OVER CBERS-2B IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES

WITH THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

First step classification using OPF and the second step using Bayes

SSL 70.1 ± 0.6 70.9 ± 0.9 70.1 ± 0.3
VO-SSL 69.7 ± 0.6 71.1 ± 0.0 69.8 ± 0.0 69.0 ± 1.1 68.2 ± 0.0 69.0 ± 0.2 67.7 ± 1.0 66.9 ± 0.4 66.7 ± 0.0
CN-SSL 70.2 ± 0.9 70.0 ± 0.0 68.7 ± 0.0 69.1 ± 1.1 68.2 ± 0.0 70.1 ± 1.0 69.1 ± 0.7 68.5 ± 0.6 66.8 ± 0.0
MR 72.6 ± 1.0 74.6 ± 1.1 74.3 ± 1.2
VO-MR 72.1 ± 0.5 72.7 ± 0.0 70.8 ± 0.0 73.3 ± 1.2 68.2 ± 0.0 68.0 ± 0.0 74.1 ± 1.0 72.8 ± 1.0 69.4 ± 0.0
CN-MR 69.3 ± 0.7 72.4 ± 0.0 71.4 ± 0.0 71.2 ± 1.1 75.2 ± 0.0 75.2 ± 0.0 72.4 ± 0.9 76.1 ± 0.5 76.5 ± 0.2
PYR 70.2 ± 1.0 71.3 ± 0.6 71.2 ± 0.4
VO-PYR 70.2 ± 0.9 70.0 ± 0.0 69.8 ± 0.0 69.1 ± 1.1 68.2 ± 0.0 68.3 ± 0.0 69.1 ± 0.7 68.6 ± 0.6 67.9 ± 0.0
CN-PYR 73.5 ± 0.8 75.4 ± 0.0 69.7 ± 0.0 74.7 ± 1.0 68.2 ± 0.0 67.0 ± 0.0 75.8 ± 0.6 76.7 ± 0.6 71.4 ± 0.2

First step classification using Bayes and the second step using OPF

SSL 68.9 ± 0.3 69.5 ± 0.1 64.0 ± 0.4
VO-SSL 63.8 ± 0.7 63.4 ± 0.0 64.1 ± 0.2 61.8 ± 0.4 62.5 ± 0.0 62.4 ± 0.1 60.9 ± 0.7 60.8 ± 0.6 60.9 ± 0.0
CN-SSL 65.4 ± 1.0 66.4 ± 0.0 65.1 ± 0.2 64.4 ± 0.5 62.5 ± 0.0 66.1 ± 0.0 65.0 ± 0.3 65.3 ± 0.4 63.9 ± 0.2
MR 68.1 ± 0.3 68.9 ± 0.1 68.4 ± 0.2
VO-MR 65.3 ± 1.0 65.1 ± 0.0 65.0 ± 0.0 64.7 ± 0.4 62.5 ± 0.0 62.0 ± 0.3 63.5 ± 0.8 64.3 ± 0.6 62.1 ± 0.1
CN-MR 67.7 ± 0.3 67.9 ± 0.0 66.0 ± 0.0 68.3 ± 0.1 68.5 ± 0.0 68.7 ± 0.0 69.0 ± 0.2 69.1 ± 0.2 67.7 ± 0.0
PYR 68.2 ± 0.3 69.3 ± 0.2 64.5 ± 0.4
VO-PYR 64.3 ± 0.8 63.7 ± 0.0 64.9 ± 0.0 62.0 ± 1.1 62.5 ± 0.0 62.9 ± 0.1 61.4 ± 0.6 61.5 ± 0.4 60.7 ± 0.0
CN-PYR 66.0 ± 0.6 66.2 ± 0.0 65.0 ± 0.0 64.8 ± 0.4 62.5 ± 0.0 66.1 ± 1.1 65.0 ± 0.2 65.2 ± 0.2 63.8 ± 0.1

TABLE VII
MEAN COMPUTATIONAL LOAD IN SECONDS REGARDING CBERS-2B IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH THREE,

FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Classifiers 3 5 7 3 5 7 3 5 7

OPF-SSL 5.3 11.3 29.3
OPF-VO-SSL 3.4 3.5 3.4 6.4 6.2 7.2 11.0 11.7 9.6
OPF-CN-SSL 11.9 12.0 11.4 28.6 26.3 26.2 52.1 52.5 49.7
OPF-MR 40.7 76.8 170.0
OPF-VO-MR 34.0 34.4 31.4 71.9 66.3 78.2 129.5 132.1 119.7
OPF-CN-MR 128.0 201.0 206.2 236.9 368.2 401.1 450.7 646.0 649.3
OPF-PYR 20.7 47.1 104.8
OPF-VO-PYR 11.8 12.4 13.4 26.7 26.3 26.3 51.8 52.6 49.7
OPF-CN-PYR 47.1 48.5 43.4 100.1 106.3 101.4 219.8 208.5 219.6
Bayes-SSL 6.6 11.1 18.0
Bayes-VO-SSL 3.4 3.4 3.3 5.6 5.8 6.8 8.8 9.2 9.0
Bayes-CN-SSL 5.9 6.2 6.3 10.7 12.8 11.5 19.1 23.3 19.0
Bayes-MR 10.3 20.9 47.1
Bayes-VO-MR 10.8 11.0 8.3 20.2 25.8 22.0 43.3 45.8 49.0
Bayes-CN-MR 12.5 13.2 13.3 38.2 38.8 32.3 95.3 96.2 99.0
Bayes-PYR 11.2 43.3 101.5
Bayes-VO-PYR 9.3 9.3 7.3 17.1 15.8 16.4 33.5 35.9 39.0
Bayes-CN-PYR 22.0 21.2 27.3 56.2 58.1 64.3 101.4 108.2 108.1

applied (only one exception was noticed with respect to
OPF-VO-SSL and OPF-SSL for all training set sizes, in
which the former did not outperform the base approach
using SSL).

5) The number of base classifiers seemed to have no good
influence over VO-SSL, since it has obtained the best
results using less classifiers (i.e., 3), being an intuitive
idea about that the fact of having a committee composed
of not so high-skilled specialists;

6) CN-SSL worked better than VO-SSL, which also corrobo-
rates our assumption stated in Section IV-A that says one

can somehow mask the effect of misclassified samples
increasing the dimensionality of the feature space.

Fig. 9 displays some of the best results obtained over CBERS-
2B image, in which the percentage in parenthesis stands for the
amount of training set used. Usually, the results using MR-SSL
and PY-SSL are more accurate than standard SSL, as observed
by Pereira et al. [28]. However, due to the computational load,
we used less scales and smaller neighborhoods, which might
have affected the results of both MR and PY with respect to
SSL. Albeit, the main goal of this work is not related to the
fact that MR-SSL and PY-SSL are better than naı̈ve SSL, since



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1535

TABLE VIII
EXPERIMENTAL RESULTS CONCERNING IKONOS-2 IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH THREE, FIVE, AND SEVEN

BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Classifiers 3 5 7 3 5 7 3 5 7

OPF 69.2 ± 0.1 71.3 ± 0.2 74.3 ± 0.1
OPF-SSL 68.5 ± 0.4 69.9 ± 0.3 72.8 ± 0.3
OPF-VO-SSL 69.0 ± 0.1 68.4 ± 0.2 68.0 ± 0.4 70.0 ± 0.2 70.0 ± 0.4 70.2 ± 0.4 73.8 ± 0.3 73.3 ± 0.3 73.7 ± 0.4
OPF-CN-SSL 68.1 ± 0.2 68.0 ± 0.2 68.4 ± 0.0 69.3 ± 0.1 69.1 + 0.1 69.8 ± 0.2 72.0 ± 0.1 71.4 ± 0.2 71.4 ± 0.2
OPF-MR-SSL 62.4 ± 0.3 64.2 ± 0.2 68.5 ± 0.1
OPF-VO-MR-SSL 69.8 ± 0.2 69.5 ± 0.3 69.3 ± 0.2 71.6 ± 0.2 71.7 ± 0.2 72.5 ± 0.2 77.0 ± 0.1 77.2 ± 0.1 77.1 ± 0.1
OPF-CN-MR-SSL 69.7 ± 0.1 69.6 ± 0.1 69.6 ± 0.1 72.0 ± 0.2 72.3 ± 0.2 72.2 ± 0.2 76.9 ± 0.2 77.1 ± 0.2 77.0 ± 0.2
OPF-PY-SSL 62.4 ± 0.3 63.0 ± 0.0 67.2 ± 0.4
OPF-VO-PY-SSL 69.1 ± 0.2 68.2 ± 0.5 69.5 ± 0.3 71.0 ± 0.2 70.9 ± 0.4 70.5 ± 0.2 75.0 ± 0.5 74.0 ± 0.2 73.8 ± 0.4
OPF-CN-PY-SSL 68.5 ± 0.2 68.5 ± 0.2 69.0 ± 0.3 70.3 ± 0.2 70.5 ± 0.3 70.4 ± 0.2 73.1 ± 0.2 73.1 ± 0.2 73.7 ± 0.2
Bayes 69.0 ± 0.9 70.1 ± 0.1 73.8 ± 0.1
Bayes-SSL 67.2 ± 0.2 65.2 ± 0.1 70.1 ± 0.2
Bayes-VO-SSL 67.8 ± 0.1 67.7 ± 0.2 67.9 ± 0.2 69.8 ± 0.1 69.8 ± 0.1 69.7 ± 0.2 73.5 ± 0.2 73.3 ± 0.2 73.2 ± 0.1
Bayes-CN-SSL 68.0 ± 0.2 67.9 ± 01 68.2 ± 0.1 69.5 ± 0.1 69.3 ± 0.2 69.1 ± 0.1 71.6 ± 0.1 71.2 ± 0.1 71.2 ± 0.1
Bayes-MR-SSL 60.1 ± 0.9 64.2 ± 0.2 68.0 ± 0.0
Bayes-VO-MR-SSL 69.6 ± 0.3 69.5 ± 0.2 69.7 ± 0.2 72.5 ± 0.2 72.3 ± 0.1 72.8 ± 0.1 77.3 ± 0.2 77.3 ± 0.3 77.4 ± 0.3
Bayes-CN-MR-SSL 69.6 ± 0.2 69.8 ± 0.1 69.7 ± 0.2 72.6 ± 0.1 72.7 ± 0.0 72.7 ± 0.1 76.9 ± 0.0 77.6 ± 0.1 77.1 ± 0.4
Bayes-PY-SSL 61.0 ± 0.1 63.9 ± 0.0 69.1 ± 0.0
Bayes-VO-PY-SSL 68.5 ± 0.2 68.0 ± 0.1 68.3 ± 0.1 70.3 ± 0.1 70.0 ± 0.1 70.4 ± 0.1 74.0 ± 0.2 74.6 ± 0.2 74.0 ± 0.2
Bayes-CN-PY-SSL 68.3 ± 0.0 69.0 ± 0.3 69.6 ± 0.2 70.3 ± 0.1 70.4 ± 0.1 70.4 ± 0.1 73.6 ± 0.1 73.6 ± 0.1 73.8 ± 0.1

Fig. 10. Ikonos-2 MS images classified using: (a) OPF (10%), (b) Bayes (10%), (c) OPF-CN-SSL (5% and seven base classifiers), (d) Bayes-CN-SSL (5% and
seven base classifiers), (e) OPF-CN-SSL (10% and seven base classifiers), (f) Bayes-CN-SSL (10% and seven base classifiers), (g) OPF-CN-SSL (20% and seven
base classifiers), and (h) Bayes-CN-SSL (20% and seven base classifiers).

such points have been extensively discussed by Gatta et al. [11],
but showing one can improve the aforementioned techniques
using a committee of classifiers to alleviate the problem of
misclassification.

Table VI presents the results with respect to the “Hybrid”
experiment. From such data, one can draw some conclusions:
1) the results using OPF as the first classifier were considerably

better than the ones using Naı̈ve-Bayes in the first step, which
suggests that OPF achieves better initial results that are propa-
gated to the next stacked classifier for feature extension; and 2)
when using the OPF as the initial learner, the hybrid protocol
obtained better results (for some situations) than using the stan-
dard protocol presented in Table V. Such results strengthen the
validity of the proposed approaches.



1536 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

TABLE IX
EXPERIMENTAL RESULTS REGARDING THE “HYBRID” EXPERIMENT OVER IKONOS-2 IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES

WITH THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Classifiers 3 5 7 3 5 7 3 5 7

First step classification using OPF and the second step using Bayes

SSL 66.6 ± 0.5 71.1 ± 0.6 72.0 ± 1.1
VO-SSL 67.0 ± 0.3 66.5 ± 0.0 67.2 ± 0.0 69.4 ± 0.7 69.4 ± 0.0 70.1 ± 0.0 73.7 ± 0.8 73.6 ± 0.6 74.4 ± 0.0
CN-SSL 66.8 ± 0.2 68.6 ± 0.0 68.1 ± 0.0 69.4 ± 0.1 69.4 ± 0.0 71.5 ± 0.1 73.3 ± 0.5 73.3 ± 0.2 74.4 ± 0.1
MR 69.4 ± 0.4 69.8 ± 0.3 70.1 ± 0.2
VO-MR 62.2 ± 0.6 61.8 ± 0.0 64.2 ± 0.0 65.2 ± 0.3 69.4 ± 0.0 67.2 ± 0.0 68.8 ± 0.4 68.5 ± 0.2 66.4 ± 0.0
CN-MR 71.5 ± 0.1 72.1 ± 0.0 73.0 ± 0.0 76.4 ± 0.4 78.4 ± 0.0 78.0 ± 0.0 81.2 ± 0.4 81.2 ± 0.2 81.1 ± 0.0
PYR 69.8 ± 0.6 75.2 ± 0.8 73.2 ± 0.4
VO-PYR 68.1 ± 0.5 68.6 ± 0.0 67.2 ± 0.0 70.4 ± 0.8 69.4 ± 0.0 71.2 ± 0.1 73.8 ± 0.4 74.2 ± 0.6 74.4 ± 0.0
CN-PYR 71.2 ± 0.6 73.1 ± 0.0 70.2 ± 0.0 74.8 ± 0.1 69.4 ± 0.0 72.5 ± 0.0 80.1 ± 0.4 81.6 ± 0.5 78.1 ± 0.3

First step classification using Bayes and the second step using OPF

SSL 62.8 ± 0.3 66.6 ± 0.2 69.4 ± 0.5
VO-SSL 62.5 ± 0.6 62.3 ± 0.0 62.4 ± 0.2 64.9 ± 0.8 65.5 ± 0.0 66.4 ± 0.0 68.9 ± 0.7 68.2 ± 0.2 68.0 ± 0.0
CN-SSL 62.9 ± 0.5 62.2 ± 0.0 61.8 ± 0.1 63.9 ± 0.2 65.5 ± 0.0 67.0 ± 0.0 66.5 ± 0.4 65.9 ± 0.1 66.1 ± 0.2
MR 63.9 ± 0.1 71.6 ± 0.5 71.2 ± 0.3
VO-MR 64.0 ± 0.4 63.7 ± 0.0 62.2 ± 0.0 66.8 ± 0.4 65.47 ± 0.0 66.8 ± 0.0 71.6 ± 0.4 70.9 ± 0.6 69.1 ± 0.0
CN-MR 63.6 ± 0.2 63.6 ± 0.0 61.4 ± 0.0 66.7 ± 0.3 65.5 ± 0.0 69.2 ± 0.1 71.3 ± 0.2 70.9 ± 0.2 69.8 ± 0.5
PYR 63.3 ± 0.3 71.1 ± 0.3 71.3 ± 0.4
VO-PYR 63.0 ± 0.6 62.6 ± 0.0 61.8 ± 0.2 65.4 ± 0.4 65.5 ± 0.0 67.0 ± 0.1 69.2 ± 0.8 69.2 ± 0.4 68.0 ± 0.0
CN-PYR 63.7 ± 0.1 64.0 ± 0.0 62.3 ± 0.1 66.5 ± 0.4 65.5 ± 0.0 66.7 ± 0.3 71.1 ± 0.2 71.2 ± 0.4 70.4 ± 0.2

TABLE X
MEAN COMPUTATIONAL LOAD IN SECONDS CONCERNING IKONOS-2 IMAGE

USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH

THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Classifiers 3 5 7 3 5 7 3 5 7

OPF-SSL 0.3 1.1 3.2
OPF-VO-SSL 0.3 0.3 0.3 0.6 0.6 0.8 1.1 1.1 1.1
OPF-CN-SSL 0.8 0.5 0.4 1.5 1.2 0.1 4.9 4.5 4.7
OPF-MR 2.7 9.4 22.4
OPF-VO-MR 2.7 2.9 2.3 5.3 5.6 6.1 9.5 10.3 9.0
OPF-CN-MR 7.9 12.8 12.6 14.7 22.0 18.7 24.4 36.8 37.8
OPF-PYR 1.1 4.8 11.9
OPF-VO-PYR 1.0 1.2 1.3 2.1 2.6 2.6 6.8 6.0 6.1
OPF-CN-PYR 3.7 4.0 4.3 7.6 7.6 9.1 13.4 18.7 17.1
Bayes-SSL 0.8 1.6 3.2
Bayes-VO-SSL 0.3 0.3 0.3 0.7 0.7 0.7 1.9 1.9 1.9
Bayes-CN-SSL 0.4 0.5 0.4 0.8 0.8 0.9 2.3 2.6 2.9
Bayes-MR 1.3 3.7 6.2
Bayes-VO-MR 0.8 0.8 0.8 1.4 1.5 1.5 3.5 3.5 3.9
Bayes-CN-MR 1.9 1.0 1.3 3.8 3.5 3.1 9.3 10.9 11.9
Bayes-PYR 1.3 3.7 6.2
Bayes-VO-PYR 0.8 0.7 0.7 1.4 1.5 1.4 2.4 2.5 2.9
Bayes-CN-PYR 0.8 0.8 1.0 1.8 1.9 2.0 3.9 3.8 3.8

Table VII presents the mean computational load in seconds
for each approach considered in this work. One can observe the
voting-based approaches are consistently more efficient than
standard SSL for both OPF and Naı̈ve-Bayes classifiers, since
the original training set is now divided in smaller disjoint subsets
to train each classifier in the ensemble. However, the complexity
grows when one uses the concatenated-oriented approach, since
one has larger feature vectors, which end up impacting in the
computational burden when computing the Euclidean distance
among feature vectors. OPF can handle that problem by using a

precomputed distance matrix, but at the price of requiring more
memory space to store such distances.

B. Ikonos-2 MS Image

Table VIII states the results considering Ikonos-2 MS image,
being their presentation the very same one used in Section VI-A,
i.e., the best results according to Wilcoxon signed-rank test for
each pair of classifier and training set size are in bold. Con-
sidering this image, we obtained some results that are slightly
different from the previous ones (see Section VI-A). First, we
observed the accuracy of both OPF and Bayes classifiers, as
well as their SSL-based versions increased with larger training
sets (only one exception with respect to Bayes-SSL with 10%
of the data for training purposes). Second, the standard SSL ver-
sions did not improve the results of traditional OPF and Bayes
techniques, probably due to large homogeneous regions, thus
making no sense the application of standard sequential learn-
ing approaches, since when taking the neighborhood of a given
sample, there is a high probability of most part of the pixels
that fall in that neighborhood have the very same label of that
sample. However, the multiresolution SSL-oriented techniques
obtained the best results for both OPF and Bayes (i.e., MR-
SSL). Actually, the proposed approaches enhanced even more
the recognition rates of MR-SSL techniques, being 13.89% more
accurate for some cases (e.g., Bayes-CN-MR-SSL using 5% of
the dataset for the training set and five classifiers to compose
the ensemble).

Fig. 10 displays some of the best results obtained over Ikonos-
2 MS image, in which the percentage in parenthesis stands
for the amount of training set used. Except for standard SSL,
the proposed approaches obtained much more accurate results
in all possible configurations, i.e., considering the classifier,



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1537

TABLE XI
EXPERIMENTAL RESULTS REGARDING LANDSAT 5 TM IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH THREE, FIVE, AND

SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

OPF 63.2 ± 0.4 63.1 ± 0.5 61.9 ± 0.2
OPF-SSL 69.2 ± 0.1 70.0 ± 0.0 71.3 ± 0.1
OPF-VO-SSL 67.5 ± 0.6 67.7 ± 0.6 66.5 ± 0.2 66.8 ± 0.6 66.5 ± 0.5 65.8 ± 0.7 64.0 ± 0.4 64.3 ± 0.3 63.9 ± 0.5
OPF-CN-SSL 71.6 ± 0.2 72.9 ± 0.5 73.3 ± 0.5 72.2 ± 0.1 73.0 ± 0.2 74.0 ± 0.4 72.6 ± 0.1 73.4 ± 0.1 73.5 ± 0.2
OPF-MR-SSL 69.2 ± 0.1 69.7 ± 0.0 71.1 ± 0.1
OPF-VO-MR-SSL 66.5 ± 0.2 65.4 ± 0.3 65.8 ± 0.5 66.1 ± 0.1 65.4 ± 0.3 66.2 ± 0.2 65.2 ± 0.2 65.3 + 0.1 65.3 ± 0.1
OPF-CN-MR-SSL 71.1 ± 0.1 71.3 ± 0.1 71.5 ± 0.1 71.4 ± 0.1 71.5 ± 0.1 71.9 ± 0.1 72.0 ± 0.1 72.1 ± 0.1 72.1 ± 0.1
OPF-PY-SSL 69.2 ± 0.1 69.0 ± 0.0 70.1 ± 0.2
OPF-VO-PY-SSL 69.4 ± 0.5 68.0 ± 0.4 68.4 ± 0.1 68.0 ± 0.1 66.0 ± 0.5 67.5 ± 0.1 65.8 ± 0.4 65.1 ± 0.5 63.4 ± 0.5
OPF-CN-PY-SSL 72.4 ± 0.4 74.6 ± 0.4 75.7 ± 0.6 72.6 ± 0.3 74.9 ± 0.6 75.4 ± 0.4 73.4 ± 0.3 74.2 ± 0.5 74.1 ± 0.5
Bayes 69.6 ± 0.1 69.0 ± 0.2 69.0 ± 0.2
Bayes-SSL 72.9 ± 0.0 73.4 ± 0.1 74.5 ± 0.3
Bayes-VO-SSL 71.9 ± 0.7 72.0 ± 0.6 71.9 ± 0.4 71.5 ± 0.3 71.6 ± 0.4 71.6 ± 0.5 69.9 ± 0.4 70.9 ± 0.6 70.6 ± 0.5
Bayes-CN-SSL 72.7 ± 0.3 73.5 ± 0.4 73.5 ± 0.2 73.1 ± 0.1 73.4 ± 0.1 74.2 ± 0.3 73.2 ± 0.1 73.7 ± 0.1 73.9 ± 0.1
Bayes-MR-SSL 70.6 ± 0.1 71.4 ± 0.0 73.0 ± 0.0
Bayes-VO-MR-SSL 70.9 ± 0.3 70.9 ± 0.1 70.9 ± 0.3 71.0 ± 0.0 70.8 ± 0.1 71.0 ± 0.1 71.0 ± 0.1 70.7 ± 0.2 70.6 ± 0.2
Bayes-CN-MR-SSL 71.6 ± 0.2 71.3 ± 0.1 71.7 ± 0.1 71.6 ± 0.2 71.6 ± 0.4 71.9 ± 0.1 72.2 ± 0.1 72.2 ± 0.1 72.3 ± 0.1
Bayes-PY-SSL 71.3 ± 0.1 72.5 ± 0.2 73.2 ± 0.0
Bayes-VO-PY-SSL 72.9 ± 0.1 73.8 ± 0.3 72.6 ± 0.7 72.6 ± 0.2 71.4 ± 0.5 73.4 ± 0.7 71.1 ± 0.5 71.3 ± 0.5 71.1 ± 0.3
Bayes-CN-PY-SSL 74.1 ± 0.5 75.3 ± 0.6 75.7 ± 0.5 73.9 ± 0.6 75.3 ± 0.3 75.4 ± 0.5 75.0 ± 0.4 75.2 ± 0.6 75.8 ± 0.6

TABLE XII
EXPERIMENTAL RESULTS REGARDING THE “HYBRID” EXPERIMENT OVER LANDSAT 5 TM IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING

PURPOSES WITH THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

First step classification using OPF and the second step using Bayes

SSL 68.7 ± 0.9 69.2 ± 0.5 72.1 ± 0.3
VO-SSL 65.4 ± 1.1 66.7 ± 0.0 68.3 ± 0.0 63.4 ± 0.6 64.4 ± 0.0 67.4 ± 0.5 61.8 ± 0.9 62.9 ± 1.1 61.6 ± 0.0
CN-SSL 65.0 ± 1.0 68.3 ± 0.0 68.4 ± 0.0 64.8 ± 1.5 64.4 ± 0.0 67.0 ± 0.2 63.4 ± 0.9 63.4 ± 1.0 62.3 ± 0.1
MR 65.1 ± 1.9 63.7 ± 2.1 64.8 ± 1.0
VO-MR 64.4 ± 1.0 64.3 ± 0.0 66.2 ± 0.0 72.9 ± 3.5 71.4 ± 0.0 70.4 ± 0.0 73.1 ± 0.8 71.5 ± 2.7 69.6 ± 0.0
CN-MR 70.7 ± 1.9 71.1 ± 0.0 73.5 ± 0.0 72.7 ± 0.7 74.0 ± 0.0 75.4 ± 0.3 80.1 ± 1.1 73.6 ± 1.2 73.6 ± 0.0
PYR 69.1 ± 1.2 69.1 ± 0.7 69.8 ± 2.1
VO-PYR 65.0 ± 1.0 68.3 ± 0.0 68.1 ± 0.0 64.8 ± 1.5 64.4 ± 0.0 64.8 ± 0.2 63.4 ± 0.9 63.4 ± 1.0 62.9 ± 0.0
CN-PYR 62.4 ± 0.8 63.3 ± 0.0 66.3 ± 0.0 63.3 ± 1.1 64.4 ± 0.0 70.1 ± 0.1 73.2 ± 1.1 74.4 ± 0.9 71.6 ± 0.5

First step classification using Bayes and the second step using OPF

SSL 66.9 ± 0.2 67.1 ± 0.2 63.1 ± 0.6
VO-SSL 60.9 ± 1.1 58.8 ± 0.0 60.4 ± 0.0 58.9 ± 0.8 58.8 ± 0.0 58.0 ± 0.0 57.1 ± 1.0 57.6 ± 0.9 56.8 ± 0.0
CN-SSL 65.2 ± 0.9 66.6 ± 0.0 65.4 ± 0.0 64.7 ± 0.6 58.8 ± 0.0 61.0 ± 0.0 65.0 ± 0.3 65.8 ± 0.8 63.8 ± 0.2
MR 65.9 ± 0.4 66.6 ± 0.3 66.6 ± 0.1
VO-MR 60.6 ± 0.1 60.1 ± 0.0 64.6 ± 0.0 59.8 ± 0.6 58.8 ± 0.0 62.8 ± 0.7 58.7 ± 0.6 59.0 ± 0.7 58.1 ± 0.1
CN-MR 66.3 ± 0.2 65.8 ± 0.0 64.9 ± 0.0 66.4 ± 0.1 58.8 ± 1.0 69.5 ± 0.9 67.0 ± 0.1 66.9 ± 0.2 67.8 ± 0.1
PYR 66.1 ± 0.1 66.8 ± 0.2 64.9 ± 0.4
VO-PYR 60.3 ± 0.9 60.9 ± 0.0 60.6 ± 0.0 58.2 ± 0.4 58.8 ± 0.0 60.0 ± 0.0 57.4 ± 0.6 57.5 ± 0.8 56.8 ± 0.0
CN-PYR 65.7 ± 0.3 66.0 ± 0.0 65.5 ± 0.0 65.6 ± 0.5 62.8 ± 0.0 67.0 ± 0.8 66.1 ± 0.5 65.9 ± 0.2 65.8 ± 0.0

percentage of training set, and the sequential learning base
method. Table IX presents the results concerning the “Hybrid”
protocol. Once again, better results than the “Standard” one
were obtained, being the OPF classifier the best choice for the
initial classification. For example, using 10% of the data for
training purposes, the “Hybrid” protocol obtained 78.4% using
five classifiers, while the “Standard” protocol achieved 72.5%
using seven classifiers.

Table X presents the mean computational load in sec-
onds concerning Ikonos-2 image. Once again, the voting-based

approaches are considerably faster than standard SSL; mean-
while, concatenation-driven ones require more computational
burden. Since the pyramid decomposition reduces the image
resolution, it requires less computational load when compared
against the multiresolution approach.

C. Landsat 5 TM and Geoeye Images

In this section, a brief discussion the experiments obtained
over Landsat 5 TM and Geoeye images is considered, since



1538 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

TABLE XIII
MEAN COMPUTATIONAL LOAD IN SECONDS REGARDING LANDSAT 5 TM IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES

WITH THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

OPF-SSL 5.2 15.0 32.0
OPF-VO-SSL 4.1 4.1 4.1 7.3 7.8 7.3 15.7 13.4 11.7
OPF-CN-SSL 13.0 12.9 14.1 31.5 27.9 31.1 55.1 53.5 51.6
OPF-MR 47.9 105.1 217.2
OPF-VO-MR 31.2 31.3 34.1 63.3 67.8 65.1 109.2 110.3 111.6
OPF-CN-MR 118.5 184.6 188.2 324.7 357.1 322.1 631.3 628.7 634.4
OPF-PYR 22.1 55.9 101.3
OPF-CN-PYR 44.0 49.3 44.1 95.7 89.1 91.0 181.1 218.0 211.7
OPF-VO-PYR 12.8 12.8 14.1 31.8 28.0 29.6 56.4 53.4 61.6
Bayes-SSL 9.8 19.2 21.0
Bayes-VO-SSL 4.1 4.0 4.0 7.1 7.2 7.7 11.3 11.2 11.1
Bayes-CN-SSL 6.1 7.4 6.1 11.5 10.2 10.7 20.1 24.6 21.0
Bayes-MR 12.0 37.5 71.1
Bayes-VO-MR 10.9 10.7 14.1 20.4 27.2 23.4 44.4 45.8 51.0
Bayes-CN-MR 11.4 10.5 14.1 37.1 37.2 34.2 94.2 91.9 101.1
Bayes-PYR 11.7 40.1 100.1
Bayes-VO-PYR 10.3 10.4 11.1 19.1 17.1 19.1 40.5 40.9 41.1
Bayes-CN-PYR 21.2 20.9 25.0 57.1 57.1 54.3 104.3 101.9 101.1

TABLE XIV
EXPERIMENTAL RESULTS REGARDING GEOEYE IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH THREE, FIVE,

AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

OPF 69.9 ± 1.3 70.5 ± 1.4 71.6 ± 2.0
OPF-SSL 71.9 ± 0.6 72.5 ± 0.4 73.4 ± 0.4
OPF-VO-SSL 71.6 ± 0.6 71.0 ± 0.0 71.3 ± 0.7 71.8 ± 0.6 71.1 ± 0.6 71.6 ± 0.6 71.5 ± 0.4 72.1 ± 0.6 71.4 ± 0.5
OPF-CN-SSL 72.0 ± 0.5 72.4 ± 0.5 72.6 ± 0.7 72.9 ± 0.4 72.5 ± 0.4 72.7 ± 0.4 73.1 ± 0.2 73.6 ± 0.3 73.3 ± 0.3
OPF-MR-SSL 70.8 ± 1.0 71.5 ± 0.2 72.8 ± 0.1
OPF-VO-MR-SSL 72.5 ± 0.1 72.6 ± 0.1 72.2 ± 0.1 71.1 ± 0.1 70.6 ± 0.0 70.7 ± 0.0 72.4 ± 0.0 70.8 ± 0.0 70.8 ± 0.0
OPF-CN-MR-SSL 72.9 ± 0.1 73.1 ± 0.1 73.1 ± 0.1 73.9 ± 0.1 74.1 ± 0.1 74.3 ± 0.1 75.2 ± 0.0 75.5 ± 0.1 75.5 ± 0.1
OPF-PY-SSL 70.7 ± 0.6 71.7 ± 0.5 73.0 ± 0.3
OPF-VO-PY-SSL 70.4 ± 0.5 71.5 ± 0.5 72.2 ± 0.5 72.2 ± 0.6 72.0 ± 0.6 71.7 ± 0.6 72.3 ± 0.4 71.6 ± 0.2 73.6 ± 0.1
OPF-CN-PY-SSL 72.3 ± 0.5 71.9 ± 0.5 72.2 ± 0.5 72.4 ± 0.5 72.5 ± 0.4 72.7 ± 0.1 73.5 ± 0.1 73.8 ± 0.3 74.4 ± 0.2
Bayes 70.3 ± 0.0 70.8 ± 0.0 71.6 ± 0.1
Bayes-SSL 69.0 ± 0.0 70.1 ± 0.0 72.4 ± 0.3
Bayes-VO-SSL 70.2 ± 0.2 70.3 ± 0.2 70.5 ± 0.3 71.2 ± 0.1 71.3 ± 0.1 71.3 ± 0.2 72.1 ± 0.0 72.1 ± 0.1 72.0 ± 0.1
Bayes-CN-SSL 71.7 ± 0.2 71.9 ± 0.3 71.9 ± 0.4 72.5 ± 0.1 72.5 ± 0.2 72.3 ± 0.3 73.7 ± 0.2 73.6 ± 0.2 73.5 ± 0.2
Bayes-MR-SSL 65.6 ± 0.1 67.1 ± 0.0 68.9 ± 0.2
Bayes-VO-MR-SSL 73.0 ± 0.1 72.9 ± 0.1 72.7 ± 0.2 73.8 ± 0.1 73.9 ± 0.1 73.7 ± 0.1 73.4 ± 0.0 75.1 ± 0.1 75.0 ± 0.1
Bayes-CN-MR-SSL 73.3 ± 0.2 73.4 ± 0.2 73.4 ± 0.2 74.2 ± 0.1 74.5 ± 0.1 74.7 ± 0.0 75.7 ± 0.1 75.9 ± 0.0 75.7 ± 0.1
Bayes-PY-SSL 66.6 ± 0.1 68.0 ± 0.0 69.1 ± 0.0
Bayes-VO-PY-SSL 70.5 ± 0.1 70.7 ± 0.1 70.6 ± 0.1 71.8 ± 0.1 71.5 ± 0.1 71.6 ± 0.1 72.4 ± 0.1 72.7 ± 0.1 72.7 ± 0.1
Bayes-CN-PY-SSL 71.6 ± 0.1 71.8 ± 0.1 72.0 ± 0.1 72.5 ± 0.1 72.5 ± 0.1 72.8 ± 0.1 73.5 ± 0.1 73.5 ± 0.1 74.2 ± 0.1

their respective covering areas are the very same ones regarding
CBERS-2B and Ikonos-2 MS images, respectively. Table XI
presents the results with respect to Landsat 5 TM image. In
this case, standard OPF did not get better when increasing the
training set size; meanwhile, OPF-SSL obtained some results
slightly more accurate with larger data for training. Once again,
the proposed approaches (CN-SSL only) obtained the best re-
sults, but in this case with pyramidal-based SSL (see Section
III-C). Considering OPF-CN-PY-SSL with 5% of the dataset for
training, for instance, the proposed approach obtained results
around 8.58% more accurate than OPF-PY-SSL. Additionally,
Table XII presents the results using the “Hybrid” protocol. In

this case, both protocols (i.e., the hybrid and non-hybrid ap-
proaches) obtained similar results using 5% and 10% for training
purposes. However, if one takes into account a larger training set
(i.e., with 20%), the accuracy rate raised from 74.2% to 80.1%
when using OPF as the first classifier, which is quite good. Fi-
nally, Table XIII presents the mean computational load of the
compared techniques, which follows the very same behavior
observed for the other images.

Table XIV presents the experimental evaluation with respect
to Geoeye image, being the results somehow similar to the ones
obtained for Ikonos-2 MS data. In this case, the most accurate
techniques were the ones based on MR- and CN-SSL for both



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1539

TABLE XV
EXPERIMENTAL RESULTS REGARDING THE “HYBRID” EXPERIMENT OVER REGARDING GEOEYE IMAGE USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING

PURPOSES WITH THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

First step classification using OPF and the second step using Bayes

SSL 70.6 ± 0.8 73.4 ± 0.6 74.1 ± 0.8
VO-SSL 71.0 ± 1.6 68.3 ± 0.0 71.9 ± 0.3 70.7 ± 1.2 71.5 ± 0.0 73.6 ± 0.1 71.6 ± 1.3 70.9 ± 1.1 70.2 ± 0.2
CN-SSL 71.4 ± 1.1 69.7 ± 0.0 70.9 ± 0.2 72.1 ± 0.7 71.5 ± 0.0 70.1 ± 0.9 72.6 ± 1.1 71.6 ± 0.4 71.2 ± 0.0
MR 68.7 ± 0.5 75.2 ± 0.4 78.1 ± 0.8
VO-MR 72.3 ± 0.4 70.6 ± 0.0 71.9 ± 0.1 74.4 ± 0.4 71.5 ± 0.0 72.4 ± 0.3 77.8 ± 0.7 77.1 ± 0.6 74.1 ± 0.5
CN-MR 80.7 ± 0.5 82.6 ± 0.0 81.2 ± 0.0 74.5 ± 0.4 74.8 ± 0.0 74.4 ± 0.4 77.8 ± 0.4 78.3 ± 0.3 78.7 ± 0.0
PYR 73.0 ± 1.2 74.4 ± 0.7 73.8 ± 1.4
VO-PYR 71.4 ± 1.1 69.7 ± 0.0 70.7 ± 0.0 72.1 ± 0.7 71.5 ± 0.0 71.5 ± 0.0 72.6 ± 1.1 71.6 ± 0.4 70.3 ± 0.0
CN-PYR 74.6 ± 0.3 75.1 ± 0.0 73.6 ± 0.3 76.0 ± 0.6 71.5 ± 0.0 73.8 ± 0.0 78.2 ± 0.3 78.1 ± 0.3 75.2 ± 0.2

First step classification using Bayes and the second step using OPF

SSL 66.3 ± 0.2 68.0 ± 0.1 69.8 ± 1.2
VO-SSL 65.3 ± 1.3 67.4 ± 0.0 64.9 ± 0.0 67.1 ± 0.9 67.6 ± 0.0 71.6 ± 0.4 66.6 ± 1.5 67.2 ± 1.3 68.7 ± 0.1
CN-SSL 66.4 ± 0.7 68.8 ± 0.0 66.8 ± 0.0 67.1 ± 1.1 67.6 ± 0.10 69.1 ± 0.1 69.2 ± 0.9 68.2 ± 0.9 68.2 ± 0.3
MR 67.6 ± 0.2 67.6 ± 0.1 69.6 ± 0.1
VO-MR 67.1 ± 0.3 67.2 ± 0.0 65.8 ± 0.2 68.0 ± 0.2 67.6 ± 0.0 69.6 ± 0.0 69.2 ± 0.2 69.1 ± 0.2 68.7 ± 0.0
CN-MR 67.8 ± 0.1 67.7 ± 0.0 66.9 ± 0.1 68.7 ± 0.1 67.6 ± 0.0 69.4 ± 0.4 69.8 ± 0.1 70.0 ± 0.1 72.7 ± 0.0
PYR 66.0 ± 0.2 68.4 ± 0.1 69.4 ± 0.2
VO-PYR 65.7 ± 1.2 64.7 ± 0.0 64.9 ± 0.0 67.1 ± 0.9 67.6 ± 0.0 68.8 ± 0.2 67.5 ± 1.0 67.3 ± 1.0 66.4 ± 0.2
CN-PYR 67.2 ± 0.8 67.9 ± 0.0 66.8 ± 0.1 67.1 ± 0.9 67.6 ± 0.0 69.9 ± 0.1 68.7 ± 0.8 69.3 ± 0.6 68.9 ± 0.0

TABLE XVI
MEAN COMPUTATIONAL LOAD IN SECONDS REGARDING GEOEYE IMAGE

USING 5%, 10%, AND 20% OF THE IMAGE FOR TRAINING PURPOSES WITH

THREE, FIVE, AND SEVEN BASE CLASSIFIERS

Accuracy (5%) Accuracy (10%) Accuracy (20%)

Base classifiers 3 5 7 3 5 7 3 5 7

OPF-SSL 0.3 1.3 3.1
OPF-VO-SSL 0.3 0.4 0.3 0.7 0.7 0.7 1.2 1.2 1.1
OPF-CN-SSL 1.2 1.2 1.3 2.7 2.7 2.6 6.2 5.8 5.1
OPF-MR 3.0 10.6 23.3
OPF-VO-MR 3.1 3.1 3.3 6.4 6.7 6.1 11.6 11.5 11.1
OPF-CN-MR 8.9 9.9 9.4 16.1 24.9 18.7 37.6 41.6 43.1
OPF-PYR 1.2 5.5 15.1
OPF-VO-PYR 1.3 1.3 1.3 2.7 2.7 2.9 6.0 6.0 6.1
OPF-CN-PYR 3.9 3.3 3.3 8.2 8.7 9.1 14.6 20.8 18.1
Bayes-SSL 0.8 2.8 5.1
Bayes-VO-SSL 0.3 0.3 0.3 0.6 0.6 0.6 1.0 1.0 1.0
Bayes-CN-SSL 0.4 0.5 0.5 0.8 0.9 0.8 1.4 1.7 1.2
Bayes-MR 3.0 10.5 35.1
Bayes-VO-MR 0.8 0.8 0.8 1.6 1.6 1.6 3.1 3.0 3.0
Bayes-CN-MR 2.0 3.1 1.4 4.0 3.6 4.9 8.2 11.9 9.0
Bayes-PYR 1.1 14.3 31.1
Bayes-VO-PYR 0.8 0.8 0.9 1.5 1.6 1.6 2.8 2.9 1.0
Bayes-CN-PYR 0.9 0.9 1.0 1.9 2.0 2.1 3.8 3.9 4.0

classifiers. Another interesting point is the capability of CN-
SSL to achieve better results with larger training sets. If one
takes into account OPF-MR-SSL, for instance, it is possible to
observe the accuracy raised from 70.8% using 5% for training
to 72.8% with 20% to compose the training set, thus increasing
the recognition rates in around 2.74%. However, considering
OPF-CN-MR-SSL for the very same range of training set size
percentages, the accuracy was increased in about 3.17%.

Table XV presents the results concerning the “Hybrid” pro-
tocol. Once again, this approach allowed better results than the

“Standard” protocol with OPF as the first classifier. If one takes
into account a training set of 5%, the accuracy raised from 73.1%
to 82.6% (i.e., the accuracy increased about 11.50%) by just con-
sidering different classifiers during the stacked approach. In fact,
since each classifier works differently, we assume they might
be complimentary to each other, thus strengthening the idea
of using the ensemble. Finally, Table XVI presents the mean
computational load in seconds concerning Geoeye image, being
the behavior pretty much similar to the aforementioned experi-
ments, i.e., the voting-based approaches are considerably faster
than naı̈ve SSL, since one has classifiers learning over smaller
training sets. The concatenated approaches pay the price of us-
ing larger feature vectors. An interesting observation concerns
Naı̈ve-Bayes, which is consistently more effective even using
the concatenation-driven approach. Actually, OPF is penalized
with larger feature vectors, since it needs to compute the distance
among them several times during learning.

‘

VII. CONCLUSION

In this paper, we coped with the problem of land-cover classi-
fication by means of SSL, which attempts at considering spatial
information during the learning process. Such techniques per-
form an additional classification step with the original feature
vector of each sample extended with the labels of the neighbor-
hood samples (pixels). Therefore, the idea is to employ some sort
of contextual learning to make the classification process smarter.

Although such techniques usually improve the recognition
rates, they may also degrade the learning process when adding
misclassified labels to extend the original feature vectors of
the dataset samples. Therefore, in this paper, we propose two
ensemble-based approaches to alleviate this problem, since we
are now considering a committee of specialists to classify each



1540 IEEE JOURNAL OF SELECTED TOPICS IN APPLIED EARTH OBSERVATIONS AND REMOTE SENSING, VOL. 10, NO. 4, APRIL 2017

sample, thus obtaining a more reliable classification process for
the further extension of the feature vectors. The first approach
concatenates the outputs of each classifier in the ensemble, while
the second takes the majority voting of them.

The proposed approaches were validated in nine different
base SSL techniques, as well as using two different classi-
fiers. Although the proposed approaches were based on the
OPF classifier, we also showed they can be used with other
machine learning techniques. In addition, four satellite images
were used in this work: CBERS-2B, Landsat 5 TM, Ikonos-2
MS, and Geoeye. The proposed approaches were adapted in
three distinct SSL-oriented learning algorithms: standard SSL,
MR-SSL, and pyramidal-decomposition SSL (PY-SSL). In all
situations, at least one of the proposed approaches obtained
the best results according to the Wilcoxon signed-rank, usually
the concatenated-based one, which can alleviate the problem of
adding misclassified samples by increasing the dimension of the
feature space.

An additional round of experiments (“Hybrid” protocol)
showed to improve the results of the proposed approaches when
we used different classifiers at each stage of the sequential learn-
ing process. As a matter of fact, the idea of using ensembles to
promote the diversity plays a big role at different steps of the
process, i.e., when we are in charge of deciding how to extend
the feature vector, or even when we try to alleviate the shortcom-
ings of a given classifier used in a previous state of the stacked
paradigm. In regard to future works, we aim at comparing dif-
ferent schemas for the decision fusion step, such as weighted
voting and metaheuristic-based ones.

REFERENCES

[1] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd. ed.
New York, NY, USA: Wiley-Interscience, 2000.

[2] M. A. Bencherif, Y. Bazi, A. Guessoum, N. Alajlan, F. Melgani, and
H. AlHichri, “Fusion of extreme learning machine and graph-based op-
timization methods for active classification of remote sensing images,”
IEEE Geosc. Remote Sens. Lett., vol. 12, no. 3, pp. 527–531, Mar. 2015.

[3] J. A. Santos, P. H. Gosselin, S. Philipp-Foliguet, R. S. Torres, and A.
X. Falcão, “Interactive multiscale classification of high-resolution remote
sensing images,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens.,
vol. 6, no. 4, pp. 2020–2034, Aug. 2013.

[4] F. Li, L. Xu, P. Siva, A. Wong, and D. A. Clausi, “Hyperspectral image
classification with limited labeled training samples using enhanced en-
semble learning and conditional random fields,” IEEE J. Sel. Topics Appl.
Earth Observ Remote Sens., vol. 8, no. 6, pp. 2427–2438, Jun. 2015.

[5] E. Maggiori, Y. Tarabalka, G. Charpiat, and P. Alliez, “Convolutional neu-
ral networks for large-scale remote-sensing image classification,” IEEE
Trans. Geosci. Remote Sens., vol. 55, no. 2, pp. 645–657, Feb. 2017.

[6] M. K. S. Varma, N. K. K. Rao, K. K. Raju, and G. P. S. Varma, “Pixel-based
classification using support vector machine classifier,” in Proc. IEEE 6th
Int. Conf. Adv. Comput., 2016, pp. 51–55.

[7] B. Kumar and O. Dikshit, “Spectral contextual classification of hyperspec-
tral imagery with probabilistic relaxation labeling,” IEEE Trans. Cybern.,
2016, doi: 10.1109/TCYB.2016.2609882.

[8] A. Masjedi, M. J. Valadan Zoej, and Y. Maghsoudi, “Classification of
polarimetric SAR images based on modeling contextual information and
using texture features,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 2,
pp. 932–943, Feb. 2016.

[9] P. Tokarczyk, J. D. Wegner, S. Walk, and K. Schindler, “Features, color
spaces, and boosting: New insights on semantic classification of remote
sensing images,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 1,
pp. 280–295, Jan. 2015.

[10] W. W. Cohen and V. R. Carvalho, “Stacked sequential learning,” in
Proc. 19th Int. Joint Conf. Artif. Intell., San Francisco, CA, USA, 2005,
pp. 671–676.

[11] C. Gatta, E. Puertas, and O. Pujol, “Multi-scale stacked sequential learn-
ing,” Pattern Recognit., vol. 44, no. 10–11, pp. 2414–2426, 2011.

[12] E. Puertas, S. Escalera, and O. Pujol, “Multi-class multi-scale stacked se-
quential learning,” in Multiple Classifier Systems (Lecture Notes in Com-
puter Science), vol. 6713, C. Sansone, J. Kittler, and F. Roli, Eds. Berlin,
Germany: Springer, 2011, pp. 197–206.

[13] F. Sampedro, S. Escalera, and A. Puig, “Iterative multi-class multi-scale
stacked sequential learning: Definition and application to medical volume
segmentation,” Pattern Recognit. Lett., vol. 46, pp. 1–10, 2014.

[14] A. González, D. Vázquez, S. Ramos, A. M. López, and J. Amores, “Spa-
tiotemporal stacked sequential learning for pedestrian detection,” in Pat-
tern Recognition and Image Analysis (Lecture Notes in Computer Sci-
ence), vol. 9117, R. Paredes, J. S. Cardoso, and X. M. Pardo, Eds., Berlin,
Germany: Springer, 2015, pp. 3–12.

[15] E. Puertas, S. Escalera, and O. Pujol, “Generalized multi-scale stacked
sequential learning for multi-class classification,” Pattern Anal. Appl.,
vol. 18, no. 2, pp. 247–261, 2015.

[16] R. Roscher, B. Waske, and W. Forstner, “Incremental import vector ma-
chines for classifying hyperspectral data,” IEEE Trans. Geosci. Remote
Sens., vol. 50, no. 9, pp. 3463–3473, Sep. 2012.

[17] K. Subramanian and S. Suresh, “A meta-cognitive sequential learning
algorithm for neuro-fuzzy inference system,” Appl. Soft Comput., vol. 12,
no. 11, pp. 3603–3614, 2012.

[18] Y. Sun, Y. Yuan, and G. Wang, “An on-line sequential learning method
in social networks for node classification,” Neurocomputing, vol. 149,
pp. 207–214, 2015.

[19] X. Wang and M. Han, “Online sequential extreme learning machine
with kernels for nonstationary time series prediction,” Neurocomputing,
vol. 145, pp. 90–97, 2014.

[20] S. Suna, P. Zhonga, H. Xiaoa, and R. Wanga, “Spatial contextual classifi-
cation of remote sensing images using a gaussian process,” Remote Sens.
Lett., vol. 7, no. 2, pp. 131–140, 2016.

[21] M. Li, S. Zang, B. Zhang, S. Li, and C. Wu, “A review of remote sensing
image classification techniques: The role of spatio-contextual informa-
tion,” Eur. J. Remote Sens., vol. 47, pp. 389–411, 2014.

[22] C. H. Li, B. C. Kuo, C. T. Lin, and C. S. Huang, “A spatial-
contextual support vector machine for remotely sensed image classifica-
tion,” IEEE Trans. Geosci. Remote Sens., vol. 50, no. 3, pp. 784–799, Mar.
2012.

[23] G. Moser and S. B. Serpico, “Combining support vector machines and
Markov random fields in an integrated framework for contextual im-
age classification,” IEEE Trans. Geosci. Remote Sens., vol. 51, no. 5,
pp. 2734–2752, May 2013.

[24] D. Osaku et al., “Improving land cover classification through contextual-
based optimum-path forest,” Inf. Sci., vol. 324, pp. 60–87, 2015.

[25] J. P. Papa, A. X. Falcão, and C. T. N. Suzuki, “Supervised pattern clas-
sification based on optimum-path forest,” Int. J. Imag. Syst. Technol.,
vol. 19, no. 2, pp. 120–131, 2009.

[26] J. P. Papa, A. X. Falcão, V. H. C. Albuquerque, and J. M. R. S. Tavares, “Ef-
ficient supervised optimum-path forest classification for large datasets,”
Pattern Recognit., vol. 45, no. 1, pp. 512–520, 2012.

[27] J. P. Papa, S. E. N. Fernandes, and A. X. Falcão, “Optimum-path forest
based on k-connectivity: Theory and applications,” Pattern Recognit. Lett.,
2016, doi: 10.1016/j.patrec.2016.07.026.

[28] D. R. Pereira, R. Y. M. Nakamura, R. J. Pisani, and J. P. Papa, “Land-cover
classification through sequential learning-based optimum-path forest,” in
Proc. IEEE Int. Geosci. Remote Sens. Symp., 2015, pp. 76–79.

[29] R. J. Pisani, R. Y. N. Mizobe, P. S. Riedel, C. R. L. Zimback, A. X. Falcão,
and J. P. Papa, “Toward satellite-based land cover classification through
optimum-path forest,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 10,
pp. 6075–6085, Oct. 2014.

[30] R. Y. M. Nakamura, L. M. G. Fonseca, J. A. dos Santos, R. S. Torres,
X.-S. Yang, and J. P. Papa, “Nature-inspired framework for hyperspec-
tral band selection,” IEEE Trans. Geosci. Remote Sens., vol. 52, no. 4,
pp. 2126–2137, Apr. 2014.

[31] C. Allène, J.-Y. Audibert, M. Couprie, and R. Keriven, “Some links be-
tween extremum spanning forests, watersheds and min-cuts,” Image Vision
Comput., vol. 28, no. 10, pp. 1460–1471, 2010.

[32] T. G. Dietterich, “Machine learning for sequential data: A review,” in Proc.
Joint IAPR Int. Workshop Struct., Syntactic, Statist. Pattern Recognit.,
2002, pp. 15–30.

[33] J. P. Papa, C. T. N. Suzuki, and A. X. Falcão, LibOPF: A Library for the
Design of Optimum-path Forest Classifiers, Software version 2.1. 2014.
[Online]. Available: http://www.ic.unicamp.br/afalcao/LibOPF

[34] F. Wilcoxon, “Individual comparisons by ranking methods,” Biometrics
Bull., vol. 1, no. 6, pp. 80–83, 1945.



PEREIRA et al.: AN ENSEMBLE-BASED STACKED SEQUENTIAL LEARNING ALGORITHM FOR REMOTE SENSING IMAGERY CLASSIFICATION 1541

Danillo R. Pereira received the B.Sc. degree from
the Faculty of Sciences and Technology, São Paulo
State University, Presidente Prudente, Brazil, in 2006,
and the M.Sc. and Ph.D. degrees from the University
of Campinas, Campinas, Brazil, in 2009 and 2013,
respectively, all in computer science.

He is a Professor with the University of the West
of São Paulo, Presidente Prudente, and a Postdoctoral
Student with the Department of Computer Science,
Federal University of São Carlos, São Carlos, France.
His research interests include machine learning and

computer graphics.

Rodrigo J. Pisani received the graduate (B.Sc.) de-
gree in geography from the Faculty of Sciences
and Technology, São Paulo State University, Presi-
dente Prudente, Brazil, in 2005, the M.Sc. degree in
agronomy (energy in agriculture) from the Faculty
of Agronomic Sciences, São Paulo State University,
Botucatu, Brazil, in 2009, and the Ph.D. degree in
geosciences and environment from São Paulo State
University, in 2013.

He is an Adjunct Professor with the Nature
Sciences Institute, Federal University of Alfenas,

Alfenas, Brazil. His research interests include remote sensing, geographic in-
formation systems, and geoprocessing tools.

André N. de Souza received the B.Sc. degree in elec-
trical engineering from Mackenzie Presbyterian Uni-
versity, São Paulo, Brazil, in 1991, and the M.Sc. and
Ph.D. degrees in electrical engineering from the Poly-
technic School, University of São Paulo, São Paulo,
in 1995 and 1999, respectively.

He has been an Associate Professor with the De-
partment of Electrical Engineering, São Paulo State
University, São Paulo, since 2005. His interests in-
clude intelligent systems, transformers, fraud detec-
tion in electrical systems, atmospheric discharges,

and power quality.

João P. Papa (M’09) received the B.Sc. degree in
information systems from São Paulo State Univer-
sity, São Paulo, Brazil, in 2003, the M.Sc. degree in
computer science from the Federal University of São
Carlos, São Carlos, Brazil, in 2005, and the Ph.D.
degree in computer science from the University of
Campinas, Campinas, Brazil, in 2008.

During 2008–2009, he was a Postdoctorate Re-
searcher at the University of Campinas, and during
2014–2015, he was a Visiting Professor at the Center
for Brain Science, Harvard University. Since 2009, he

has been an Associate Professor with the Department of the Computer Science,
São Paulo State University. His research interests include machine learning,
pattern recognition, and image processing.



<<
  /ASCII85EncodePages false
  /AllowTransparency false
  /AutoPositionEPSFiles true
  /AutoRotatePages /None
  /Binding /Left
  /CalGrayProfile (Gray Gamma 2.2)
  /CalRGBProfile (sRGB IEC61966-2.1)
  /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2)
  /sRGBProfile (sRGB IEC61966-2.1)
  /CannotEmbedFontPolicy /Warning
  /CompatibilityLevel 1.4
  /CompressObjects /Off
  /CompressPages true
  /ConvertImagesToIndexed true
  /PassThroughJPEGImages true
  /CreateJobTicket false
  /DefaultRenderingIntent /Default
  /DetectBlends true
  /DetectCurves 0.0000
  /ColorConversionStrategy /sRGB
  /DoThumbnails true
  /EmbedAllFonts true
  /EmbedOpenType false
  /ParseICCProfilesInComments true
  /EmbedJobOptions true
  /DSCReportingLevel 0
  /EmitDSCWarnings false
  /EndPage -1
  /ImageMemory 1048576
  /LockDistillerParams true
  /MaxSubsetPct 100
  /Optimize true
  /OPM 0
  /ParseDSCComments false
  /ParseDSCCommentsForDocInfo true
  /PreserveCopyPage true
  /PreserveDICMYKValues true
  /PreserveEPSInfo false
  /PreserveFlatness true
  /PreserveHalftoneInfo true
  /PreserveOPIComments false
  /PreserveOverprintSettings true
  /StartPage 1
  /SubsetFonts false
  /TransferFunctionInfo /Remove
  /UCRandBGInfo /Preserve
  /UsePrologue false
  /ColorSettingsFile ()
  /AlwaysEmbed [ true
    /Algerian
    /Arial-Black
    /Arial-BlackItalic
    /Arial-BoldItalicMT
    /Arial-BoldMT
    /Arial-ItalicMT
    /ArialMT
    /ArialNarrow
    /ArialNarrow-Bold
    /ArialNarrow-BoldItalic
    /ArialNarrow-Italic
    /ArialUnicodeMS
    /BaskOldFace
    /Batang
    /Bauhaus93
    /BellMT
    /BellMTBold
    /BellMTItalic
    /BerlinSansFB-Bold
    /BerlinSansFBDemi-Bold
    /BerlinSansFB-Reg
    /BernardMT-Condensed
    /BodoniMTPosterCompressed
    /BookAntiqua
    /BookAntiqua-Bold
    /BookAntiqua-BoldItalic
    /BookAntiqua-Italic
    /BookmanOldStyle
    /BookmanOldStyle-Bold
    /BookmanOldStyle-BoldItalic
    /BookmanOldStyle-Italic
    /BookshelfSymbolSeven
    /BritannicBold
    /Broadway
    /BrushScriptMT
    /CalifornianFB-Bold
    /CalifornianFB-Italic
    /CalifornianFB-Reg
    /Centaur
    /Century
    /CenturyGothic
    /CenturyGothic-Bold
    /CenturyGothic-BoldItalic
    /CenturyGothic-Italic
    /CenturySchoolbook
    /CenturySchoolbook-Bold
    /CenturySchoolbook-BoldItalic
    /CenturySchoolbook-Italic
    /Chiller-Regular
    /ColonnaMT
    /ComicSansMS
    /ComicSansMS-Bold
    /CooperBlack
    /CourierNewPS-BoldItalicMT
    /CourierNewPS-BoldMT
    /CourierNewPS-ItalicMT
    /CourierNewPSMT
    /EstrangeloEdessa
    /FootlightMTLight
    /FreestyleScript-Regular
    /Garamond
    /Garamond-Bold
    /Garamond-Italic
    /Georgia
    /Georgia-Bold
    /Georgia-BoldItalic
    /Georgia-Italic
    /Haettenschweiler
    /HarlowSolid
    /Harrington
    /HighTowerText-Italic
    /HighTowerText-Reg
    /Impact
    /InformalRoman-Regular
    /Jokerman-Regular
    /JuiceITC-Regular
    /KristenITC-Regular
    /KuenstlerScript-Black
    /KuenstlerScript-Medium
    /KuenstlerScript-TwoBold
    /KunstlerScript
    /LatinWide
    /LetterGothicMT
    /LetterGothicMT-Bold
    /LetterGothicMT-BoldOblique
    /LetterGothicMT-Oblique
    /LucidaBright
    /LucidaBright-Demi
    /LucidaBright-DemiItalic
    /LucidaBright-Italic
    /LucidaCalligraphy-Italic
    /LucidaConsole
    /LucidaFax
    /LucidaFax-Demi
    /LucidaFax-DemiItalic
    /LucidaFax-Italic
    /LucidaHandwriting-Italic
    /LucidaSansUnicode
    /Magneto-Bold
    /MaturaMTScriptCapitals
    /MediciScriptLTStd
    /MicrosoftSansSerif
    /Mistral
    /Modern-Regular
    /MonotypeCorsiva
    /MS-Mincho
    /MSReferenceSansSerif
    /MSReferenceSpecialty
    /NiagaraEngraved-Reg
    /NiagaraSolid-Reg
    /NuptialScript
    /OldEnglishTextMT
    /Onyx
    /PalatinoLinotype-Bold
    /PalatinoLinotype-BoldItalic
    /PalatinoLinotype-Italic
    /PalatinoLinotype-Roman
    /Parchment-Regular
    /Playbill
    /PMingLiU
    /PoorRichard-Regular
    /Ravie
    /ShowcardGothic-Reg
    /SimSun
    /SnapITC-Regular
    /Stencil
    /SymbolMT
    /Tahoma
    /Tahoma-Bold
    /TempusSansITC
    /TimesNewRomanMT-ExtraBold
    /TimesNewRomanMTStd
    /TimesNewRomanMTStd-Bold
    /TimesNewRomanMTStd-BoldCond
    /TimesNewRomanMTStd-BoldIt
    /TimesNewRomanMTStd-Cond
    /TimesNewRomanMTStd-CondIt
    /TimesNewRomanMTStd-Italic
    /TimesNewRomanPS-BoldItalicMT
    /TimesNewRomanPS-BoldMT
    /TimesNewRomanPS-ItalicMT
    /TimesNewRomanPSMT
    /Times-Roman
    /Trebuchet-BoldItalic
    /TrebuchetMS
    /TrebuchetMS-Bold
    /TrebuchetMS-Italic
    /Verdana
    /Verdana-Bold
    /Verdana-BoldItalic
    /Verdana-Italic
    /VinerHandITC
    /Vivaldii
    /VladimirScript
    /Webdings
    /Wingdings2
    /Wingdings3
    /Wingdings-Regular
    /ZapfChanceryStd-Demi
    /ZWAdobeF
  ]
  /NeverEmbed [ true
  ]
  /AntiAliasColorImages false
  /CropColorImages true
  /ColorImageMinResolution 150
  /ColorImageMinResolutionPolicy /OK
  /DownsampleColorImages true
  /ColorImageDownsampleType /Bicubic
  /ColorImageResolution 150
  /ColorImageDepth -1
  /ColorImageMinDownsampleDepth 1
  /ColorImageDownsampleThreshold 1.50000
  /EncodeColorImages true
  /ColorImageFilter /DCTEncode
  /AutoFilterColorImages false
  /ColorImageAutoFilterStrategy /JPEG
  /ColorACSImageDict <<
    /QFactor 0.76
    /HSamples [2 1 1 2] /VSamples [2 1 1 2]
  >>
  /ColorImageDict <<
    /QFactor 0.40
    /HSamples [1 1 1 1] /VSamples [1 1 1 1]
  >>
  /JPEG2000ColorACSImageDict <<
    /TileWidth 256
    /TileHeight 256
    /Quality 15
  >>
  /JPEG2000ColorImageDict <<
    /TileWidth 256
    /TileHeight 256
    /Quality 15
  >>
  /AntiAliasGrayImages false
  /CropGrayImages true
  /GrayImageMinResolution 150
  /GrayImageMinResolutionPolicy /OK
  /DownsampleGrayImages true
  /GrayImageDownsampleType /Bicubic
  /GrayImageResolution 300
  /GrayImageDepth -1
  /GrayImageMinDownsampleDepth 2
  /GrayImageDownsampleThreshold 1.50000
  /EncodeGrayImages true
  /GrayImageFilter /DCTEncode
  /AutoFilterGrayImages false
  /GrayImageAutoFilterStrategy /JPEG
  /GrayACSImageDict <<
    /QFactor 0.76
    /HSamples [2 1 1 2] /VSamples [2 1 1 2]
  >>
  /GrayImageDict <<
    /QFactor 0.40
    /HSamples [1 1 1 1] /VSamples [1 1 1 1]
  >>
  /JPEG2000GrayACSImageDict <<
    /TileWidth 256
    /TileHeight 256
    /Quality 15
  >>
  /JPEG2000GrayImageDict <<
    /TileWidth 256
    /TileHeight 256
    /Quality 15
  >>
  /AntiAliasMonoImages false
  /CropMonoImages true
  /MonoImageMinResolution 1200
  /MonoImageMinResolutionPolicy /OK
  /DownsampleMonoImages true
  /MonoImageDownsampleType /Bicubic
  /MonoImageResolution 600
  /MonoImageDepth -1
  /MonoImageDownsampleThreshold 1.50000
  /EncodeMonoImages true
  /MonoImageFilter /CCITTFaxEncode
  /MonoImageDict <<
    /K -1
  >>
  /AllowPSXObjects false
  /CheckCompliance [
    /None
  ]
  /PDFX1aCheck false
  /PDFX3Check false
  /PDFXCompliantPDFOnly false
  /PDFXNoTrimBoxError true
  /PDFXTrimBoxToMediaBoxOffset [
    0.00000
    0.00000
    0.00000
    0.00000
  ]
  /PDFXSetBleedBoxToMediaBox true
  /PDFXBleedBoxToTrimBoxOffset [
    0.00000
    0.00000
    0.00000
    0.00000
  ]
  /PDFXOutputIntentProfile (None)
  /PDFXOutputConditionIdentifier ()
  /PDFXOutputCondition ()
  /PDFXRegistryName ()
  /PDFXTrapped /False

  /CreateJDFFile false
  /Description <<
    /CHS <FEFF4f7f75288fd94e9b8bbe5b9a521b5efa7684002000410064006f006200650020005000440046002065876863900275284e8e55464e1a65876863768467e5770b548c62535370300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c676562535f00521b5efa768400200050004400460020658768633002>
    /CHT <FEFF4f7f752890194e9b8a2d7f6e5efa7acb7684002000410064006f006200650020005000440046002065874ef69069752865bc666e901a554652d965874ef6768467e5770b548c52175370300260a853ef4ee54f7f75280020004100630072006f0062006100740020548c002000410064006f00620065002000520065006100640065007200200035002e003000204ee553ca66f49ad87248672c4f86958b555f5df25efa7acb76840020005000440046002065874ef63002>
    /DAN <FEFF004200720075006700200069006e0064007300740069006c006c0069006e006700650072006e0065002000740069006c0020006100740020006f007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400650072002c0020006400650072002000650067006e006500720020007300690067002000740069006c00200064006500740061006c006a006500720065007400200073006b00e60072006d007600690073006e0069006e00670020006f00670020007500640073006b007200690076006e0069006e006700200061006600200066006f0072007200650074006e0069006e006700730064006f006b0075006d0065006e007400650072002e0020004400650020006f007000720065007400740065006400650020005000440046002d0064006f006b0075006d0065006e0074006500720020006b0061006e002000e50062006e00650073002000690020004100630072006f00620061007400200065006c006c006500720020004100630072006f006200610074002000520065006100640065007200200035002e00300020006f00670020006e0079006500720065002e>
    /DEU <FEFF00560065007200770065006e00640065006e0020005300690065002000640069006500730065002000450069006e007300740065006c006c0075006e00670065006e0020007a0075006d002000450072007300740065006c006c0065006e00200076006f006e002000410064006f006200650020005000440046002d0044006f006b0075006d0065006e00740065006e002c00200075006d002000650069006e00650020007a0075007600650072006c00e40073007300690067006500200041006e007a006500690067006500200075006e00640020004100750073006700610062006500200076006f006e00200047006500730063006800e40066007400730064006f006b0075006d0065006e00740065006e0020007a0075002000650072007a00690065006c0065006e002e00200044006900650020005000440046002d0044006f006b0075006d0065006e007400650020006b00f6006e006e0065006e0020006d006900740020004100630072006f00620061007400200075006e0064002000520065006100640065007200200035002e003000200075006e00640020006800f600680065007200200067006500f600660066006e00650074002000770065007200640065006e002e>
    /ESP <FEFF005500740069006c0069006300650020006500730074006100200063006f006e0066006900670075007200610063006900f3006e0020007000610072006100200063007200650061007200200064006f00630075006d0065006e0074006f0073002000640065002000410064006f00620065002000500044004600200061006400650063007500610064006f007300200070006100720061002000760069007300750061006c0069007a00610063006900f3006e0020006500200069006d0070007200650073006900f3006e00200064006500200063006f006e006600690061006e007a006100200064006500200064006f00630075006d0065006e0074006f007300200063006f006d00650072006300690061006c00650073002e002000530065002000700075006500640065006e00200061006200720069007200200064006f00630075006d0065006e0074006f00730020005000440046002000630072006500610064006f007300200063006f006e0020004100630072006f006200610074002c002000410064006f00620065002000520065006100640065007200200035002e003000200079002000760065007200730069006f006e0065007300200070006f00730074006500720069006f007200650073002e>
    /FRA <FEFF005500740069006c006900730065007a00200063006500730020006f007000740069006f006e00730020006100660069006e00200064006500200063007200e900650072002000640065007300200064006f00630075006d0065006e00740073002000410064006f006200650020005000440046002000700072006f00660065007300730069006f006e006e0065006c007300200066006900610062006c0065007300200070006f007500720020006c0061002000760069007300750061006c00690073006100740069006f006e0020006500740020006c00270069006d007000720065007300730069006f006e002e0020004c0065007300200064006f00630075006d0065006e00740073002000500044004600200063007200e900e90073002000700065007500760065006e0074002000ea0074007200650020006f007500760065007200740073002000640061006e00730020004100630072006f006200610074002c002000610069006e00730069002000710075002700410064006f00620065002000520065006100640065007200200035002e0030002000650074002000760065007200730069006f006e007300200075006c007400e90072006900650075007200650073002e>
    /ITA (Utilizzare queste impostazioni per creare documenti Adobe PDF adatti per visualizzare e stampare documenti aziendali in modo affidabile. I documenti PDF creati possono essere aperti con Acrobat e Adobe Reader 5.0 e versioni successive.)
    /JPN <FEFF30d330b830cd30b9658766f8306e8868793a304a3088307353705237306b90693057305f002000410064006f0062006500200050004400460020658766f8306e4f5c6210306b4f7f75283057307e305930023053306e8a2d5b9a30674f5c62103055308c305f0020005000440046002030d530a130a430eb306f3001004100630072006f0062006100740020304a30883073002000410064006f00620065002000520065006100640065007200200035002e003000204ee5964d3067958b304f30533068304c3067304d307e305930023053306e8a2d5b9a3067306f30d530a930f330c8306e57cb30818fbc307f3092884c3044307e30593002>
    /KOR <FEFFc7740020c124c815c7440020c0acc6a9d558c5ec0020be44c988b2c8c2a40020bb38c11cb97c0020c548c815c801c73cb85c0020bcf4ace00020c778c1c4d558b2940020b3700020ac00c7a50020c801d569d55c002000410064006f0062006500200050004400460020bb38c11cb97c0020c791c131d569b2c8b2e4002e0020c774b807ac8c0020c791c131b41c00200050004400460020bb38c11cb2940020004100630072006f0062006100740020bc0f002000410064006f00620065002000520065006100640065007200200035002e00300020c774c0c1c5d0c11c0020c5f40020c2180020c788c2b5b2c8b2e4002e>
    /NLD (Gebruik deze instellingen om Adobe PDF-documenten te maken waarmee zakelijke documenten betrouwbaar kunnen worden weergegeven en afgedrukt. De gemaakte PDF-documenten kunnen worden geopend met Acrobat en Adobe Reader 5.0 en hoger.)
    /NOR <FEFF004200720075006b00200064006900730073006500200069006e006e007300740069006c006c0069006e00670065006e0065002000740069006c002000e50020006f0070007000720065007400740065002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e00740065007200200073006f006d002000650072002000650067006e0065007400200066006f00720020007000e5006c006900740065006c006900670020007600690073006e0069006e00670020006f00670020007500740073006b007200690066007400200061007600200066006f0072007200650074006e0069006e006700730064006f006b0075006d0065006e007400650072002e0020005000440046002d0064006f006b0075006d0065006e00740065006e00650020006b0061006e002000e50070006e00650073002000690020004100630072006f00620061007400200065006c006c00650072002000410064006f00620065002000520065006100640065007200200035002e003000200065006c006c00650072002e>
    /PTB <FEFF005500740069006c0069007a006500200065007300730061007300200063006f006e00660069006700750072006100e700f50065007300200064006500200066006f0072006d00610020006100200063007200690061007200200064006f00630075006d0065006e0074006f0073002000410064006f00620065002000500044004600200061006400650071007500610064006f00730020007000610072006100200061002000760069007300750061006c0069007a006100e700e3006f002000650020006100200069006d0070007200650073007300e3006f00200063006f006e0066006900e1007600650069007300200064006500200064006f00630075006d0065006e0074006f007300200063006f006d0065007200630069006100690073002e0020004f007300200064006f00630075006d0065006e0074006f00730020005000440046002000630072006900610064006f007300200070006f00640065006d0020007300650072002000610062006500720074006f007300200063006f006d0020006f0020004100630072006f006200610074002000650020006f002000410064006f00620065002000520065006100640065007200200035002e0030002000650020007600650072007300f50065007300200070006f00730074006500720069006f007200650073002e>
    /SUO <FEFF004b00e40079007400e40020006e00e40069007400e4002000610073006500740075006b007300690061002c0020006b0075006e0020006c0075006f0074002000410064006f0062006500200050004400460020002d0064006f006b0075006d0065006e007400740065006a0061002c0020006a006f0074006b006100200073006f0070006900760061007400200079007200690074007900730061007300690061006b00690072006a006f006a0065006e0020006c0075006f00740065007400740061007600610061006e0020006e00e400790074007400e4006d0069007300650065006e0020006a0061002000740075006c006f007300740061006d0069007300650065006e002e0020004c0075006f0064007500740020005000440046002d0064006f006b0075006d0065006e00740069007400200076006f0069006400610061006e0020006100760061007400610020004100630072006f0062006100740069006c006c00610020006a0061002000410064006f00620065002000520065006100640065007200200035002e0030003a006c006c00610020006a006100200075007500640065006d006d0069006c006c0061002e>
    /SVE <FEFF0041006e007600e4006e00640020006400650020006800e4007200200069006e0073007400e4006c006c006e0069006e006700610072006e00610020006f006d002000640075002000760069006c006c00200073006b006100700061002000410064006f006200650020005000440046002d0064006f006b0075006d0065006e007400200073006f006d00200070006100730073006100720020006600f60072002000740069006c006c006600f60072006c00690074006c006900670020007600690073006e0069006e00670020006f006300680020007500740073006b007200690066007400650072002000610076002000610066006600e4007200730064006f006b0075006d0065006e0074002e002000200053006b006100700061006400650020005000440046002d0064006f006b0075006d0065006e00740020006b0061006e002000f600700070006e00610073002000690020004100630072006f0062006100740020006f00630068002000410064006f00620065002000520065006100640065007200200035002e00300020006f00630068002000730065006e006100720065002e>
    /ENU (Use these settings to create PDFs that match the "Suggested"  settings for PDF Specification 4.0)
  >>
>> setdistillerparams
<<
  /HWResolution [600 600]
  /PageSize [612.000 792.000]
>> setpagedevice