Waste Management 57 (2016) 46–56 Contents lists available at ScienceDirect Waste Management journal homepage: www.elsevier .com/ locate/wasman Sustainability assessment and prioritisation of e-waste management options in Brazil http://dx.doi.org/10.1016/j.wasman.2016.01.034 0956-053X/� 2016 Elsevier Ltd. All rights reserved. ⇑ Corresponding author at: Department of Environmental Engineering, Sao Paulo State University ‘Julio de Mesquita Filho’ (UNESP), Parque Tecnológico de São José dos Campos – SP, Rod. Presidente Dutra, km 137.8, 12.247-004 São José dos Campos, SP, Brazil. E-mail address: ricardo@sage.coppe.ufrj.br (R.G. de Souza). Ricardo Gabbay de Souza a,⇑, João C. Namorado Clímaco b, Annibal Parracho Sant’Anna c, Tiago Barreto Rocha d, Rogério de Aragão Bastos do Valle a, Osvaldo Luiz Gonçalves Quelhas c a Production Engineering Program, Federal University of Rio de Janeiro, Centro de Tecnologia, Sala F-101, Cidade Universitária, 21941-909 Rio de Janeiro, RJ, Brazil b INESC-C, Universidade de Coimbra, Rua Antero de Quental 199, 3000 Coimbra, Portugal cProgram of Sustainable Management Systems, Fluminense Federal University, Rua Passos da Pátria 156, Sala 329-A, 24.020-971 Niteroi, RJ, Brazil dCenter for Information Technology Renato Archer, Rodovia Dom Pedro I, Km 143.6, 13069-901 Campinas, SP, Brazil a r t i c l e i n f o a b s t r a c t Article history: Received 30 September 2015 Revised 11 January 2016 Accepted 26 January 2016 Available online 4 February 2016 Keywords: Waste electric and electronic equipment Reverse logistics Multicriteria decision analysis Composition of probabilistic preferences Life cycle assessment Sustainability assessment Brazil has an increasing rate of e-waste generation, but there are currently few adequate management systems in operation, with the largest share of Waste Electrical and Electronic Equipment (WEEE) going to landfill sites or entering informal chains. The National Solid Waste Policy (2010) enforces the imple- mentation of reverse logistics systems under the shared responsibility of consumers, companies and gov- ernments. The objective of this paper is to assess sustainability and prioritise system alternatives for potential implementation in the metropolitan region of Rio de Janeiro. Sustainability criteria and decision alternatives were defined by elicitation of stakeholders. The adopted multicriteria approach combines Life Cycle Assessment with qualitative evaluations by a small sample of regional experts with knowledge of the problem. The recommended system consists of a hybrid WEEE collection scheme with delivery points at shops, metro stations and neighbourhood centres; a pre-treatment phase with the involvement of private companies, cooperatives and social enterprises; and full recycling of all components in the country. � 2016 Elsevier Ltd. All rights reserved. 1. Introduction Adequate e-waste management is still a challenge in most parts of the world, especially in developing countries. It is estimated that Brazil generated 3.8 kg of Waste Electrical and Electronic Equip- ment (WEEE) per capita in 2008 (Araújo et al., 2012), and 7 kg/ capita in 2014 (StEP, 2015). This may be less than Mexico’s 2014 generation rate (8.2), but it is more than the other BRICS countries, with the exception of Russia (China 4.4 kg/capita, India 1.3, South Africa 6.6, Russia 8.7) (StEP, 2015). Despite such a rapidly increas- ing generation rate, only a few adequate WEEE management sys- tems are currently operating in the country. A large share of the e-waste produced is still disposed mixed with household waste and is destined for landfill sites, or informal chains operated by waste pickers, cooperatives and scrap dealers. The estimated recy- cling rate for the country is 2% (Bandini, 2009 apud Araujo, 2012). In Rio de Janeiro city, the composition of collected household waste in 2012 indicated that 3.7 k tonnes of WEEE were sent to landfill sites (COMLURB, 2013), wasting valuable and non-renewable resources with considerable environmental risks. It is known that rough recycling techniques like burning cables and acid leaching are commonly applied by the informal sector in the country (Souza, 2014; Lundgren, 2012). This is an insalubrious and ineffi- cient practice to recover materials. In addition, the country seems to be an illegal receiver of e-waste from developed countries in North America (Lundgren, 2012). Illegal and informal activities are also responsible for a large amount of Electrical and Electronic Equipment (EEE) consumed in the country; in 2014 non-official markets accounted for 1.5 million purchased computers, corre- sponding to 15% of the total for the year, and 35% of desktops (ABINEE, 2015). In order to tackle those issues and to implement adequate e-waste management, the Brazilian Solid Waste National Policy – PNRS (Brazil, 2010) enforced the implementation of WEEE reverse logistics under the shared responsibility of EEE producers, importers, distributors and retailers (direct chain), with broader responsibilities of governments and other actors. In order to imple- ment reverse logistics, those EEE direct chain actors must analyse http://crossmark.crossref.org/dialog/?doi=10.1016/j.wasman.2016.01.034&domain=pdf http://dx.doi.org/10.1016/j.wasman.2016.01.034 mailto:ricardo@sage.coppe.ufrj.br http://dx.doi.org/10.1016/j.wasman.2016.01.034 http://www.sciencedirect.com/science/journal/0956053X http://www.elsevier.com/locate/wasman R.G. de Souza et al. /Waste Management 57 (2016) 46–56 47 different WEEE reverse logistics options, assess technical and eco- nomic feasibility, and propose a model satisfying the objectives and principles defined in the Policy as far as possible, which include the protection of public health and environmental quality; incentivising the recycling industry and resource recovery; inte- grating waste management; making the articulation of the differ- ent sectors; promoting operational and financial sustainability; giving stimulus to Life Cycle Assessment, and to the integration of waste pickers’ cooperatives (Brazil, 2010). Proposals of WEEE reverse logistics systems have been submitted by those actors to a council of related Federal Ministries, headed by the Ministry of the Environment (MMA), which is entrusted with analysing, sug- gesting alterations and selecting the model to be implemented in the country. By August 2013 four proposals had been received, but due to the complexity of these multiple interests as well as the complexity of the decision problem, namely the tasks of build- ing a coherent set of criteria and to evaluate and compare system alternatives, two years later there was still no approved final model. Proponents pointed out some issues that still needed to be addressed, such as: the implementation of a clearly communi- cated recycling fee; the control of imported electronic products and the simplification of WEEE transportation and WEEE ownership (Brazil, 2015). PNRS stimulates this decision process to be reproduced at lower government levels. For instance, a São Paulo state resolution calls for industry to introduce WEEE reverse logistics proposals. Com- mitments must be signed by the end of 2015 (Sao Paulo, 2015). Local government resolutions are essential because municipalities are legally responsible for Municipal Solid Waste Management (MSWM) and because reverse logistics systems must be aligned with the mandatory Municipal Waste Management Plans. The Brazilian National Solid Waste Plan (PLANARES) targets indicate that the implementation of PNRS waste management strategies must start first with the largest cities, with progressive expansion to the smallest ones. Adequate WEEE management implementation should consider a set of sustainability criteria, aligned with both the PNRS objec- tives and context-specific stakeholder values. Souza et al. (2015) elicited the perspectives of stakeholders involved in the Brazilian WEEE context and specifically in Rio de Janeiro, and using decision science techniques derived a set of relevant social and economic criteria to support this particular decision. These criteria were: social inclusion; employment and generation of income; profes- sional development; health risks and working conditions; workers access to education and healthcare; system feasibility and effi- ciency; population awareness and adhesion to reverse logistics; innovation and stimulus of new economic activities; and compet- itiveness of formal EEE products in regard to the informal ones. Assessment of environmental performances needs a systematic approach which calculates impacts based on systemmodelling and resources flows along the different EEE/WEEE life cycle stages. Because of its capacity to analyse complex systems and a large amount of data, Life Cycle Assessment (LCA) has been widely applied in the context of waste management and particularly WEEE management. Besides this fact, there are still few LCA appli- cations in Brazilian waste management. Despite the need for relevant information to assess potential impacts of system alternatives and to make decisions on Brazilian e-waste management, there is a lack of an adequate database. Col- lection of primary data is often obstructive, especially in regional, local and organisational scales. A practical solution to facilitate such decision could be to promote the integration of Multicriteria Decision Analysis (MCDA) with LCA, qualitative evaluations of social and economic indicators. A robust approach to sustainability assessment and prioritisation of alternatives should be a multicri- teria method that, among other features, allows for the adoption of a life cycle perspective and for a non-compensatory integration of both quantitative and qualitative indicators (Cinelli et al., 2014). Regarding qualitative assessment of waste management sus- tainability indicators. In some contexts the available sample of evaluators with proper knowledge of the problem may not be suf- ficient to enable statistical analyses of the qualitative and quantita- tive measures, but the intervention of human expertise must still be considered in the decision. This can be the case, for example, of technical councils that may be organised to evaluate local WEEE management systems in Brazilian cities. The objective of this paper is to assess sustainability and priori- tise system alternatives for e-waste management in the city of Rio de Janeiro, Brazil. Specific objectives are: � to develop an approach and consult experts for qualitative eval- uation of social and economic relative performances of the sys- tem alternatives; � to build a multicriteria analysis model, adequate for cases with small samples of evaluators; and � to integrate MCDA with Life Cycle Assessment and with qualita- tive social and economic assessment. This paper builds upon previous studies of some of the authors, namely Souza et al. (2013), where e-waste management system alternatives for Brazil/Rio de Janeiro were identified based on stakeholder elicitation; and Souza et al. (2015), where a set of sus- tainability criteria for Brazilian/Rio de Janeiro e-waste manage- ment was derived from stakeholder perspectives. The present study targets e-waste management specialists and decision- makers in Brazil, and seeks to recommend a solution to the deci- sion problem, leading to the implementation of an e-waste man- agement system in Rio de Janeiro. 2. Background knowledge 2.1. E-waste management in the Brazilian National Solid Waste Policy According to the National Solid Waste Policy (PNRS), consumers (both population and institutions) are responsible for disposing of their e-waste separately at adequate delivery points defined in Municipal Solid Waste Plans. Retailers and distributors are respon- sible for returning the products to producers and importers, who in turn have to provide adequate treatment and final disposal of refuse. PNRS (Brazil, 2010) permits that these actors formalise one or more management entities, which can outsource reverse logistics operations to private waste management companies, MSWM schemes, skilled cooperatives or social enterprises. PNRS stimulates involvement of cooperatives if they have adequate training, working conditions and environmental licences to per- form the required activities. Proposals of a WEEE reverse logistics system for the country should be presented to the Ministry of the Environment by produc- ers, importers, distributors and retailers. The selected model must be formalised into a Sectorial Agreement (SA), a contract signed by all aforementioned parties. It can also be specific to State SA and to Municipal SA. The SAs have to detail, among other information: � descriptions of the set of integrated activities by each partici- pant in the reverse logistics system, in the processes of collec- tion, storage, transport, recycling and final disposal, indicating: – technical recommendations at each stage; – criteria to install and operate delivery points; – adopted collection schemes; – procedures and responsibilities for sorting, reuse, recycling, treatment, and final disposal activities; 48 R.G. de Souza et al. /Waste Management 57 (2016) 46–56 � possible hiring of waste pickers’ cooperatives and associations; � assessment of social and economic impacts of the reverse logis- tics system (Brazil, 2010). 2.2. E-waste management situation and feasibility assessment in Brazil In order to structure the Sectorial Agreement, the Brazilian gov- ernment previously requested a feasibility study (FS) of the WEEE reverse logistics implementation in Brazil (ABDI, 2012). This FS pointed out that Brazil produces around 1 million tonnes of WEEE per year and that around 4 thousand collection points in 2016 would be necessary in order to achieve around 70% of the collec- tion rate. The FS also pointed out that Brazil has 94 WEEE recycling facilities, most of them in the state of São Paulo (Southeast of Bra- zil, the same region as the state of Rio de Janeiro). Due to the large geographical distances within Brazil, the installation of sorting units that store WEEE components before their transportation to recycling facilities was suggested. In order to optimise cost, the FS recommend that cities with more than 150 thousand inhabi- tants must have a sorting unit. In addition, the FS proposed a WEEE reverse logistics model coordinated by one or more management entity and divided in two stages: � Primary stage: involves WEEE collection and transportation to sorting units. At this stage, the consumer should dispose of small devices in collection points available at EEE shops. The industry, importer or management entity should provide door collection of large devices. The commerce or management entity is responsible for storing WEEE collected and transport- ing it to sorting units. � Secondary stage: involves WEEE storage and transportation from sorting units to recycling facilities and final disposal. The man- agement entity should coordinate the sorting units and the transportation to recycling facilities. The FS discussed alternatives and made recommendations in eight key modelling variables: sources of resources for implemen- tation; responsibility for orphan products (from illegal or informal markets); targets for recovery and recycling; level of responsibility of the public administration; WEEE hazardousness classification; reuse in the reverse logistics system; WEEE sorting by brands; pro- portional responsibility for WEEE; and competition model. Their recommendations were, respectively: shared costs by the actors in the EEE chain (from industry to consumer); no recommendation for orphan products; recycling target is 100% of products declared by the SA signed companies; government will be responsible for providing resources for research, infrastructure and campaigns; WEEE should be treated as non-hazardous but only be dismantled by recyclers; delivery points and consumer service should be made available to organise the reuse chain; WEEE should be monitored by sampling at the sorting units for identification of orphans, data checking and reporting to authorities; cost share for each producer is defined in proportion with their relative sales in the previous year; and stimulating competition among several management entities created by partnerships of producers and importers. The FS was the base for a call for SA proposals to be presented by producers, importers, distributors and retailers. The Ministry of Environment received 11 proposals in 2013, but only 4 were accepted for evaluation (Veloso, 2015). 2.3. Existing studies in Brazilian e-waste management There is increasing interest in developing studies on Brazilian WEEE management, but there are currently few published studies on this topic. Araújo et al. (2012) proposed a model to estimate WEEE generation and applied it at national-level using primary data from the EEE market. Franco and Lange (2011), using a survey of householders’ behaviour, estimated the WEEE generation in the city of Belo Horizonte and tracked the current flows, identifying that most is either donated, kept or sold. Trying to describe the current situation, Saavedra and Ometto (2012) identified some existing initiatives referring to state legisla- tion, social and digital inclusion and recycling companies. Oliveira et al. (2012) provided an overview on current e-waste manage- ment practices in the country, highlighting the need for a well- defined model, the existence of a cascade reuse market, the lack of companies for complete e-waste recycling (PCBs are exported), the need for an efficient collection scheme, and the need to include waste pickers. Ongondo et al. (2011) added the evidence of WEEE being dumped. Quariguasi Frota Neto and Van Wassenhove (2013) found that existing WEEE take-back initiatives in the country are led by large multinational manufacturers, but are less advanced than those performed. Focusing the development of solutions for the e-waste problem in the country, Pimentel et al. (2013) presented a research project, called Ambientronics, which aims at technological development to support the Brazilian recycling industry. Chatterjee and Kumar (2009) proposed an outsourcing model which integrates non- formal operators in collection, disassembly and segregation of e- waste, whereas formal actors process PCBs for resource recovery. Campos et al. (2014) highlighted the relevance of creating a 5R net- work – Reduce, Redesign, Recycle, Reuse and Repurpose. Hirayama and Saron (2015) analysed the composition of Brazilian waste computer equipment (WCE) and concluded that implementation of stricter regulations for identification of thermoplastic polymers in WCE in Brazil is an important step for successful mechanical recycling of these materials. Other studies discussed Brazilian legal framework in compar- ison to other countries. These highlighted the need for national and global standardisation (Sant’Anna et al., 2014), the limitation of the PNRS in just requiring implementation of reverse logistics without an efficient management and control framework (Barboza et al., 2014), and the need for decisions from competent authorities observing diverging interpretations of legal stipulations and administrative procedures (Brandmann and Altvater, 2012). None of these identified studies focused on modelling and selec- tion of an e-waste management model for the country or a city by sustainability assessment and prioritisation of alternatives. 2.4. LCA studies in WEEE management Life Cycle Assessment (LCA) has been established as a technique to quantify the potential environmental impact using a systematic approach. LCA can provide objective indicators to compare pro- cesses or products, being an important tool in environmental man- agement and pollution prevention (ISO, 2006). It has been widely applied in the context of waste management (Laurent et al., 2014a) and particularly WEEE management (Wäger et al., 2011; Bigum et al., 2012; Hong et al., 2015; among others). There are few LCA studies in Brazilian waste management (e.g. Mendes et al., 2003; Leme et al., 2014; Reichert and Mendes, 2014), and none specifically for Brazilian household WEEE (although Rubin et al., 2014 analyse a generic recycling system that could be applied in Brazil, as suggested by the authors). Several studies have used LCA to analyse and compare different scenarios for e-waste treatment, aiming to support decisions, iden- tify key factors and improve opportunities associated with all stages within a determined system boundary. Using LCA, Huisman et al. (2008) concluded that the implementation of the European WEEE Directive contributed to reducing environmental impacts in all categories evaluated. Emphasis is set on the reduc- tion of 36 million tons of CO2 and 34 million tons of CFCs, which R.G. de Souza et al. /Waste Management 57 (2016) 46–56 49 are no longer discharged into the environment due to WEEE recycling. Rubin et al. (2014) used global data to apply LCA and compare two processes for recovering copper from PCBs, both adopting mechanical and electrochemical processing. They concluded that the process which employs acqua regia (a combination of nitric and chloridric acid) had the best environmental performance. Because of the narrow scope of their LCA study, with little neces- sity for regional data like distances or local WEEE composition, this recommended solution can be employed in Brazil or elsewhere. 2.5. Qualitative sustainability assessment of waste management There are just a few studies adopting qualitative evaluations in waste management, most of which are relatively recent. They are essentially based on the application of survey questionnaires to a sample of specialists or stakeholders (Glew et al., 2013; Khalili and Ehrlich, 2013; Nichols et al., 2013), or judgment by the ana- lysts based on observation of the context (Troschinetz and Mihelcic, 2009; Manhart, 2011; Al Sabbagh et al., 2012). Specifically in the context of e-waste management, Manhart (2011) assessed a reference (baseline) and two alternative scenar- ios for an international recycling cooperation between developing and developed countries: in the baseline, e-waste generated in low-income countries undergoes crude recycling and uncontrolled disposal; in the first scenario, e-waste components containing valuable metals are exported from developing to developed coun- tries for resource recovery; and in the second scenario, low-income countries import e-waste from developed ones for pre-treatment, and then export it back for recovery of precious metals. Some of these scenarios are rather similar to the alternative systems anal- ysed in the present study (Section 3.1). Qualitatively evaluating the scenarios by own judgment, based on the study of context information, the author concluded that Scenario 1 allows for ‘‘bet- ter management of hazardous substances, higher recycling rate of scarce and valuable metals, reduced pressure on mining, lower GHG emissions, income generation for the urban poor, and invest- ments into social and environmental standards”. 2.6. MCDA sustainability assessment in waste management There is a remarkable number of publications on the use of MCDA in waste management decision problems. Soltani et al. (2015) carried out a literature review of MCDA applications in MSWM, identifying 68 references published up to 2013. Addition- ally, we could identify another 27 articles from 2014 and 2015 (searched at Scopus in 4/Sep/2015). Probably the most common MSWM decision problem analysed with MCDA is the location of facilities and landfills (as studied by Liu et al., 2013; Eiselt and Marianov, 2015; Hamzeh et al., 2015). Among the 68 studies identified by Soltani et al., only 26 have acknowledged multiple stakeholders using MCDA, mainly for assigning weights to criteria. Most of these stakeholders were experts (69% of the 26 papers), governments or municipalities (62%) and public or residents (50%). Most of the papers adopted a popular MCDA method called AHP (34 of total studies and 15 with stakeholders), which is an aggregative and compensatory method, whilst other MCDA methods were used in 15 papers with stake- holder consultation. In this case, the methods PROMETHEE and ELECTRE, which are not compensatory, were used respectively in 1 and 3 papers. Although less commonly used, non- compensatory methods (in which the criteria are not aggregated in a single synthetizing criterion) are preferable when assessing sustainability, because they enforce a strong sustainability approach (for instance, a good economic performance cannot com- pensate a bad environmental or social performance) and because they can handle uncertainty (Cinelli et al., 2014). There are just a few studies of MCDA applications in WEEE management. Rousis et al. (2008) used PROMETHEE to examine 12 alternative WEEE systems for Cyprus, and concluded that the best option is ‘‘partial disassembly and forwarding of recyclable materials to the native existing market and disposal of the residues at landfill sites”. In this study, the scores were attributed by the analysts and validated by experts. Gamberini et al. (2010) analysed alternative WEEE transport routes in a region of Italy, evaluated with LCA and technical parameters, and ranked using fuzzy optimi- sation. Wibowo and Deng (2015) also adopted a fuzzy approach, aiming to evaluate e-waste recycling programs in Sri Lanka by con- sulting a group of three decision makers. Each one evaluated the alternatives considering four criteria, representing the three sus- tainability dimensions and a technical category. The evaluations were measured on a qualitative 5-point scale from Very Poor to Very Good. The best ranked program was ‘‘Recovery of precious metals and other recyclable materials such as metals, plastic from e-waste”. A problem with this approach is that the decision criteria are too broad and may lead to high subjectivity, whilst an advan- tage is the ability to handle subjectiveness and imprecision of qual- itative evaluations. Specifically regarding Brazilian WEEE, Guarnieri et al. (2014) proposed an MCDA framework to the problem of selecting third- party reverse logistics providers. The proposed model compre- hended the following criteria categories: logistics, financial, capac- ity/infrastructure, value added services to customers, alliances with suppliers, and environmental issues. It is not proposed as either a method for performance evaluations, nor as an MCDA approach to prioritise the alternatives. Among the identified studies in WEEE MCDA, the most similar to the present research was proposed by Wibowo and Deng (2015), because it worked with a small sample of evaluators (deci- sion makers) and with qualitative indicator performances esti- mated by the decision makers using predefined scales. The study of Gamberini et al. (2010) is similar to the present in the aspect of integrating LCA evaluations with other performance indicators in an MCDA approach. The principal differences are associated with the decision problem analysed, the sustainability criteria and the approaches adopted for MCDA qualitative evaluation (ours are described in Section 3). 2.7. The relevance to waste management of combining MCDA and LCA LCA, qualitative assessment and MCDA are innovative, powerful tools to improve waste management practices. The Brazilian National Solid Waste Policy mentions explicitly the objective of stimulating LCA implementations and implicitly the need for qual- itative assessment by social actors. MCDA is not mentioned, but could be an appropriated tool for obeying its principle that, when managing solid waste systems, ‘‘a systemic view that considers environmental, social, cultural, economic, technological, and Public Health variables” should be adopted. The three previous sections showed that there is vast literature underlining the specific role of each of these tools in waste man- agement. Kiddee et al. (2013) has highlighted the usefulness of LCA to ameliorate most e-waste problems to human health and the environment, admitting nevertheless, that it must be comple- mented by other tools in order to enhance the decision robustness. Troschinetz and Mihelcic (2009) explored the benefits of using qualitative data when there is a lack of numeric indicators to eval- uate performances of waste management systems. Wibowo and Deng (2015) concluded that MCDA is an effective and efficient approach to evaluate the performances of alternative e-waste recy- cling programs in a specific situation. It is a complex and challeng- 50 R.G. de Souza et al. /Waste Management 57 (2016) 46–56 ing problem, as it involves ‘‘several decision makers and multiple evaluation criteria with the presence of subjective and imprecise assessments”. The present paper looks for an integrated approach combining those three tools, which represents an innovation in WEEE theory and practice. Such an integrated approach offers an interesting potential solution to the current challenge of incorporating all dimensions of sustainability in waste management decisions. Some authors have already anticipated the importance of this inte- gration. Munda (2008) argues that every sustainability assessment approach is based on some form of multicriterial aggregation, explicit or not. For Cinelli et al. (2014), sustainability assessment must happen in a structured, transparent and reliable way and MCDA can largely contribute to this; moreover, they stress that an adequate MCDA method for this purpose should be able to inte- grate LCA and qualitative indicators. When adopting the strong sustainability paradigm, such integration should not involve a compensatory aggregation of the criteria (Valle and Clímaco, 2015). 3. Materials and methods 3.1. Alternative scenarios under evaluation The alternative WEEE systems considered for implementation in Rio de Janeiro are based on the previous mapping by Souza et al. (2013). The baseline scenario A (Fig. 1) corresponds to the existing collection and to the treatment activities for WEEE gener- ated in Rio de Janeiro, with emphasis on informality and landfill- ing. As there is no available data that indicates the total amounts of WEEE collected and processed by the existing chains (besides the landfill waste composition), the WEEE flows represented in the baseline scenario were assumed but may not correspond to reality. Rather, this scenario is relevant for assessment because it allows for perceiving the relative importance of informality and landfilling to sustainability performances. All the other four scenarios analysed (B1, B2, C1 and C2) con- sider the same hybrid solution for WEEE pre-treatment, consisting of 20% of WEEE being sorted and dismantled by social enterprises and skilled cooperatives, and 80% by formal WEEE recycling com- Door-to-door collec�on by truck dismantled by recycling companies Landfilled refu soc WEEE 45% 20% 20% Recycled material Re Baseline system adequately recycled regionally adeq recy regio 4% 20% Material Energy Fig. 1. Baseline system with panies. No informality takes place in these scenarios; cooperatives involved are adequately trained and licensed. These scenarios also consider that most WEEE components are adequately recycled regionally, assuming that recycling plants are located around São Paulo (ABDI, 2012). Some functional appliances and components are refurbished by a social program in a model similar to the Fábrica Verde (Green Factory), a Rio de Janeiro governmental social project (suspended early 2015) that offered training to youngsters from favelas in computer dismantling, refurbishment and mainte- nance. Refurbished computers were donated to NGOs and favela communities. Alternatives B1 and B2 (Fig. 2) have a collection scheme based on WEEE delivery only at EEE shops. In B1 PCBs are exported for recycling in Europe, whilst in B2 all components are recycled regionally. C1 and C2 (Fig. 3) consider a hybrid collection scheme with delivery points at EEE shops, metro stations and neighbourhood stations. C1 and C2 differ in the same way as B1 and B2. 3.2. LCA modelling The LCA model used in this research was developed and is described, seeking to satisfy the recommendations by Laurent et al. (2014b) for LCA applications in waste management. It also satisfices the guidance of ISO 14040 (ISO, 2006). LCA was applied to evaluate and compare the environmental performances of the alternative systems proposed in Section 3.1 (goal of the study). The functional unit was 1 ton of WEEE (Swiss mix as reported in SWICO, 2013). Because there is limited information about current e-waste flows and processes as currently occurring in Brazil, this study used international data mostly from the Ecoinvent database (Hischier et al., 2007) and others (SWICO, 2013; Hong et al., 2015). The modelling also adopted assumptions in the representa- tion of Brazilian e-waste treatment processes. Table 1 summarises the data used, model assumptions and sources of data. The distance of 150 km adopted as a pattern in all transport processes is a standard used in the Ecoinvent database when there is no detailed local data. A sensitivity analysis was applied to all variables marked with ‘‘Own Assumption” Table 1, with a variation Door-to-door collec�on by informal picker rbished by ial program dismantled informally 55% 5% 55% use Recycled material recycled informally uately cled nally 55% Emissions Effluent Residue respective WEEE flows. Delivery at EEE shops dismantled by regional recycling companies refurbished by social program WEEE 100% Recycled material B1 system Social enterprises/coopera�ves 20% 80% adequately recycled regionally adequately recycled regionally PWBs exported overseas 68% 12% 17% 3% 11% PWBs exported overseas Recycled material Reuse Delivery at EEE shops dismantled by regional recycling companies refurbished by social program WEEE 100% Recycled material B2 system Social enterprises/coopera�ves 20% 80% adequately recycled regionally adequately recycled regionally 68% 12% 17% 3% 11% Recycled material reuse Emissions Effluent Residue Material Energy Emissions Effluent Residue Material Energy Fig. 2. B1 and B2 alternative systems with respective WEEE flows. Delivery at metro stations dismantled by regional recycling companies refurbished by social program WEEE 33,3% Recycled material C1 system Social enterprises/cooperatives adequately recycled regionally adequately recycled regionally PWBs exported overseas 68% 12% 17% 3% 11% PWBs exported overseas Recycled material Reuse Delivery at EEE shops delivery stations 33,3% 33,3% 20% 80% Delivery at metro stations dismantled by regional recycling companies refurbished by social program WEEE 33,3% Recycled material C2 system Social enterprises/cooperatives adequately recycled regionally adequately recycled regionally 68% 12% 17% 3% 11% Recycled material Reuse Delivery at EEE shops delivery stations 33,3% 33,3% 20% 80% Emissions Effluent Residue Emissions Effluent Residue Material Energy Material Energy Fig. 3. C1 and C2 alternative systems with respective WEEE flows. R.G. de Souza et al. /Waste Management 57 (2016) 46–56 51 of 50% in the values. There was no allocation of impacts, but EcoIn- vent usually adopts the economic allocation. In order to support LCA data analysis and interpretation, this study used SimaPro� 8.1, a piece of software that supports various Life Cycle Inventory (LCI) databases. Among the Life Cycle Impact Assessment (LCIA) methods provided by the software, CML (Center of Environmental Science of Leiden University) has been chosen to classify and characterise inputs and outputs from the e-waste treatment process (e.g. energy, transportation, emissions, etc.) into potential environmental impacts. The impact categories adopted are in line with previous WEEE LCA applications found in literature (Section 2.3). These are: Abiotic Depletion (and Fuels); Global Warming; Human Toxicity; Ecotoxicity (Fresh Water, Marine, Ter- restrial); and Acidification. 3.3. Qualitative assessment of social and economic indicators 3.3.1. Social and economic impact categories The social and economic impact categories assessed in this study are those previously defined in Souza et al. (2015). These cat- egories were defined specifically for the decision problem under study, based on the elicitation of Rio de Janeiro and Brazilian stake- Table 1 LCA model input data, assumptions and sources. Process Data used and assumptions Source Door-to-door collection by truck Assumed 150 km between home and recycling companies with small truck Own assumption Door-to-door collection by pickers Assumed work labour has no environmental impact Own assumption Delivery at EEE shops collection Delivery at metro stations Delivery at delivery points Assumed 150 km between delivery points and recycling companies with medium truck Own assumption Dismantled by recycling company Assumed manual disassembly of e-waste in parts and material as reported by the SWICO organisation SWICO (2013) Refurbishment by social program Data collected at Fabrica Verde social program in Brazil which refurbished computers for donation (15% of e-waste received) Collected by the authors Informal dismantling Informal recycling Assumed Chinese data with no treatment at end-life disposal Hong et al. (2015) Adequately recycled regionally Social Enterprises/cooperatives Assumed European data from Ecoinvent database for dismantling companies. In case of regional PWB recycling, Chinese data was used with treatment of end- life disposal Hischier et al. (2007) and Hong et al. (2015) PWB recycling exported overseas Assumed European data for recycling with use of pyrometallurgical process to recycle PWB and recovery metals Hischier et al. (2007) and Classen et al. (2009) Landfill sites Assumed European data for landfilling with inert material Doka (2009) Table 2 Sustainability impact categories and indicators evaluated in this study. Source: Adapted from Souza et al. (2015) Categories Indicators Economic System feasibility Direct and indirect costs – total Profit and avoided costs – total System efficiency Amount of e-waste with adequate treatment/total collected Balanced demand/capacity of the system Population awareness and adhesion to reverse logistics Number of citizens and companies delivering E-WASTE to formal collection points Innovation and generation of new economic activities Number of recently created companies within the e-product and e-waste chains Number of companies within the EEE and E-WASTE chains with innovation recognised by the Brazilian Ministry of Science and Technology Competitiveness of formal products in regard to informal ones Increase in prices of formal e- products due to increased costs by formal e-waste reverse logistics Growth of the informal market (e- products and e-waste) Social Social inclusion Number of e-waste workers and relatives provided with social and psychological assistance Number of new e-waste workers that come from such groups: women; informality; prison; slums; alcoholism; drug addiction; crime; physical and mental disabilities Formal employment Number of formal e-waste workers Generation of income Average income of e-waste workers – formal and informal Opportunity for professional development Number of workers that undertook professional training and refresher courses Health risks and working conditions Number of e-waste workers (formal and informal) working in adequate conditions (equipment, protection, training) Occurrence of accidents at work and diseases directly related to the risks of the e-waste chain 52 R.G. de Souza et al. /Waste Management 57 (2016) 46–56 holders directly involved with the WEEE context. Table 2 sum- marises the social and economic impact categories together with respective indicators under evaluation. Access to healthcare Number of e-waste workers and their relatives provided with health insurance Access to education Number of e-waste workers and relatives with high level of education 3.3.2. Specialists sample, process of inquiry and data compilation The set of social and economic indicators was evaluated indi- vidually by a group of five experts. All of them are from universities or research institutes in Rio de Janeiro and São Paulo, and all of them have published in the areas of Brazilian waste management, WEEE and/or waste Life Cycle Assessment, in indexed international journals. For the purposes of this research, this group works as a consultancy team for the decision-maker – the WEEE management entity or the municipality. This way, the framework for decision- support, except for the LCA model, is completely based on the per- ceptions and judgments of regional stakeholders and experts. The number of experts selected is aimed at the minimum pos- sible evaluations to make a mathematical analysis acceptable. The mathematical approach used is explained in Section 3.4. The present study adopted a decision context where few experts with knowledge of the problem are available to make qualitative evalu- ations, in order that the approach can be replicated in other critical decision situations. The inquiry process consisted of either face-to-face or online interviews. In the first case, evaluations were made in writing, whilst in the second case aWord file is used. First, the system alter- natives and the sustainability categories were presented to the evaluators. Furthermore, they also have had the opportunity to clarify possible doubts and to validate these decision parameters. Secondly, they were asked to evaluate the relative performances of alternatives for each criterion, according to their perceptions. For this evaluation, a continuous interval scale (Fig. 1a) was pre- sented blank, indicating a direction of preference from the lower (�) to the higher (+) impact in each criterion. Evaluators had to mark the relative position of each alternative in that scale, taking into consideration the distances between alternatives themselves and the worst/best possible scores. An example of such evaluation by one of the consulted experts is presented in Fig. 4b. The reason for using such a scale instead of the Likert scale – the five-point scale (Very Poor – . . . – Very Good) is that it can be more intuitive to evaluators (alternatives are placed on the line as if they are in a ‘‘race”). It is also possible to translate the data into a Likert scale by defining thresholds. After obtaining evaluations from the consulted experts, the rel- ative distances of alternatives in the scales were measured with a rule, and normalised in the form percentage of the total length of the line. Normalisation also involved the correction of preference Fig. 4. Example of the evaluation scale used for qualitative assessment. (a) Blank scale. (b) Filled evaluation by an expert. R.G. de Souza et al. /Waste Management 57 (2016) 46–56 53 directions in the form of 0% to the worst and 100% to the best pos- sible performances. Data provided by all experts were compiled in Excel tables. 3.4. Multicriteria analysis: composition of probabilistic preferences The present study required a MCDA approach refusing the pos- sibility of bad performances being compensated by good perfor- mances (i.e., non-compensatory), allowing the integration of qualitative and quantitative indicators, and permitting a reason- ably trustworthy analysis of qualitative evaluations by a small sample of specialists. A good option is the recently developed Com- position of Probabilistic Preferences (CPP). CPP is a MCDA approach which takes into account imprecise measurements. Evaluation of an alternative is given by the probability of such alternative being the best or the worst in comparison to all other alternatives. The main steps in CPP are: (a) defining the criteria and the alter- natives; (b) transforming the initial evaluations into probabilistic distributions; (c) calculating probabilities of preference for each alternative; and (d) obtaining the global preferences. This method is fully described in Sant’Anna (2015). In CPP, the required input data are the preference evaluations, in our case established by five experts, as outlined above. When scores of alternatives in each criterion are obtained by those eval- uations, the sets of measurements for a same alternative can be treated as samples of that distribution, and can be used to estimate all of its parameters. For the purposes of this study, the perfor- mance measurements by evaluators in the continuous scale were divided into nine preference classes, defined as profiles with stretched extremes and shrunk centrals, in order to avoid empty or too crowded classes, and thus making a better distinction of the alternatives. A variation called CPP Beta was designed to analyse evaluations made by multiple experts. The beta distribution is applied to cases where asymmetry is assumed, thus permitting more flexibility in modelling the sample distribution. It also adopts an evaluation metric from 0 to 1, which is suitable to the scale adopted in the present study (Section 3.3). The density of the beta distribution is f(x) = [(x � L)/(U � L)]a�1[(U � x)/(U � L)]b�1/Beta(a,b), for x vary- ing between L and U and Beta denoting the beta function with pos- itive shape parameters a and b. The standard beta distribution, with extremes 0 and 1, is obtained by substituting the observed x by (x � L)/(U � L). In the beta model, it is possible to model the gain in precision with the increase in the information, by fixing a + b = N, denoting N as the number of experts. In this distribution the variance decreases with the sum of the parameters a and b (where a is f(x) times N, and b is (1 � f(x)) times N). The beta distribution is unimodal (has a single local maximum), which occurs for a > 1 or b > 1. Assuming that the xijk for alternative i according to criterion j are allowed to vary on bounded intervals (aij,bij) which are sufficiently far from L and U (0 and 1), the initial vector of evaluations (xij1, . . ., xijN) gives rise to a unimodal standard beta distribution with shape parameters aij and bij. Theoretically, unimodality is desirable in CPP because the values obtained from experts’ evaluations are a location around which the variable is dis- tributed. Because the evaluation scale is a vector of observations, some experts will necessarily be attracted to extreme evaluations. If this is observed, distorting the expected unimodal shape of the beta distributions for many values for i and j, this attraction can be compensated by slightly dislocating the measures to the inter- val between 1/(N � 1) and (N � 2)/(N � 1). For instance, unimodal- ity is granted replacing xijl by [1 + (N � 3)(xijl � aij)/(bij � aij)]/ (N � 1). When N = 1, 2 or 3, it is not possible to have 1/(N � 1) < (N � 2)/(N � 1). Because of this, CPP beta requires a number of experts N > 3, in order to have a completely acceptable interval. CPP calculates the following parameters: probabilities of prefer- ence of alternatives in each criterion, which are further combined in a final probability of preference; and the importance parameter of each criterion (capacity). Capacity is calculated to each possible subset of criteria. It is different from criteria weighting because there is not a fixed weight but combined values taking into account the relative importance of each criterion in a specific evaluation. The importance of each criterion is calculated by the Shapley index. Considering the possibility of interaction between sustain- ability categories, the preferences in each dimension can be com- bined into a final probability of preference with the use of the Choquet integral, which is a non-compensatory model (Grabisch et al., 2008). In the present study, CPP Beta was modelled as following: (1) each dimension was assessed independently, assuming interaction between criteria, and using the Choquet integral to aggregate their respective probabilities of preferences; (2) the best alternative in each dimension was selected to the next stage; (3) the criteria which did not significantly distinguish between the selected alter- natives were discarded; only the criteria where there was a mini- mum gap of 0.25 between at least two alternatives were selected for the next step; (4) the probabilities of preferences of selected alternatives (step 2) in the distinguishing criteria of the three dimensions (step 3) were aggregated, again with the use of the Choquet integral to admit interactions, in a final composition of probabilistic preferences. 4. Results and discussion The environmental assessment of scenarios using LCA is sum- marised in Fig. 5. Scenarios B1 and C1, which performed best in most of the impact categories, are very similar to each other. B2 -100% -80% -60% -40% -20% 0% 20% 40% 60% 80% 100% Abiotic depletion Abiotic depletion (fossil fuels) Global warming (GWP100a) Human toxicity Fresh water aquatic ecotox. Marine aquatic ecotoxicity Terrestrial ecotoxicity Acidification B2 to BR B1 to EU Baseline (A) C1 to EU C2 to BR Fig. 5. Results of the LCA for the alternative WEEE system scenarios. Table 3 Minimum probability of preference and classification of alternatives (in a 9-point scale) in the social and economic dimensions. System Economic Social Min. Rank Min. Rank A 0.04766 2 0.01798 5 B1 0.01580 6 0.07203 6 B2 0.05232 6 0.02254 7 C1 0.00770 7 0.01625 5 C2 0.06817 7 0.06190 5 54 R.G. de Souza et al. /Waste Management 57 (2016) 46–56 and C2 are also very similar. Scenario A (baseline) is the worst solu- tion in almost all categories. Observing the LCA results and sensitivity analyses, it is possible to conclude that WEEE collection and transport activities had little contribution to environmental impacts, in comparison to the recy- cling and treatment processes. Based on the life cycle perspective and restricted by the model assumptions (Table 1), all system alternatives present benefits in almost all environmental impact categories, except for system A in categories related to fresh water aquatic ecotoxicity and terrestrial ecotoxicity, which is related to contamination by inadequate disposal of hazardous material. Dif- ferences in the three ecotoxicity categories are explained by the large variations of characterisation factors of determined sub- stances in the Life Cycle Inventory (e.g. Hydrogen fluoride has a 1E+7 highest impact potential for marine toxicity than for fresh water toxicity). On the other hand, due to the large uncertainties, no available method is recommended to address marine and terrestrial ecotox- icity (JRC, 2011) and no conclusion should be taken about those categories. The large amount of copper, zinc and nickel released into soil in the informal recycling contributes to the major impacts on the fresh water toxicity in baseline system. Results indicate that recycling PCBs in Europe is more positive to the environment than recycling regionally, considering the assumption that Brazil would adopt a technology similar to the Chinese (Table 1). We have to consider that Brazil is still developing a national technology to recycle PCBs, and when this technology and national data is made available, those results may be changed. These results alsopointout that the collectionschemewithWEEE delivery only at EEE shops is environmentally better than the hybrid scheme with metro and neighbourhood stations. The sensitivity related to assumed distances was analysed for 50% variation, and there was no significant difference (<5%) to the final LCA results. The qualitative assessment of social and economic categories, as it could be expected, presented significant variations in the evalu- ations of the experts. This is mainly due to subjectivity in the inter- pretation of system alternatives and impact categories, though the inquiry process searches to minimise this effect with clear explanations. CPP was applied to integrate the criteria within each dimension, using the Choquet integral, and the minimum and maximum prob- abilities of preference for each alternative in each criterion were calculated. Table 3 presents the results of this round. In the eco- nomic dimension, C1 and C2 were the best alternatives (class 7 of 9); in the social dimension, B2 is the most recommended system. Applying CPP to the environmental dimension, B1 is the best- ranked alternative. In the social dimension, B2 is the best alterna- tive, and in the economic assessment, C1 and C2 (economic) are the highest ranked alternatives (Table 4). These four alternatives were selected to the next assessment stage, which is the integra- tion of the three dimensions, adopting only the criteria that pro- voked a considerable distinction of the alternatives in each dimension (at least 0.25 between two alternatives). Results of this final stage are summarised in Table 4. Table 4 Final ranking of alternatives. System Maximum probabilities of preferences (shapley values of criteria) Final rank (%) 0.1621 0.1546 0.1885 0.1819 0.1950 0.1179 Acidification Resource depletion System efficiency 2 Innovation 2 Social inclusion 2 Health risks 2 B1 18.60% 18.74% 13.82% 13.24% 12.88% 29.56% 21.73% B2 36.04% 36.70% 12.84% 34.32% 24.89% 19.66% 33.41% C1 16.10% 19.55% 33.80% 15.59% 21.27% 34.94% 29.04% C2 29.27% 25.01% 39.54% 36.85% 40.95% 15.84% 38.36% R.G. de Souza et al. /Waste Management 57 (2016) 46–56 55 Adopting the CPP approach described in Section 3, C2 was con- sidered the best WEEE management system to be implemented in Rio de Janeiro. This system consists of a hybrid WEEE collection system with delivery points at EEE shops, metro stations and neighbourhood centres, and comprehends recycling of all compo- nents (PCBs included) regionally in Brazil. This solution is different than that proposed by Manhart (2011) for developing countries, which consisted of exporting PCBs to developed countries (more similar to B1 or C1). This can be explained by the different approaches adopted to assess the alter- natives. Whilst the method by Manhart (2011) was based on the author’s judgements in six decision criteria (Section 2.5), the cur- rent research adopted a framework fully based on stakeholder per- spectives and experts evaluations, fully designed for the specific Brazilian context. On the other hand, B1 or C1 could be an interme- diate step for WEEE management implementation in the country, before the installation of adequate technology to recycle PCBs. The adopted methodology in this study can be considered ade- quate to the kind of problem that is under consideration – lack of data and few evaluators available. Positive points are the integra- tion of LCA with qualitative evaluations, the assumption of interac- tions between criteria, the non-compensatory aggregation, and the presentation of results as probabilities of preference. A limitation of the approach is that N > 3 is a mathematical requirement for the equations, but does not guarantee as precise results as could be obtained with statistical inference and a larger sample of experts. This approach can be a feasible solution to similar decision-support problems with both quantitative data and qualitative assessment by a small sample of evaluators. 5. Conclusions The present paper developed and applied an approach to priori- tise alternative WEEE management systems in Brazil according to their sustainability performances, comprehending the integration of LCA with qualitative evaluations, and considering a small sample of evaluators. This multicriteria approach, which integrated LCA, CPP and qualitative measures, can be useful in cases where there is a lack of data and a small number of evaluators with knowledge of the problem. However, precision of results could only be ensured with the use of statistical inference, which requires a lar- ger sample of respondents. So, it adds scientific value to waste management literature and practice by proposing a much needed method that integrates LCA, qualitative evaluation and MCDA, and that does not require a large number of evaluating experts. Another contribution of this paper is suggesting the most ade- quate WEEE take-back system for the case of Rio de Janeiro, with basis on a sustainability assessment of available alternatives. The recommended solution is a hybrid collection system with delivery points at EEE shops, metro stations and neighbourhoods, integra- tion of social enterprises and cooperatives in the pre-treatment processes, and adequate recycling of all components in the coun- try. Achieving this scenario, however, may require a progressive implementation, in which a starting point could be organising collection and pre-treatment phases and avoiding informality and landfilling, but temporarily exporting PCBs to developed coun- tries with adequate technologies. Further steps in this research are: to make qualitative evalua- tions with a large sample of both national and international experts, in order to allow for statistical inference; to apply different MCDA methods using the same collected data, in order to test their performances in such a decision problem; to collect primary data for more precise evaluations, both for the environmental assess- ment (LCA) and social and economic indicators. Another possibility is to integrate GIS with the methodology in search for logistics optimisation (routing and siting). Acknowledgement The first author thanks the support from CAPES-Ministry of Education, Brazil. References ABDI, 2012. Logística Reversa de Equipamentos Eletroeletrônicos Análise de Viabilidade Técnica e Econômica. ABDI, Brasília. ABINEE, 2015. Economic Panorama and Sectorial Performance – 2015. Brazilian Association of Electrical and Electronic Industry–ABINEE. Available in Portuguese. . Al Sabbagh, M.K., Velis, C.A., Wilson, D.C., Cheeseman, C.R., 2012. Resource management performance in Bahrain: a systematic analysis of municipal waste management, secondary material flows and organizational aspects. Waste Manage. Res. 30 (8), 813–824. http://dx.doi.org/10.1177/ 0734242X12441962. Araújo, M.G., Magrini, A., Mahler, C.F., Bilitewski, B., 2012. A model for estimation of potential generation of waste electrical and electronic equipment in Brazil. Waste Manage. 32 (2), 335–342. http://dx.doi.org/10.1016/j. wasman.2011.09.020. Barboza, M.R., Gonçalves, R.F., D’Onofrio, E., Costa, I., 2014. Reverse logistics of information and communication technology equipment: a comparative assessment of laws and programs. In: IFIP WG 5.7 International Conference on Advances in Production Management Systems, APMS 2014, 439, 2, pp. 114– 121. http://dx.doi.org/10.1007/978-3-662-44736-9_14. Bigum, M., Brogaard, L., Christensen, T.H., 2012. Metal recovery from high-grade WEEE: a life cycle assessment. J. Hazard. Mater. 207–208, 8–14. http://dx.doi. org/10.1016/j.jhazmat.2011.10.001. Brandmann, C., Altvater, M., 2012. Growing regulatory complexity as challenge – strategies for managing compliance with international waste regulations. Paper presented at the Electronics Goes Green 2012+, ECG 2012 – Joint International Conference and Exhibition, Proceedings. Brazil, 2015. Ministério do Meio Ambiente. Desafios e oportunidades da logística reversa como instrumento da política nacional de resíduos sólidos. In: 6o Forum Internacional de Resíduos Sólidos, São José dos Campos. Brazil, Presidência da Republica, 2010. National Solid Waste Policy (Politica Nacional de Residuos Solidos – Lei No. 12.305/2010) (in Portuguese). Campos, T.R.T., Fonseca, M.V.A., Morais, R.M.N., 2014. Reverse logistics: a route that only makes sense when adopting a systemic vision. WIT Trans. Ecol. Environ. 180, 41–52. http://dx.doi.org/10.2495/WM140041. Chatterjee, S., Kumar, K., 2009. Self sustained model of electronic waste management and recycling. Electron. Inform. Plann. 37 (3-4), 57–69, . Cinelli, M., Coles, S.R., Kirwan, K., 2014. Analysis of the potentials of multi criteria decision analysis methods to conduct sustainability assessment. Ecol. Ind. 46, 138–148. http://dx.doi.org/10.1016/j.ecolind.2014.06.011. Classen, M., Althaus, H.-J., Blaser, S., Tuchschimid, M., Jungbluth, N., Doka, G., Faist Emmenegger, M., Sharnhorst, W., 2009. Life cycle inventory of metals. Final Report Data v2.1, No 10. EMPA dubendorf, Swiss Centre for Life Cycle Inventories, Dubendof, CH. http://refhub.elsevier.com/S0956-053X(16)30033-2/h0005 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0005 http://www.abinee.org.br/programas/50anos/public/panorama/#20/z http://www.abinee.org.br/programas/50anos/public/panorama/#20/z http://dx.doi.org/10.1177/0734242X12441962 http://dx.doi.org/10.1177/0734242X12441962 http://dx.doi.org/10.1016/j.wasman.2011.09.020 http://dx.doi.org/10.1016/j.wasman.2011.09.020 http://dx.doi.org/10.1007/978-3-662-44736-9_14 http://dx.doi.org/10.1016/j.jhazmat.2011.10.001 http://dx.doi.org/10.1016/j.jhazmat.2011.10.001 http://dx.doi.org/10.2495/WM140041 http://www.scopus.com http://www.scopus.com http://dx.doi.org/10.1016/j.ecolind.2014.06.011 56 R.G. de Souza et al. /Waste Management 57 (2016) 46–56 COMLURB, 2013. Caracterizações Gravimétrica e Bacteriológica de Resíduos Sólidos Domiciliares da Cidade do Rio de Janeiro – 2013. Rio de Janeiro. Doka, G., 2009. Life cycle inventories of waste treatment services. Ecoinvent Report No. 13. Swiss Centre for Life Cycle Inventories, Dubendorf. Eiselt, H.A., Marianov, V., 2015. Location modeling for municipal solid waste facilities. Comput. Oper. Res. 62, 305–315. http://dx.doi.org/10.1016/ j.cor.2014.05.003. Franco, R.G.F., Lange, L.C., 2011. Flow of e-waste at the city of belo horizonte, minas gerais, brazil. [Estimativa do fluxo dos resíduos de equipamentos eleiacute; tricos e eletrônicos no município de Belo Horizonte, Minas Gerais, Brasil]. Engenharia Sanitaria e Ambiental 16 (1), 73–82. http://dx.doi.org/10.1590/ S1413-41522011000100011. Gamberini, R., Gebennini, E., Manzini, R., Ziveri, A., 2010. On the integration of planning and environmental impact assessment for a WEEE transportation network—a case study. Resour. Conserv. Recycl. 54 (11), 937–951. http://dx.doi. org/10.1016/j.resconrec.2010.02.001. Glew, D., Stringer, L.C., McQueen-Mason, S., 2013. Achieving sustainable biomaterials by maximising waste recovery. Waste Manage. 33 (6), 1499– 1508. http://dx.doi.org/10.1016/j.wasman.2013.03.005. Grabisch, M., Kojadinovic, I., Meyer, P., 2008. A review of methods for capacity identification in choquet integral based multi-attribute utility theory. Applications of the kappalab R package. Eur. J. Oper. Res. 186 (2), 766–785. http://dx.doi.org/10.1016/j.ejor.2007.02.025. Guarnieri, P., Sobreiro, V.A., Nagano, M.S., Marques Serrano, A.L., 2014. The challenge of selecting and evaluating third-party reverse logistics providers in a multicriteria perspective: a Brazilian case. J. Cleaner Prod. 96, 209–219. http:// dx.doi.org/10.1016/j.jclepro.2014.05.040. Hamzeh, M., Ali, Abbaspour R., Davalou, R., 2015. Raster-based outranking method: a new approach for municipal solid waste landfill (MSW) siting. Environ. Sci. Pollut. Res. 22 (16), 12511–12524. http://dx.doi.org/10.1007/s11356-015- 4485-8. Hirayama, D., Saron, C., 2015. Characterisation of recycled acrylonitrile–butadiene– styrene and high-impact polystyrene fromwaste computer equipment in Brazil. Waste Manage. Res. 33 (6), 543–549. http://dx.doi.org/10.1177/ 0734242X15584845. Hischier, R., Classen, M., Lehmann, M., Sharnhorst, W., 2007. Life Cycle inventories of electric and electronic equipment: production, use and disposal. Ecoinvent Report No. 18. EMPA/Technology & Society Lab, Swiss Centre for Life Cycle Inventories, Dubendorf, 2007. Hong, J., Shi, W., Wang, Y., Chen, W., Li, X., 2015. Life cycle assessment of electronic waste treatment. Waste Manage. 38, 357–365. http://dx.doi.org/10.1016/j. wasman.2014.12.022. Huisman, J., Delgado, C., Magalini, F., Kuehr, R., 2008. 2008 Review of Directive 2002/96 on Waste Electrical and Electronic Equipment (WEEE). United Nations University, Bonn. ISO, 2006. Environmental Management–Life Cycle Assessment – General Principles and Framework. ISO 14040. ISO, Geneva. Joint Research Centre, 2011. International Reference Life Cycle Data System (ILCD) Handbook – Recommendations for Life Cycle Impact Assessment in the European Context. Publications Office of the European Union, Luxemburg. Khalili, N.R., Ehrlich, D., 2013. A qualitative multi-criteria, multi stakeholder decision making tool for sustainable waste management. Prog. Ind. Ecol. 8 (1/ 2), 114. Kiddee, P., Naidu, R., Wong, M.H., 2013. Electronic waste management approaches: an overview. Waste Manage. 33 (5), 1237–1250. http://dx.doi.org/10.1016/j. wasman.2013.01.006. Laurent, A., Bakas, I., Clavreul, J., Bernstad, A., Niero, M., Gentil, E., Hauschil, M., Christensen, T.H., 2014a. Review of LCA studies of solid waste management systems – part I: lessons learned and perspectives. Waste Manage. 34 (3), 573– 588. http://dx.doi.org/10.1016/j.wasman.2013.10.045. Laurent, A., Clavreul, J., Bernstad, A., Bakas, I., Niero, M., Gentil, E., et al., 2014b. Review of LCA studies of solid waste management systems – part II: methodological guidance for a better practice. Waste Manage. 34 (3), 589– 606. http://dx.doi.org/10.1016/j.wasman.2013.12.004. Leme, M.M.V., Rocha, M.H., Lora, E.E.S., Venturini, O.J., Lopes, B.M., Ferreira, C.H., 2014. Techno-economic analysis and environmental impact assessment of energy recovery from municipal solid waste (MSW) in Brazil. Resour. Conserv. Recycl. 87, 8–20. http://dx.doi.org/10.1016/j.resconrec.2014.03.003. Liu, H.-C., Wu, J., Li, P., 2013. Assessment of health-care waste disposal methods using a VIKOR-based fuzzy multi-criteria decision making method. Waste Manage. 33 (12), 2744–2751. http://dx.doi.org/10.1016/j.wasman.2013.08.006. Lundgren, K., 2012. The global impact of e-waste addressing the challenge. International Labour Office, Programme on Safety and Health at Work and the Environment (SafeWork), Sectoral Activities Department (SECTOR), Geneva. Manhart, A., 2011. International cooperation for metal recycling from waste electrical and electronic equipment: an assessment of the ‘‘best-of-two- worlds” approach. J. Ind. Ecol. 15 (1), 13–30. http://dx.doi.org/10.1111/j.1530- 9290.2010.00307.x. Mendes, M.R., Aramaki, T., Hanaki, K., 2003. Assessment of the environmental impact of management measures for the biodegradable fraction of municipal solid waste in São Paulo city. Waste Manage. 23 (5), 403–409. http://dx.doi.org/ 10.1016/S0956-053X(03)00058-8. Munda, G., 2008. Social Multi-Criteria Evaluation for a Sustainable Economy. Springer, Heidelberg. Nichols, A., Grose, J., Bennallick, M., Richardson, J., 2013. Sustainable healthcare waste management: a qualitative investigation of its feasibility within a county in the south west of England. J. Infect. Prev. 14 (2), 60–64. http://dx.doi.org/ 10.1177/1757177412471411. Oliveira, C.R.D., Bernardes, A.M., Gerbase, A.E., 2012. Collection and recycling of electronic scrap: a worldwide overview and comparison with the Brazilian situation. Waste Manage. 32 (8), 1592–1610. http://dx.doi.org/10.1016/j. wasman.2012.04.003. Ongondo, F.O., Williams, I.D., Cherrett, T.J., 2011. How are WEEE doing? A global review of the management of electrical and electronic wastes. Waste Manage. 31 (4), 714–730. http://dx.doi.org/10.1016/j.wasman.2010.10.023. Pimentel, M., Rocha, J., Filho, S.E., Rocha, T.B., Moraes, D., 2013. Ambientronic: a Brazilian program to support the development of innovative projects in e-waste reverse logistics. In: Paper Presented at the Proceedings of the 5th International Conference on Management of Emergent Digital EcoSystems, MEDES 2013, pp. 285–288. http://dx.doi.org/10.1145/2536146.2536188. Quariguasi Frota Neto, J., Van Wassenhove, L.N., 2013. Original equipment manufacturers’ participation in take-back initiatives in Brazil: an analysis of engagement levels and obstacles quariguasi and van wassenhove OEM participation in take-back initiatives in Brazil. J. Ind. Ecol. 17 (2), 238–248. http://dx.doi.org/10.1111/jiec.12019. Reichert, G.A., Mendes, C.A.B., 2014. Life cycle assessment and decision making support in integrated and sustainable municipal solid waste management. [Avaliação do ciclo de vida e apoio à decisão em gerenciamento integrado e sustentável de resíduos sólidos urbanos]. Engenharia Sanitaria e Ambiental 19 (3), 301–313. http://dx.doi.org/10.1590/S1413-41522014019000001145. Rousis, K., Moustakas, K., Malamis, S., Papadopoulos, a., Loizidou, M., 2008. Multi- criteria analysis for the determination of the best WEEE management scenario in Cyprus. Waste Manage. 28 (10), 1941–1954. http://dx.doi.org/10.1016/j. wasman.2007.12.001. Rubin, R.S., Castro, M.A.S.D., Brandão, D., Schalch, V., Ometto, A.R., 2014. Utilization of life cycle assessment methodology to compare two strategies for recovery of copper from printed circuit board scrap. J. Cleaner Prod. 64, 297–305. http://dx. doi.org/10.1016/j.jclepro.2013.07.051. Saavedra, Y.M.B., & Ometto, A.R., 2012. E-waste in the Brazilian context. Paper presented at the Electronics Goes Green 2012+, ECG 2012 – Joint International Conference and Exhibition, Proceedings. Sant’Anna, L.T., Machado, R.T.M., De Brito, M.J., 2014. E-waste in Brazil and abroad: legal differences and the urgent need of a global standardization. [Os resíduos eletroeletrônicos no Brasil e no exterior: Diferenças legais e a premência de uma normatização mundial]. Revista De Gestao Social e Ambiental 8 (1), 37–53. Sant’Anna, A.P., 2015. Probabilistic Composition of Preferences, Theory and Applications. Springer, Heidelberg. Sao Paulo, 2015. RESOLUÇÃO SMA No 45, DE 23 DE JUNHO DE 2015: define as diretrizes para implementação e operacionalização da responsabilidade pósconsumo no Estado de São Paulo, e dá providências correlatas. Secretary of Environment, pp. 1–3. Soltani, A., Hewage, K., Reza, B., Sadiq, R., 2015. Multiple stakeholders in multi- criteria decision-making in the context of Municipal Solid Waste Management: a review. Waste Manage. 35, 318–328. http://dx.doi.org/10.1016/j. wasman.2014.09.010. Souza, R.G., Rosenhead, J., Lins, M.P.E., Valle, R.A.B., 2013. Multiple perspectives on waste management systems: towards more effective models for decision support. In: Proceedings Sardinia 2013, Fourteenth International Waste Management and Landfill Symposium. Souza, R.G., 2014. Modelling and Sustainability Assessment of Waste Management Systems Based on Multiple Stakeholders’ Perspectives. PhD Thesis, Program of Production Engineering, Federal University of Rio de Janeiro. Souza, R.G., Rosenhead, J., Salhofer, S.P., Valle, R.A.B., Lins, M.P.E., 2015. Definition of sustainability impact categories based on stakeholder perspectives. J. Cleaner Prod. 105, 41–51. http://dx.doi.org/10.1016/j.jclepro.2014.09.051. StEP, 2015. E-Waste World Map. (acessed September 12th, 2015). SWICO, 2013. Technical Report 2013. . Troschinetz, A.M., Mihelcic, J.R., 2009. Sustainable recycling of municipal solid waste in developing countries. Waste Manage. 29 (2), 915–923. http://dx.doi. org/10.1016/j.wasman.2008.04.016. Valle, R.A.B., Clímaco, J.C.N., 2015. A new tool to facilitate quantitative assessment of green activities – A trial application for Rio de Janeiro. Technol. Forecast. Soc. Change 98, 336–344. http://dx.doi.org/10.1016/j.techfore.2015.03.00. Veloso, Z.M.F., 2015. Desafios e oportunidades da logística reversa como instrumento da política nacional de resíduos sólidos. In: 6� Forum Internacional de Resíduos Sólidos. São José dos Campos, 2015. . Wäger, P.A., Hischier, R., Eugster, M., 2011. Environmental impacts of the swiss collection and recovery systems for waste electrical and electronic equipment (WEEE): a follow-up. Sci. Total Environ. 409 (10), 1746–1756. http://dx.doi.org/ 10.1016/j.scitotenv.2011.01.050. Wibowo, S., Deng, H., 2015. Multi-criteria group decision making for evaluating the performance of e-waste recycling programs under uncertainty. Waste Manage. 40, 127–135. http://dx.doi.org/10.1016/j.wasman.2015.02.035. http://dx.doi.org/10.1016/j.cor.2014.05.003 http://dx.doi.org/10.1016/j.cor.2014.05.003 http://dx.doi.org/10.1590/S1413-41522011000100011 http://dx.doi.org/10.1590/S1413-41522011000100011 http://dx.doi.org/10.1016/j.resconrec.2010.02.001 http://dx.doi.org/10.1016/j.resconrec.2010.02.001 http://dx.doi.org/10.1016/j.wasman.2013.03.005 http://dx.doi.org/10.1016/j.ejor.2007.02.025 http://dx.doi.org/10.1016/j.jclepro.2014.05.040 http://dx.doi.org/10.1016/j.jclepro.2014.05.040 http://dx.doi.org/10.1007/s11356-015-4485-8 http://dx.doi.org/10.1007/s11356-015-4485-8 http://dx.doi.org/10.1177/0734242X15584845 http://dx.doi.org/10.1177/0734242X15584845 http://dx.doi.org/10.1016/j.wasman.2014.12.022 http://dx.doi.org/10.1016/j.wasman.2014.12.022 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0135 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0135 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0135 http://refhub.elsevier.com/S0956-053X(16)30033-2/h9005 http://refhub.elsevier.com/S0956-053X(16)30033-2/h9005 http://refhub.elsevier.com/S0956-053X(16)30033-2/h9005 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0145 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0145 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0145 http://dx.doi.org/10.1016/j.wasman.2013.01.006 http://dx.doi.org/10.1016/j.wasman.2013.01.006 http://dx.doi.org/10.1016/j.wasman.2013.10.045 http://dx.doi.org/10.1016/j.wasman.2013.12.004 http://dx.doi.org/10.1016/j.resconrec.2014.03.003 http://dx.doi.org/10.1016/j.wasman.2013.08.006 http://dx.doi.org/10.1111/j.1530-9290.2010.00307.x http://dx.doi.org/10.1111/j.1530-9290.2010.00307.x http://dx.doi.org/10.1016/S0956-053X(03)00058-8 http://dx.doi.org/10.1016/S0956-053X(03)00058-8 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0190 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0190 http://dx.doi.org/10.1177/1757177412471411 http://dx.doi.org/10.1177/1757177412471411 http://dx.doi.org/10.1016/j.wasman.2012.04.003 http://dx.doi.org/10.1016/j.wasman.2012.04.003 http://dx.doi.org/10.1016/j.wasman.2010.10.023 http://dx.doi.org/10.1145/2536146.2536188 http://dx.doi.org/10.1111/jiec.12019 http://dx.doi.org/10.1590/S1413-41522014019000001145 http://dx.doi.org/10.1016/j.wasman.2007.12.001 http://dx.doi.org/10.1016/j.wasman.2007.12.001 http://dx.doi.org/10.1016/j.jclepro.2013.07.051 http://dx.doi.org/10.1016/j.jclepro.2013.07.051 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0240 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0240 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0240 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0240 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0245 http://refhub.elsevier.com/S0956-053X(16)30033-2/h0245 http://dx.doi.org/10.1016/j.wasman.2014.09.010 http://dx.doi.org/10.1016/j.wasman.2014.09.010 http://dx.doi.org/10.1016/j.jclepro.2014.09.051 http://www.step-initiative.org/step-e-waste-world-map.html http://www.step-initiative.org/step-e-waste-world-map.html http://www.swicorecycling.ch/downloads/dokumente/technical-report-swico-sens-slrs-2013.pdf/1400 http://www.swicorecycling.ch/downloads/dokumente/technical-report-swico-sens-slrs-2013.pdf/1400 http://dx.doi.org/10.1016/j.wasman.2008.04.016 http://dx.doi.org/10.1016/j.wasman.2008.04.016 http://dx.doi.org/10.1016/j.techfore.2015.03.00 http://www.6firs.institutoventuri.org.br/images/apresentacoes/Zilda.pdf http://www.6firs.institutoventuri.org.br/images/apresentacoes/Zilda.pdf http://dx.doi.org/10.1016/j.scitotenv.2011.01.050 http://dx.doi.org/10.1016/j.scitotenv.2011.01.050 http://dx.doi.org/10.1016/j.wasman.2015.02.035 Sustainability assessment and prioritisation of e-waste management options in Brazil 1 Introduction 2 Background knowledge 2.1 E-waste management in the Brazilian National Solid Waste Policy 2.2 E-waste management situation and feasibility assessment in Brazil 2.3 Existing studies in Brazilian e-waste management 2.4 LCA studies in WEEE management 2.5 Qualitative sustainability assessment of waste management 2.6 MCDA sustainability assessment in waste management 2.7 The relevance to waste management of combining MCDA and LCA 3 Materials and methods 3.1 Alternative scenarios under evaluation 3.2 LCA modelling 3.3 Qualitative assessment of social and economic indicators 3.3.1 Social and economic impact categories 3.3.2 Specialists sample, process of inquiry and data compilation 3.4 Multicriteria analysis: composition of probabilistic preferences 4 Results and discussion 5 Conclusions Acknowledgement References