A scoping review of digital open badge ecosystems in relation to resource-constrained environments

The purpose of this article is to present a scoping review for systematically assessing the breadth of available research literature regarding the nature, extent and current understanding of digital open badge ecosystems. The results of the scoping review allow for a deeper understanding and better conceptualisation of implementation-specific valuable lessons which can be applied to a digital open badge ecosystem for a resource-constrained environment. The objectives are to (1) apply a scoping review for the literature study; (2) clarify the specific concepts: digital badges, open badges and digital open badge ecosystems; and (3) indicate how digital open badge ecosystem can be relevant to resource-constrained environments.


Introduction
The purpose of this article is to present a scoping review for systematically assessing the breadth of available research literature regarding the nature, extent and current understanding of digital open badge ecosystems.The results of the scoping review allow for a deeper understanding and better conceptualisation of implementation-specific valuable lessons which can be applied to a digital open badge ecosystem for a resource-constrained environment.The objectives are to (1) apply a scoping review for the literature study; The pursuit of knowledge and skills has steadily and vigorously expanded, forcing pedagogies to follow in its progressive footsteps (Mayrath 2012).The learning environment is no longer limited to the classroom or governed by seat time (Hess 2011;Morrison & DiSalvo 2014).Knowledge and skills are created, shared and valued in various different ways which continuously breach the context in which current education systems were developed and standardised.Alternate learning opportunities include Internet-based projects, self-directed tinkering, community participation and on-the-job training (The Mozilla Foundation 2012).The knowledge, achievements and skills earned outside the traditional formal education system are often either overlooked or not accredited (Mudavanhu 2015;Rooyen 2011).Outcomes, such as achievements in after-school classes, extra-curricular activities, work-based experience and mentoring, to name but a few, are often not recognised in formal education accreditation and therefore seldom acknowledged as a valuable accomplishment.This article addressed the following research question: How can a scoping review on digital open badge ecosystems allow for the conceptualisation of such a system for resource-constrained environments?
Badges are often used to recognise learning (that which an individual knows), skill (that which an individual can do) or a role in an organisation or community (that which an individual has become) (Randall, Harrison & West 2013).Gibson et al. (2015) argue that badges and badging systems are emerging phenomena used to incentivise learners to engage in positive learning behaviours, identify progress in learning and content trajectories as well as signify learning and This article presents a scoping review of digital open badge ecosystems.A scoping review can be seen as a methodology for systematically assessing the breadth of available research literature.This article is aimed at identifying the nature, extent and current understanding of digital open badge ecosystems.Scoping reviews follow many of the same methodological steps as systematic reviews where the reliability of results and the potential for replication remain essential.As such, this review follows the eight steps outlined by Okoli and Schabram.From the 1608 studies identified, 41 were considered relevant for inclusion criteria and thus included in a final sample.Some relevant websites and the so-called grey literature were included.These were evaluated according to the inclusion criteria and were categorised.

Read online:
Scan this QR code with your smart phone or mobile device to read online.
achievement.Digital badges are icons and image files which serve as visual metaphors for skills, roles, accomplishments, experiences and interests (Gibson et al. 2015) and have been touted as a common currency to signify learning outcomes across all contexts (Bowen & Thomas 2014).
In 2011, the Mozilla Foundation, with the support of the MacArthur Foundation, launched the Open Badges Project (Mozilla Foundation 2011).Mozilla Open Badges offers a free, open-source digital badge infrastructure to allow institutions, endorsers and learning providers to issue badges and for job seekers and learners to collect and display said badges (Muilenburg & Berge 2016).The Open Badges Project provides an accreditation system which intends to unlock new career and educational opportunities by promoting the recognition of skills and achievements earned through formal and informal learning (Knight et al. 2014).
The following section offers an overview of the methodology followed.
Method Brien et al. (2010), applying the argument of Arksey and O'Malley (2005), suggest that scoping reviews are a relatively new research tool through which domain-specific literature overviews can be performed.They posit that scoping reviews are similar to systematic reviews in some respects, as they provide a method through which a body of literature can be consolidated.Brien et al. (2010) provide a number of distinguishing characteristics of scoping reviews in comparison to traditional systematic reviews (see Table 1).
Table 1 indicates the differences between systematic and scoping reviews.Brien et al. (2010) highlight the focused nature of the systematic review in that it mostly attempts to answer a clearly defined question through the use of explicit methodologies to assess the quality of included articles.Scoping reviews, on the other hand, are by and large conducted to examine the extent, range and nature of research activity in a particular field Brien et al. (2010:2).Scoping reviews thus focus on the research findings (Lambert 2006) and not on how the literature was obtained (Arskey & O'Malley 2005) or an assessment of its quality (Sandelowski, Docherty & Emden 1997).
Scoping reviews follow many of the same methodological steps as systematic reviews including the essential tenets of result reliability and replication potential (Glasziou et al. 2001;Linde 2002;White & Schmidt 2005) This was particularly relevant to address the purpose and objectives of this article.

Data sources
After taking the purpose of the scoping review into consideration, search keywords were identified and used.These keywords were 'digital badges', 'digital badge ecosystem' and 'open badges'.
The keywords were traced using Google Trends (Figure 1) to identify a search data range.Premier rendered results with a pedagogical implementation in professional practice and was thus deemed outside the scope of the study.The search was consequently extended by not limiting it to specific discipline or database and using frequently cited publications as a measure of credibility.
The keyword search was redone using Harzing's Publish or Perish software (Harzing 2010), which offered a substantial set of publications as well as the number of citations per publications.A high number of citations suggest that a publication has impacted significantly the field.A low number of citations may, but not necessarily, suggest a limited current impact.A low citation per publication metric can be attributed to a limited or recently developed working field or a restricted audience as a result of the article having been published in a language other than English (Harzing 2010).The use of Harzing's The publish or Perish Book (2010) allowed the research to assess the impact of the publication and thus the resultant conceptualisation of a domain understanding.The assumption made was that articles, which were cited more often, were inclined to influence this domain understanding.Researcher bias is acknowledged in this step of the process, as the researcher had to make judgement calls regarding the perceived value of specific articles.This value call was guided by the h factor (Harzing 2010) and the research purpose.The keyword search yielded 1586 results as shown in Table 3.
After removing duplicate studies, the remainder were screened for eligibility using a method similar to systematic review.Studies were assessed for eligibility based on responses to the questions detailed in Table 4.It was decided to limit the study language to English, as it is the home language of all three authors and the cost of translating the paper exceeded both budget and time constraints.If the publication was not written in English and the response to question 1 thus 'No', it was immediately excluded.If the response to any of questions 2-5 was 'Yes', the publication was deemed relevant and shortlisted for quality appraisal as indicated in Table 4.
Approximately 58% of the articles related to Bio Med Central (BMC) Public Health, which presented limited relevance to the research purpose.Hence, publications were excluded if they were not written in English, if they fell in the BMC Public Health field and if the content was not pertinent to the research purpose (see Figure 2).
The remaining 167 eligible studies were examined in detail.
The quality appraisal of the publication was not considered as an inclusion criterion; however, the publications were broadly assessed for relevance.After screening for duplicates and eligibility, the quality appraisal identified 41 publications which were then used in the final review.Figure 3, the PRISMA four-phase flow diagram as relevant to this research, illustrates the flow of information through the phases of searching, evaluating and filtering.
A standard coding template was used to extract data from each of the subsequent selected 41 publications.Four  In an effort to systematically assess the breadth of available research literature and to identify the nature, extent and current understanding of digital open badge ecosystems, a summary of the publications and their focus areas is presented in Table 5 under the four subcategories that emerged.
A qualitative descriptive approach as described by Sandelowski (2000) and applied by Weeks and Strudsholm (2008) was used to review the results, grouping together statements that linked to each of the identified subcategories (Sandelowski, Barroso & Voils 2007).
The outcome is presented and discussed in the following sections.

Results
This section presents a synthesis of the statements linked to the identified subcategories: digital badges, open digital badges, elements of a digital open badge ecosystem and reported implementations.
Mind maps were used to organise and summarise the most important points and concepts.Having met stricter inclusion criteria, the 41 publications were processed using the guidelines outlined by Ellis and Levy (2010:192), namely know the literature, comprehend the literature, apply, analyse, synthesise and evaluate.These mind maps are presented in Figures 4 and 5 with literature linked through the article number as allocated in Table 5.
Digital badges Devedžić and Jovanović (2015) argue that a badge is an image that serves as an indicator of skills, competencies, interests, achievements or hierarchy acquired over time and across all contexts.The affordances of digital badges can be categorised into four key areas: • motivation -to foster discovery, promote engagement, drive the acquisition of knowledge and incentivise learning (Gibson et   Digital badges can be embedded with metadata which provide information regarding the issuer, recipient and why the badge had been awarded, along with the associated evidence.The granularity associated with awarding badges for competencies creates a broader representation of an individual's capabilities, thus allowing him or her the opportunity to present a more complete picture of his or her competencies by signalling specific critical skills.
The value of the digital badge is backed by the issuing authority and the decided assessment of recognition.
A competence-or educational-based assessment must be linked to evidence of activities, learning, experiences, artefacts and skills development (Gibson et al. 2015:404;Mayrath 2012:46).This is an important issue for resource-constrained environments as well (objectives 2 and 3 of the article).Badges are issued in accordance with success criteria which vary in definition, level, weight, quality, rigour, motivation and reward.A single badge may be designed and issued by more than one authority, making the badge achievable via multiple paths and assessment options.
In conclusion, this article will adopt the following definition for a digital badge: a digital badge is a validated representation or indicator of an accomplishment, competency, interest, affiliation, experience or skill that is visual and available online (Gibson et al. 2015;Jovanović & Devedžić 2015).

Open badges
Building on the merits and potential of digital badges and the badging systems, Mozilla and the MacArthur Foundation developed the Open Badges Project.This badge system displays the following key elements: badges, assessment, collecting and sharing tools as well as criteria and evidence.Mozilla Open Badges has refined the concept of the digital badge, as a static image, by hard-coding metadata on the badge which communicates the specific skill or achievement (Knight & Mozilla Foundation 2012:8).This provides a level of security and reliability (Randall et al. 2013).The metadata details, illustrated in Figure 6, include: the badge name, issuer, endorser, competency statement, performance criteria, method of assessment, evidence of performance, date issued and standards with which the badge is aligned (Badges/ FAQs -Mozilla Foundation 2014a).

Digital open badge ecosystem elements 1
The literature documenting digital badge ecosystems is relatively scarce and, with the exception of the Mozilla Open Badges ecosystem, consists mainly of single institutionally localised ecosystems.These seem to function within a single institution, enabling the use and application of open digital badges for institutional consumption.
A digital badge ecosystem allows tertiary institutions and employers to bridge the identified skills shortage gap with badge ecosystems that are developed around higher education outcomes (Pearson Education 2013).The Mozilla Foundation, Peer 2 Peer University and MacArthur ( 2012) have indicated that a definition of digital badge ecosystem has to include descriptions of badges, assessment and infrastructure.The infrastructure of a digital badge ecosystem needs to provide an online, open and decentralised vehicle to issue, collect and display badges online.This is operationalised as a badge backpack.
Universities and higher education institutions can leverage the connected learning outcomes to contextualise an individual's achievements and readily demonstrate the return on investment and economic impact of their programmes (Jovanović & Devedžić 2015).Universities and education providers who endorse badges will acknowledge 1.Used under CC-BY-ND License.
and credit the learner's soft skills and job-ready competencies, and therefore increase the transparency of learning pathways (Glover & Latif 2013).A badge ecosystem can help students make better-informed decisions through connected learning pathways between tertiary education providers and the career-specific skills required by employers (Crafford & Matthee 2016;Itow & Hickey 2013).For employers and advanced education providers, the required skills, competencies and outcomes, articulated through badges, will simplify the communication of changing needs to the learners, tertiary institutions and training providers (Pearson Education 2013).As a result, curricula can easily be tailored to suit the market needs and afford learners the opportunity to better equip themselves with a more market-responsive skill set.
The infrastructure must accommodate the issuing of badges from all contexts of learning environments (e.g.schools, universities, online learning, professional bodies, employers, non-formal education and learning providers); the display and management of badges to afford the learner the power to control what badges are publicly displayed; the impact of metadata attached to a badge; the authentication of badges to verify their validity; the interface to external websites to support badge display and the endorsement of badges to signify the weight and value of a badge.
The Open Badge Infrastructure (OBI) defines three user roles for the open badges ecosystem (Mozilla Foundation 2014b) as the badge earner, the badge issuer and the badge displayer.
From the above literature overview, the following components are identified for an open badge ecosystem, which are regarded as the final components that should also be considered for resource-constrained environments as well when using digital badges (see the 'Digital open badges ecosystem for resource-constrained environments' section): • the badge issuer • the badge (badge metadata) • the badge earner • the badging infrastructure (to facilitate the badging process) • the badge displayer.

Existing implementations of digital badges internationally
There are several small-scale, mostly institutional, initiatives which have already implemented open badges to incentivise learning and introduce micro-credentialing to increase the scope of learning recognition:

Digital open badges ecosystem for resourceconstrained environments
The experiences, merits and challenges documented, contain valuable lessons which can be applied to conceptualising a digital open badges ecosystem for a resource-constrained environment.Resource-constrained environments in context are environments characterised by these economic, technological and cultural conditions: low-income communities where power and network connectivity are scarce and expensive with low bandwidth, and which also present unique cultural constraints where people are unfamiliar with technology or are technophobic (Niehaus et al. 2017).The following should be considered: • Randall et al. (2013) stated that the use of expert assessment before a badge is awarded, albeit a reliable evaluation of high quality comes at a high price and financial sustainability should be treated as a major concern.Rosewell (2012) proposed partnering with existing Open Educational Resource (OER) providers and using existing evaluation models to obviate the cost implications of designing a robust validation and assessment rubric.2015), a challenge exists with demonstrating digital badges as a currency of achievement and credibility and not just as a motivational tool.On the contrary, Davis and Singh (2015) reported that the credibility of badge metadata and the gaining of recognition for skills earned across contexts were met with enthusiasm and are a major drawcard for badge adoption.• A shared challenge concerns the assessment element of a badging system, specifically the success criteria for conceptual learning, critical thinking, creativity and other soft skills (Mewburn et al. 2014;Myllymäki & Hakala 2014;Niehaus et al. 2017;Randall et al. 2013;Rosewell 2012).Abramovich, Schunn and Higashi (2013) claimed that the 'potential benefit of an assessment is determined by its ability to both maintain learning motivation and accurately communicate a student's learning'.Competency-based learning, which a badge will validate, has to reflect and validate the true meaning of the competence or skills gained.Reputation is built on valid, verified and quality judgements -not only for badge earners but also for the badges themselves and the medium of exchange within which they have to have currency.The openness of such a system is regarded as essential to long-term sustainability so that exporting and importing of badges can be modified and adapted by the open source community (De Villiers & Sauls 2017;Meyer & Marais 2015;Niehaus et al. 2017).• It is also important that validating institutions and organisations are not attached to commercial constraints when doing validation.The validation process of badges should minimise the amount of data necessary for checking badges whilst also mapping progress, signalling reputation beyond the community where it was earned and incentivising learners to engage in pro-social behaviours (Crafford & Matthee 2016;Jabbar et al. 2016;Muilenburg & Berge 2016).• The ability to set an expiration date on a badge has been beneficial for badges that are awarded for the mastery of skills in a field that is rapidly changing.The badge expiration date, if not carefully determined, can result in a lack of interest, making earners less likely to collect this badge because of its lack of longevity (Myllymäki & Hakala 2014;Randall et al. 2013).
This section addressed the research question of this article.
It is evident that a digital open badge ecosystem can meaningfully contribute to accreditation challenges in resource-constrained environments if the above issues are addressed appropriately.

Conclusion
Knight and Mozilla Foundation (2012:1), the senior director of learning at Mozilla, describes our current accreditation systems as a 'shared monopoly across education where you have to go down a very prescribed path to get learning that quote-unquote counts' and that Mozilla wants to 'open that up'.Digital badges can advance the reimagining of accreditation practices by measuring competencies accurately and with a finer granularity than seat time and formal degrees do.Mozilla's Open Badges offers the infrastructure to create and develop a digital badge ecosystem that incentivises and recognises accomplishments and experiences in all learning contexts.This is particularly relevant for resource-constrained environments as these badges can assist them to show their own credibility on social media.A digital open badges ecosystem can potentially create a connected learning environment where skills, achievements and competencies are fully recognised and accredited in specifically resourceconstrained environments so that, once community members have a digital identity, their paper-based badges can be replaced with digital badges which acknowledge their competency learning.
(2) clarify the specific concepts: digital badges, open badges and digital open badge ecosystems; and (3) indicate how digital open badge ecosystem can be relevant to resource-constrained environments.
Further subcategories were identified which include the notion of digital badges (n = 35) and open digital badges (n = 18), elements of a digital open badge ecosystem (n = 27) and reported implementations (n = 29).The scoping review presents a literature repository that can be used by practitioners and researchers interested in the application of digital open badges for the establishment of a digital open badge ecosystem.The review also serves to highlight the potential benefits this can have for resource-constrained environments.
subcategories emerged: digital badges, open digital badges, elements of a digital open badge ecosystem and reported implementations.

FIGURE 5 :
FIGURE 5: Literature linked to case studies and implementations.

TABLE 1 :
A comparison of the characteristics of scoping and systematic reviews.
The results displayed a shared spike in interest from April 2011 to January 2015.Accordingly, the search was limited to studies, articles and books published between 2011 and 2016.The keyword search strategy used the following search terms: 'digital badges' or 'badge ecosystem' or 'digital badges ecosystem' or 'open badges' and 'framework' or 'implementation'.The word 'framework' was used as a subcategory, as it was found that digital badge frameworks existed and this was also applied to refine the search.The word 'environment' was used as a subcategory and was only applicable to how digital badges were used in resource-constrained environments.An initial search of only educational databases Educational Resource Information Centre (ERIC) (via EBSCOhost), Professional Development Collection, Professional Development Collection and Academic Search

TABLE 3 :
Search terms used in keyword search strategy and returned results.

TABLE 5 :
Summary of publications and focus areas.

:
Overview of digital badges as extracted from the selected literature.

•
(Makewaves 2016safe, social learning platform for schools that provides social media for education to develop digital literacy skills, learn about Internet safety and how to use social media responsibly.It has formed a community of thousands of schools sharing their creativity and raising achievement with badges(Makewaves 2016).
• The design of the digital open badges ecosystem needs to stimulate and sustain motivation in badge earners.