Abstract
This article presents a scoping review of digital open badge ecosystems. A scoping review can be seen as a methodology for systematically assessing the breadth of available research literature. This article is aimed at identifying the nature, extent and current understanding of digital open badge ecosystems. Scoping reviews follow many of the same methodological steps as systematic reviews where the reliability of results and the potential for replication remain essential. As such, this review follows the eight steps outlined by Okoli and Schabram. From the 1608 studies identified, 41 were considered relevant for inclusion criteria and thus included in a final sample. Some relevant websites and the so-called grey literature were included. These were evaluated according to the inclusion criteria and were categorised. Further subcategories were identified which include the notion of digital badges (n = 35) and open digital badges (n = 18), elements of a digital open badge ecosystem (n = 27) and reported implementations (n = 29). The scoping review presents a literature repository that can be used by practitioners and researchers interested in the application of digital open badges for the establishment of a digital open badge ecosystem. The review also serves to highlight the potential benefits this can have for resource-constrained environments.
Introduction
The purpose of this article is to present a scoping review for systematically assessing the breadth of available research literature regarding the nature, extent and current understanding of digital open badge ecosystems. The results of the scoping review allow for a deeper understanding and better conceptualisation of implementation-specific valuable lessons which can be applied to a digital open badge ecosystem for a resource-constrained environment. The objectives are to (1) apply a scoping review for the literature study; (2) clarify the specific concepts: digital badges, open badges and digital open badge ecosystems; and (3) indicate how digital open badge ecosystem can be relevant to resource-constrained environments.
The pursuit of knowledge and skills has steadily and vigorously expanded, forcing pedagogies to follow in its progressive footsteps (Mayrath 2012). The learning environment is no longer limited to the classroom or governed by seat time (Hess 2011; Morrison & DiSalvo 2014). Knowledge and skills are created, shared and valued in various different ways which continuously breach the context in which current education systems were developed and standardised. Alternate learning opportunities include Internet-based projects, self-directed tinkering, community participation and on-the-job training (The Mozilla Foundation 2012). The knowledge, achievements and skills earned outside the traditional formal education system are often either overlooked or not accredited (Mudavanhu 2015; Rooyen 2011). Outcomes, such as achievements in after-school classes, extra-curricular activities, work-based experience and mentoring, to name but a few, are often not recognised in formal education accreditation and therefore seldom acknowledged as a valuable accomplishment.
This article addressed the following research question: How can a scoping review on digital open badge ecosystems allow for the conceptualisation of such a system for resource-constrained environments?
Badges are often used to recognise learning (that which an individual knows), skill (that which an individual can do) or a role in an organisation or community (that which an individual has become) (Randall, Harrison & West 2013). Gibson et al. (2015) argue that badges and badging systems are emerging phenomena used to incentivise learners to engage in positive learning behaviours, identify progress in learning and content trajectories as well as signify learning and achievement. Digital badges are icons and image files which serve as visual metaphors for skills, roles, accomplishments, experiences and interests (Gibson et al. 2015) and have been touted as a common currency to signify learning outcomes across all contexts (Bowen & Thomas 2014).
In 2011, the Mozilla Foundation, with the support of the MacArthur Foundation, launched the Open Badges Project (Mozilla Foundation 2011). Mozilla Open Badges offers a free, open-source digital badge infrastructure to allow institutions, endorsers and learning providers to issue badges and for job seekers and learners to collect and display said badges (Muilenburg & Berge 2016). The Open Badges Project provides an accreditation system which intends to unlock new career and educational opportunities by promoting the recognition of skills and achievements earned through formal and informal learning (Knight et al. 2014).
The following section offers an overview of the methodology followed.
Method
Brien et al. (2010), applying the argument of Arksey and O’Malley (2005), suggest that scoping reviews are a relatively new research tool through which domain-specific literature overviews can be performed. They posit that scoping reviews are similar to systematic reviews in some respects, as they provide a method through which a body of literature can be consolidated. Brien et al. (2010) provide a number of distinguishing characteristics of scoping reviews in comparison to traditional systematic reviews (see Table 1).
| TABLE 1: A comparison of the characteristics of scoping and systematic reviews. |
Table 1 indicates the differences between systematic and scoping reviews. Brien et al. (2010) highlight the focused nature of the systematic review in that it mostly attempts to answer a clearly defined question through the use of explicit methodologies to assess the quality of included articles. Scoping reviews, on the other hand, are by and large conducted to examine the extent, range and nature of research activity in a particular field Brien et al. (2010:2). Scoping reviews thus focus on the research findings (Lambert 2006) and not on how the literature was obtained (Arskey & O’Malley 2005) or an assessment of its quality (Sandelowski, Docherty & Emden 1997).
Scoping reviews follow many of the same methodological steps as systematic reviews including the essential tenets of result reliability and replication potential (Glasziou et al. 2001; Linde 2002; White & Schmidt 2005). Ellis and Levy (2010) define a systematic literature review as a reproducible procedure performed to identify, evaluate and synthesise quality literature using an explicit algorithm. They argue that a systematic literature review, conducted with sufficient scientific rigour, can provide a solid grounding for a research area and the selection of a suitable research methodology.
Scoping review aims to review relevant literature for systematically assessing the breadth of available research literature whilst describing the nature, extent and current understanding of digital open badge ecosystems. It was operationalised through Okoli and Schabram’s (2010) eight steps: (1) identify the purpose, scope and goal; (2) develop a protocol with specific steps and procedures to be followed; (3) search and eliminate studies that do not meet the requirements; (4) screen the retrieved studies according to relevance and defined criteria; (5) screen the remaining studies according to the quality of the articles; (6) systematically extract applicable information; (7) aggregate, analyse, discuss and organise studies; and (8) report on the findings in detail to ensure that the systematic literature review can be independently reproducible.
The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Moher et al. 2009) guidelines, embodied in the four-phase flow diagram, were implemented to guide steps 2–5 of Okoli and Schabram’s (2010) eight-step model (see Table 2).
| TABLE 2: Implementation adapted to Okoli and Schabram (2010) and preferred reporting items for systematic reviews and meta-analyses. |
This was particularly relevant to address the purpose and objectives of this article.
Data sources
After taking the purpose of the scoping review into consideration, search keywords were identified and used. These keywords were ‘digital badges’, ‘digital badge ecosystem’ and ‘open badges’.
The keywords were traced using Google Trends (Figure 1) to identify a search data range. The results displayed a shared spike in interest from April 2011 to January 2015. Accordingly, the search was limited to studies, articles and books published between 2011 and 2016. The keyword search strategy used the following search terms: ‘digital badges’ or ‘badge ecosystem’ or ‘digital badges ecosystem’ or ‘open badges’ and ‘framework’ or ‘implementation’. The word ‘framework’ was used as a subcategory, as it was found that digital badge frameworks existed and this was also applied to refine the search. The word ‘environment’ was used as a subcategory and was only applicable to how digital badges were used in resource-constrained environments. An initial search of only educational databases Educational Resource Information Centre (ERIC) (via EBSCOhost), Professional Development Collection, Professional Development Collection and Academic Search Premier rendered results with a pedagogical implementation in professional practice and was thus deemed outside the scope of the study. The search was consequently extended by not limiting it to specific discipline or database and using frequently cited publications as a measure of credibility. The keyword search was redone using Harzing’s Publish or Perish software (Harzing 2010), which offered a substantial set of publications as well as the number of citations per publications. A high number of citations suggest that a publication has impacted significantly the field. A low number of citations may, but not necessarily, suggest a limited current impact. A low citation per publication metric can be attributed to a limited or recently developed working field or a restricted audience as a result of the article having been published in a language other than English (Harzing 2010). The use of Harzing’s The publish or Perish Book (2010) allowed the research to assess the impact of the publication and thus the resultant conceptualisation of a domain understanding. The assumption made was that articles, which were cited more often, were inclined to influence this domain understanding. Researcher bias is acknowledged in this step of the process, as the researcher had to make judgement calls regarding the perceived value of specific articles. This value call was guided by the h factor (Harzing 2010) and the research purpose. The keyword search yielded 1586 results as shown in Table 3.
 |
FIGURE 1: Google Trends search results for keywords. |
|
| TABLE 3: Search terms used in keyword search strategy and returned results. |
After removing duplicate studies, the remainder were screened for eligibility using a method similar to systematic review. Studies were assessed for eligibility based on responses to the questions detailed in Table 4. It was decided to limit the study language to English, as it is the home language of all three authors and the cost of translating the paper exceeded both budget and time constraints. If the publication was not written in English and the response to question 1 thus ‘No’, it was immediately excluded. If the response to any of questions 2–5 was ‘Yes’, the publication was deemed relevant and shortlisted for quality appraisal as indicated in Table 4.
| TABLE 4: Eligibility assessment questions. |
Approximately 58% of the articles related to Bio Med Central (BMC) Public Health, which presented limited relevance to the research purpose. Hence, publications were excluded if they were not written in English, if they fell in the BMC Public Health field and if the content was not pertinent to the research purpose (see Figure 2).
The remaining 167 eligible studies were examined in detail. The quality appraisal of the publication was not considered as an inclusion criterion; however, the publications were broadly assessed for relevance. After screening for duplicates and eligibility, the quality appraisal identified 41 publications which were then used in the final review. Figure 3, the PRISMA four-phase flow diagram as relevant to this research, illustrates the flow of information through the phases of searching, evaluating and filtering.
 |
FIGURE 3: Preferred reporting items for systematic reviews and meta-analyses flow diagram. |
|
A standard coding template was used to extract data from each of the subsequent selected 41 publications. Four subcategories emerged: digital badges, open digital badges, elements of a digital open badge ecosystem and reported implementations.
In an effort to systematically assess the breadth of available research literature and to identify the nature, extent and current understanding of digital open badge ecosystems, a summary of the publications and their focus areas is presented in Table 5 under the four subcategories that emerged.
| TABLE 5: Summary of publications and focus areas. |
A qualitative descriptive approach as described by Sandelowski (2000) and applied by Weeks and Strudsholm (2008) was used to review the results, grouping together statements that linked to each of the identified subcategories (Sandelowski, Barroso & Voils 2007).
The outcome is presented and discussed in the following sections.
Results
This section presents a synthesis of the statements linked to the identified subcategories: digital badges, open digital badges, elements of a digital open badge ecosystem and reported implementations.
Mind maps were used to organise and summarise the most important points and concepts. Having met stricter inclusion criteria, the 41 publications were processed using the guidelines outlined by Ellis and Levy (2010:192), namely know the literature, comprehend the literature, apply, analyse, synthesise and evaluate. These mind maps are presented in Figures 4 and 5 with literature linked through the article number as allocated in Table 5.
 |
FIGURE 4: Overview of digital badges as extracted from the selected literature. |
|
 |
FIGURE 5: Literature linked to case studies and implementations. |
|
Digital badges
Devedžić and Jovanović (2015) argue that a badge is an image that serves as an indicator of skills, competencies, interests, achievements or hierarchy acquired over time and across all contexts. The affordances of digital badges can be categorised into four key areas:
- motivation – to foster discovery, promote engagement, drive the acquisition of knowledge and incentivise learning (Gibson et al. 2015; Knight & Mozilla Foundation, 2012; Muilenburg & Berge 2016; Randall et al. 2013; Tran, Schenke & Hickey, 2014)
- recognition and credentialing – to validate, measure and accredit knowledge and skills gained across all contexts of learning, to build and formalise an identity and reputation, to symbolise an association with a community or group (Davis & Singh 2015; Gibson et al. 2015; Law et al. 2015; Pedro et al. 2015)
- evidence of achievement – by linking a digital badge with metadata containing evidence of skills, accomplishment or knowledge, creating a granular representation of capabilities, to map an individual’s progress (Hole 2014; Ifenthaler et al. 2016; Pedro et al. 2015)
- research – to form a pathway model for achievement, to make learning more transparent and accessible (Davis & Singh 2015; Sullivan 2013; The Mozilla Foundation, 2012).
Digital badges can be embedded with metadata which provide information regarding the issuer, recipient and why the badge had been awarded, along with the associated evidence. The granularity associated with awarding badges for competencies creates a broader representation of an individual’s capabilities, thus allowing him or her the opportunity to present a more complete picture of his or her competencies by signalling specific critical skills.
The value of the digital badge is backed by the issuing authority and the decided assessment of recognition. A competence- or educational-based assessment must be linked to evidence of activities, learning, experiences, artefacts and skills development (Gibson et al. 2015:404; Mayrath 2012:46). This is an important issue for resource-constrained environments as well (objectives 2 and 3 of the article). Badges are issued in accordance with success criteria which vary in definition, level, weight, quality, rigour, motivation and reward. A single badge may be designed and issued by more than one authority, making the badge achievable via multiple paths and assessment options.
In conclusion, this article will adopt the following definition for a digital badge: a digital badge is a validated representation or indicator of an accomplishment, competency, interest, affiliation, experience or skill that is visual and available online (Gibson et al. 2015; Jovanović & Devedžić 2015).
Open badges
Building on the merits and potential of digital badges and the badging systems, Mozilla and the MacArthur Foundation developed the Open Badges Project. This badge system displays the following key elements: badges, assessment, collecting and sharing tools as well as criteria and evidence. Mozilla Open Badges has refined the concept of the digital badge, as a static image, by hard-coding metadata on the badge which communicates the specific skill or achievement (Knight & Mozilla Foundation 2012:8). This provides a level of security and reliability (Randall et al. 2013). The metadata details, illustrated in Figure 6, include: the badge name, issuer, endorser, competency statement, performance criteria, method of assessment, evidence of performance, date issued and standards with which the badge is aligned (Badges/FAQs – Mozilla Foundation 2014a).
Although learning is similarly validated via standards, evaluation and evidence, in contrast to the current top-down and closed accreditation system, the open badges accreditation system uses a bottom-up, open and distributed approach (Knight & Mozilla Foundation 2012). The Open Badges badging process is outlined below (Mozilla Foundation 2014b):
- A badge issuer creates a certifiable badge and makes it available online to their audience of earners.
- When an earner meets the defined criteria of a badge, the badge is awarded to an earner, and the earner can choose to store the badge in his or her Backpack.
- The Backpack is used to accumulate and manage badges, allowing earners to select privacy and publishing preferences (Goligoski 2012; Niehaus et al. 2017).
- Earners can share and display authenticated badges publicly on social network sites, blogs, profiles and resumes.
- Potential employers, recruiters, organisations and peers can view badges that are displayed publicly and click on the badges to view the badge metadata (Crafford & Matthee 2016; Ifenthaler et al. 2016; Jabbar, Gasser & Lodge 2016).
The following definition is adopted for an open badge: open badges are standards-based badges that have hard-coding metadata on reliability and securely communicating a specific skill or achievement (Knight & Mozilla Foundation 2012; Randall et al. 2013).
Digital open badge ecosystem elements
The literature documenting digital badge ecosystems is relatively scarce and, with the exception of the Mozilla Open Badges ecosystem, consists mainly of single institutionally localised ecosystems. These seem to function within a single institution, enabling the use and application of open digital badges for institutional consumption.
A digital badge ecosystem allows tertiary institutions and employers to bridge the identified skills shortage gap with badge ecosystems that are developed around higher education outcomes (Pearson Education 2013). The Mozilla Foundation, Peer 2 Peer University and MacArthur (2012) have indicated that a definition of digital badge ecosystem has to include descriptions of badges, assessment and infrastructure. The infrastructure of a digital badge ecosystem needs to provide an online, open and decentralised vehicle to issue, collect and display badges online. This is operationalised as a badge backpack.
Universities and higher education institutions can leverage the connected learning outcomes to contextualise an individual’s achievements and readily demonstrate the return on investment and economic impact of their programmes (Jovanović & Devedžić 2015). Universities and education providers who endorse badges will acknowledge and credit the learner’s soft skills and job-ready competencies, and therefore increase the transparency of learning pathways (Glover & Latif 2013). A badge ecosystem can help students make better-informed decisions through connected learning pathways between tertiary education providers and the career-specific skills required by employers (Crafford & Matthee 2016; Itow & Hickey 2013). For employers and advanced education providers, the required skills, competencies and outcomes, articulated through badges, will simplify the communication of changing needs to the learners, tertiary institutions and training providers (Pearson Education 2013). As a result, curricula can easily be tailored to suit the market needs and afford learners the opportunity to better equip themselves with a more market-responsive skill set.
The infrastructure must accommodate the issuing of badges from all contexts of learning environments (e.g. schools, universities, online learning, professional bodies, employers, non-formal education and learning providers); the display and management of badges to afford the learner the power to control what badges are publicly displayed; the impact of metadata attached to a badge; the authentication of badges to verify their validity; the interface to external websites to support badge display and the endorsement of badges to signify the weight and value of a badge.
The Open Badge Infrastructure (OBI) defines three user roles for the open badges ecosystem (Mozilla Foundation 2014b) as the badge earner, the badge issuer and the badge displayer.
From the above literature overview, the following components are identified for an open badge ecosystem, which are regarded as the final components that should also be considered for resource-constrained environments as well when using digital badges (see the ‘Digital open badges ecosystem for resource-constrained environments’ section):
- the badge issuer
- the badge (badge metadata)
- the badge earner
- the badging infrastructure (to facilitate the badging process)
- the badge displayer.
Existing implementations of digital badges internationally
There are several small-scale, mostly institutional, initiatives which have already implemented open badges to incentivise learning and introduce micro-credentialing to increase the scope of learning recognition:
- Makewaves is a safe, social learning platform for schools that provides social media for education to develop digital literacy skills, learn about Internet safety and how to use social media responsibly. It has formed a community of thousands of schools sharing their creativity and raising achievement with badges (Makewaves 2016).
- Kent University has used badges to validate and capture competencies gained through workshops for faculty professional development (Nestor 2014).
- Aurora Public Schools (APS) are working with community colleges and regional employers in the state of Colorado to award badges for 21st century skills and to gain endorsement for these badges (Smore 2016).
- DigitalMe is using the Open Badges platform to work together with teachers, charities and employers to transform the learning landscape and prepare young people with skills and competencies to enable real-world opportunities. Current badge programmes include ‘Badge the UK’ which allows learners to showcase all achievements using digital badges, and ‘Young Carers in Focus’ which empowers young carers to share stories and campaign for change (DigitalMe 2016).
- Newport City (Wales) Homes provides community services to residents and has created a set of badges for key competencies that are linked to industry standards, thus developing and rewarding employee skills (Price 2014).
- The University of California is aware that traditional testing and grading would not accurately reflect the outcomes of its agriculture and food systems curriculum, and linked competency badges to the outcomes defined in collaboration with targeted employers (Muilenburg & Berge 2016).
- The Grading Soft Skills (GRASS) project develops innovative pedagogical approaches to support and represent the soft skills of learners. GRASS is investigating the idea of digital badges and open badges to formally measure, validate and recognise these skills (The GRASS Project 2016).
- Providence after School Alliance (PASA 2016) launched a badge system targeted at the Providence youth to expand and improve learning opportunities by motivating, tracking, recognising and validating learner interests to create connected learning pathways.
- Badge Europe! (2016) is an international initiative working towards promoting open badges as the infrastructure to provide formal and informal learning recognition, to increase the transparency and quality of learning outcomes and to create opportunities for learning, employment and social inclusion.
Digital open badges ecosystem for resource-constrained environments
The experiences, merits and challenges documented, contain valuable lessons which can be applied to conceptualising a digital open badges ecosystem for a resource-constrained environment. Resource-constrained environments in context are environments characterised by these economic, technological and cultural conditions: low-income communities where power and network connectivity are scarce and expensive with low bandwidth, and which also present unique cultural constraints where people are unfamiliar with technology or are technophobic (Niehaus et al. 2017). The following should be considered:
- Randall et al. (2013) stated that the use of expert assessment before a badge is awarded, albeit a reliable evaluation of high quality comes at a high price and financial sustainability should be treated as a major concern. Rosewell (2012) proposed partnering with existing Open Educational Resource (OER) providers and using existing evaluation models to obviate the cost implications of designing a robust validation and assessment rubric.
- The design of the digital open badges ecosystem needs to stimulate and sustain motivation in badge earners. Haaranen et al. (2014) attested that a simple design reduced the learning curve for student badge earners and encouraged learning; however, providing more statistics for earners (e.g. leader boards and notifications about badges earned) might increase motivation. In a pilot test, Pedro et al. (2015) discovered that users of the SAPO Campus (the major Internet portal and ISP in Portugal) badging systems (developed at the University of Aveiro, Portugal) felt more engaged and motivated during participatory learning exercises as a group or community.
- Learners embrace informal learning and the value paid for certification and recognition from a Massive Open Online Learning (MOOC) provider. As experienced by Law et al. (2015), a challenge exists with demonstrating digital badges as a currency of achievement and credibility and not just as a motivational tool. On the contrary, Davis and Singh (2015) reported that the credibility of badge metadata and the gaining of recognition for skills earned across contexts were met with enthusiasm and are a major drawcard for badge adoption.
- A shared challenge concerns the assessment element of a badging system, specifically the success criteria for conceptual learning, critical thinking, creativity and other soft skills (Mewburn et al. 2014; Myllymäki & Hakala 2014; Niehaus et al. 2017; Randall et al. 2013; Rosewell 2012). Abramovich, Schunn and Higashi (2013) claimed that the ‘potential benefit of an assessment is determined by its ability to both maintain learning motivation and accurately communicate a student’s learning’. Competency-based learning, which a badge will validate, has to reflect and validate the true meaning of the competence or skills gained. Reputation is built on valid, verified and quality judgements – not only for badge earners but also for the badges themselves and the medium of exchange within which they have to have currency. The openness of such a system is regarded as essential to long-term sustainability so that exporting and importing of badges can be modified and adapted by the open source community (De Villiers & Sauls 2017; Meyer & Marais 2015; Niehaus et al. 2017).
- It is also important that validating institutions and organisations are not attached to commercial constraints when doing validation. The validation process of badges should minimise the amount of data necessary for checking badges whilst also mapping progress, signalling reputation beyond the community where it was earned and incentivising learners to engage in pro-social behaviours (Crafford & Matthee 2016; Jabbar et al. 2016; Muilenburg & Berge 2016).
- The ability to set an expiration date on a badge has been beneficial for badges that are awarded for the mastery of skills in a field that is rapidly changing. The badge expiration date, if not carefully determined, can result in a lack of interest, making earners less likely to collect this badge because of its lack of longevity (Myllymäki & Hakala 2014; Randall et al. 2013).
This section addressed the research question of this article. It is evident that a digital open badge ecosystem can meaningfully contribute to accreditation challenges in resource-constrained environments if the above issues are addressed appropriately.
Conclusion
Knight and Mozilla Foundation (2012:1), the senior director of learning at Mozilla, describes our current accreditation systems as a ‘shared monopoly across education where you have to go down a very prescribed path to get learning that quote-unquote counts’ and that Mozilla wants to ‘open that up’. Digital badges can advance the reimagining of accreditation practices by measuring competencies accurately and with a finer granularity than seat time and formal degrees do. Mozilla’s Open Badges offers the infrastructure to create and develop a digital badge ecosystem that incentivises and recognises accomplishments and experiences in all learning contexts. This is particularly relevant for resource-constrained environments as these badges can assist them to show their own credibility on social media. A digital open badges ecosystem can potentially create a connected learning environment where skills, achievements and competencies are fully recognised and accredited in specifically resource-constrained environments so that, once community members have a digital identity, their paper-based badges can be replaced with digital badges which acknowledge their competency learning.
Acknowledgements
Competing interests
The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.
Authors’ contributions
P.M. was the main author and mostly did the literature searches and the literature study. M.H. wrote the methodology section and contributed to the overall editing of the article. A.B. contributed to the scientific rigour of the literature, the structure of the article and the referencing.
References
Abramovich, S., Schunn, C. & Higashi, R.M., 2013, ‘Are badges useful in education?: It depends upon the type of badge and expertise of learner’, Educational Technology Research and Development 61(2), 217–232. https://doi.org/10.1007/s11423-013-9289-2
Ahn, J., Pellicone, A. & Butler, B.S., 2014, ‘Open badges for education: What are the implications at the intersection of open systems and badging?’, Research in Learning Technology 22, 1–8. https://doi.org/10.3402/rlt.v22.23563
Anderson, A., Huttenlocher, D., Kleinberg, J. & Leskovec, J., 2013, ‘Steering user behavior with badges’, paper presented at the 22nd International Conference on World Wide Web, Rio de Janeiro, Brazil, 13–17th May.
Arksey, H. & O’Malley, L., 2005, ‘Scoping studies: Towards a methodological framework’, International Journal of Social Research Methodology 8(1), 19–32. https://doi.org/10.1080/1364557032000119616
Ash, K., 2012, ‘Digital badges. Would represent students’ skill acquisition’, Education Week, viewed 18 June 2016, http://www.edweek.org/dd/articles/2012/06/13/03badges.h05.html
Badge Europe!, 2016, About badge Europe!, viewed 18 June 2016, http://www.openbadges.eu/content/about-badge-europe
Bowen, K. & Thomas, A., 2014, ‘Badges: A common currency for learning’, Change: The Magazine of Higher Learning 46(1), 21–25. https://doi.org/10.1080/00091383.2014.867206
Brien, S.E., Lorenzetti, D.L., Lewis, S., Kennedy, J. & Ghali, W.A., 2010, ‘Overview of a formal scoping review on health system report cards’, Implementation Science 5(1), 2. https://doi.org/10.1186/1748-5908-5-2
Buckingham, J., 2014, ‘Open digital badges for the uninitiated’, The Electronic Journal for English as a Second Language 18(1), 1–11.
Carey, K., 2012, ‘Show me your badge’, The New York Times, on page ED28 of Education Life, viewed 18 June 2016, http://www.nytimes.com/2012/11/04/education/edlife/show-me-your-badge.html
Crafford, R. & Matthee, M., 2016, ‘Implementing open badges for recognition of learning achievements in South African organisations’, paper presented at the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, Johannesburg, South Africa, 26–28th September.
Davis, K. & Singh, S., 2015, ‘Digital badges in afterschool learning: Documenting the perspectives and experiences of students and educators’, Computers & Education 88, 72–83. https://doi.org/10.1016/j.compedu.2015.04.011
Devedžić, V. & Jovanović, J., 2015, ‘Developing open badges: A comprehensive approach’, Educational Technology Research and Development 63(4), 603–620. https://doi.org/10.1007/s11423-015-9388-3
De Villiers, A.C. & Sauls, M.M., 2017, ‘Changing teachers’ practice in the Creative Arts classroom: The case for educational technologies’, The Journal for Transdisciplinary Research in Southern Africa 13(1), 9. https://doi.org/10.4102/td.v13i1.371
DigitalMe, 2016, Projects – DigitalMe, viewed 16 August 2016, http://www.digitalme.co.uk/projects-1/
Ellis, T.J. & Levy, Y., 2010, ‘A guide for novice researchers: Design and development research methods’, paper presented at the Informing Science & IT Education Conference, SITE, Cassino, Italy, 19–24th June.
Fain, P., 2014, Badging from within, Inside Higher Ed., pp. 1–4, viewed 13 July 2016, https://immagic.com/eLibrary/ARCHIVES/GENERAL/GENPRESS/I140103F.pdf
Frederiksen, L., 2013, ‘Digital badges’, Public Services Quarterly 9(4), 321–325. https://doi.org/10.1080/15228959.2013.842414
Gamrat, C., Zimmerman, H.T., Dudek, J. & Peck, K., 2014, ‘Personalized workplace learning: An exploratory study on digital badging within a teacher professional development program’, British Journal of Educational Technology 45(6), 1136–1148. https://doi.org/10.1111/bjet.12200
Gibson, D., Ostashewski, N., Flintoff, K., Grant, S. & Knight, E., 2015, ‘Digital badges in education’, Education and Information Technologies 20(2), 403–410. https://doi.org/10.1007/s10639-013-9291-7
Glasziou, P., Irwi, L., Bain, C. & Colditz, G., 2001, Systematic reviews in health care: A practical guide, Cambridge University Press, Cambridge.
Glover, I., 2013, ‘Open badges: A visual method of recognising achievement and increasing learner motivation’, Student Engagement and Experience Journal 2(1), 1–4.
Glover, I. & Latif, F., 2013, ‘Investigating perceptions and potential of open badges in formal higher education’, in J. Herrington, A. Couros, & V. Irvine (eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2013, pp. 1398–1402, AACE, Chesapeake, VA.
Goligoski, E., 2012, ‘Motivating the learner: Mozilla’s open badges program’, Access to Knowledge: A Course Journal 4(1), 1–8.
Grant, S., 2014, What counts as learning: DML Research Hub, viewed 20 August 2016, from http://dmlhub.net/publications/what-counts-learning.
Haaranen, L., Hakulinen, L., Ihantola, P. & Korhonen, A., 2014, ‘Software architectures for implementing achievement badges-practical experiences’, paper presented at the 2014 International Conference Teaching and Learning in Computing and Engineering (LaTiCE), Kuching, Malaysia, 11–13th April, viewed 16 August 2016, https://www.computer.org/csdl/proceedings/latice/2014/3592/00/index.html
Halavais, A.M., 2012, ‘A genealogy of badges: Inherited meaning and monstrous moral hybrids’, Information, Communication & Society 15(3), 354–373. https://doi.org/10.1080/1369118X.2011.641992
Harzing, A.-W., 2010, The publish or perish book, Tarma Software Research, Melbourne, Australia.
Hess, F.M., 2011, Quality control in K-12 digital learning: Three (imperfect) approaches. Creating Healthy Policy for Digital Learning, Thomas B. Fordham Institute, Washington, DC. (A Working Paper Series).
Hickey, D.T., Itow, R.C., Rehak, A., Schenke, K. & Tran, C., 2013, ‘Speaking personally – With Erin Knight’, American Journal of Distance Education 27(2), 134–138. https://doi.org/10.1080/08923647.2013.783268
Hickey, D.T., Willis, J. & Quick, J., 2015, Where badges work better, Learning Initiative ELI, EDUCAUSE, Washington, DC.
Hole, A., 2014, ‘Open badges: Exploring the value, potential and practicalities of a new way of recognising skills in higher education’, Journal of Learning Development in Higher Education special edn., November, 1–9.
Ifenthaler, D., Bellin-Mularski, N. & Mah, D.-K., 2016, Foundations of digital badges and micro-credentials, Springer, New York.
Itow, R.C. & Hickey, D.T., 2013, Design principles for assessing learning in learning ecosystems: Fostering productive interactions around digital badges, viewed 20 August 2016, http://rebeccaitow.com/wp-content/uploads/2014/08/Assessing-Learning-with-Digital-Badgesv6.pdf
Jabbar, A., Gasser, R.B. & Lodge, J., 2016, ‘Can new digital technologies support parasitology teaching and learning?’, Trends in Parasitology 32(7), 522–530. https://doi.org/10.1016/j.pt.2016.04.004
Jovanović, J. & Devedžić, V., 2015, ‘Open badges: Novel means to motivate, scaffold and recognize learning’, Technology, Knowledge and Learning 20(1), 115–122. https://doi.org/10.1007/s10758-014-9232-6
Knight, E., Casilli, C., Lee, S., Goligoski, E., McAvoy, C., Brennan, B., et al., 2014, ‘Badges’, MozillaWiki, viewed 19 March 2014, https://wiki.mozilla.org/Badges.
Knight, E. & Mozilla Foundation, 2012, RFC – An open, distributed system for badge validation (Working Paper), Mozilla, Mountain View, CA.
Lambert, H., 2006, ‘Accounting for EBM: Notions of evidence in medicine’, Social Science & Medicine 62, 1–6. https://doi.org/10.1016/j.socscimed.2005.11.023
Law, P., 2015a, ‘Digital badging at The Open University: Recognition for informal learning’, Open Learning: The Journal of Open, Distance and e-Learning 30(3), 221–234.
Law, P., 2015b, ‘Recognising informal elearning with digital badging: Evidence for a sustainable business model’, Open Praxis 7(4), 299–310. https://doi.org/10.5944/openpraxis.7.4.247
Law, P., Perryman, L.-A. & Law, A., 2015, ‘Badging and employability at The Open University’, European Journal of Open, Distance and E-learning 18(2), 1–8.
Linde, K., 2002, ‘Systematic reviews and meta-analyses’, in G. Lewith, W. Jonas & H. Walach (eds.), Clinical research in complementary therapies: Principles, problems and solutions, pp. 119–135, Elsevier, Churchill Livingstone, Edinburgh.
Ma, X., 2015, ‘Evaluating the implication of open badges in an open learning environment to higher education’, paper presented at the 2015 International Conference on Education Reform and Modern Management, Hong Kong, 19–20th April.
Makewaves, 2016, ‘Share what you make’, in Free school blogs and secure social learning platform, viewed 05 August 2016, https://www.makewav.es
Masura, S., 2013, Digital badges, Cherry Lake Publishing, Mannesota, FL.
Mayrath, M.C., 2012, Technology-based assessments for 21st century skills: Theoretical and practical implications from modern research, IAP, Washington, DC.
McDaniel, R., Lindgren, R. & Friskics, J., 2012, ‘Using badges for shaping interactions in online learning environments’, paper presented at the 2012 IEEE International Professional Communication Conference (IPCC), Orlando, FL, 8–10th October.
Mewburn, I., Freund, K. & Rutherford, E., 2014, Badge trouble: Piloting open badges at the Australian National University, ANU Research Publications, Melbourne, Australia.
Meyer, I. & Marais, M., 2015, ‘Design for sustainability: Countering the drivers of unsustainability in development projects’, The Journal of Community Informatics, 11(3), 1–9.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G. & Group, P., 2009, ‘Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement’, PLoS Medicine 6(7), e1000097. https://doi.org/10.1371/journal.pmed.1000097
Morrison, B.B. & DiSalvo, B., 2014, ‘Khan academy gamifies computer science’, proceedings of the 45th ACM technical symposium on Computer Science Education, Pretoria, 27 July, pp. 39–44.
Mozilla Foundation, 2011, ‘Mozilla launches open badges project’, Mozilla Blog, viewed 16 June 2016, https://blog.mozilla.org/blog/2011/09/15/openbadges/
Mozilla Foundation, 2014a, ‘Badges/FAQs’, MozillaWiki, viewed 16 June 2016, https://wiki.mozilla.org/Badges/FAQs#Open_Badge_Infrastructure_.28OBI.29
Mozilla Foundation, 2014b, ‘Badges/onboarding-issuer’, MozillaWiki, viewed 17 June 2016, https://wiki.mozilla.org/Badges/Onboarding-Issuer.
Mudavanhu, F., 2015, Challenges facing South Africa – Massive Open Online College South Africa, MOOC SA, Pretoria, South Africa.
Muilenburg, L.Y. & Berge, Z.L., 2016, Digital badges in education: Trends, issues, and cases, Routledge, Taylor & Francis Group, London, UK.
Myllymäki, M. & Hakala, I., 2014, ‘Open badges in higher education’, paper presented at the 6th International Conference on Education and New Learning Technologies (EDULEARN14), Barcelona, 4–6th July.
Nestor, M., 2014, Badges, Kent State University, viewed 16 June 2016, http://www.kent.edu/onlineteaching/badges
Niehaus, E., Platz, M., Herselman, M. & Botha, A., 2017, ‘Using digital badges in South Africa informing the validation of a multi-channel open badge system at a German University’, paper presented at the IST Africa 2017, Windhoek, Namibia, 11–13th May.
Okoli, C. & Schabram, K., 2010, ‘A guide to conducting a systematic literature review of information systems research’, Sprouts Working Paper Information Systems 10(26), 1–49.
PASA, 2016, Case study: PASA’s open badges, viewed 18 August 2016, http://www.mypasa.org/2014/03/19/case-study-pasas-open-badges/
Pearson Education, 2013, Open badges are unlocking the emerging jobs economy, viewed 18 August 2013, http://www.pearsoned.com/wp-content/uploads/Open-Badges-for-Higher-Education.pdf
Pedro, L., Santos, C., Aresta, M. & Almeida, S., 2015, ‘Peer-supported badge attribution in a collaborative learning platform: The SAPO Campus case’, Computers in Human Behavior 51, 562–567. https://doi.org/10.1016/j.chb.2015.03.024
Price, B., 2014, Newport city homes – Open badges criteria, viewed 12 August 2016 http://www.newportcityhomes.com/Downloads/NCH_Criteria.pdf
Raish, V. & Rimland, E., 2015, ‘Employer perceptions of critical information literacy skills and digital badges’, College & Research Libraries 77(1), 87–113.
Randall, D.L., Harrison, J.B. & West, R.E., 2013, ‘Giving credit where credit is due: Designing open badges for a technology integration course’, TechTrends 57(6), 88–95. https://doi.org/10.1007/s11528-013-0706-5
Rooyen, M.V., 2011, ‘State “freezes out” private skills provider’, MG Online, viewed 01 September 2016, http://mg.co.za/article/2011-11-04-state-freezes-out-private-skills-providers/
Rosewell, J., 2012, A speculation on the possible use of badges for learning at the UK Open University, Open University Press, Milton Keynes, UK.
Rughinis, R., 2013, ‘Talkative objects in need of interpretation. Re-thinking digital badges in education’, paper presented at the CHI’13 Extended Abstracts on Human Factors in Computing Systems, Paris, France, 27 April–01 May.
Sandelowski, M., 2000, ‘Focus on research methods-whatever happened to qualitative description?’, Research in Nursing and Health 23(4), 334–340. https://doi.org/10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-G
Sandelowski, M., Barroso, J. & Voils, C.I., 2007, ‘Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings’, Research in Nursing & Health 30(1), 99–111. https://doi.org/10.1002/nur.20176
Sandelowski, M., Docherty, S. & Emden, C., 1997, ‘Qualitative metasynthesis: Issues and techniques’, Research in Nursing and Health 20, 365–372.
Santos, C., Almeida, S., Pedro, L., Aresta, M. & Koch-Grunberg, T., 2013, ‘Students’ perspectives on badges in educational social media platforms: The case of SAPO campus tutorial badges’, paper presented at the 2013 IEEE 13th International Conference on Advanced Learning Technologies (ICALT), 15–18 July, Beijing, China.
Smore, 2016, Digital badging, viewed 18 August 2016, from https://www.smore.com/y0fg0-digital-badging.
Sullivan, F.M., 2013, New and alternative assessments, digital badges, and civics: An overview of emerging themes and promising directions, CIRCLE Working Paper No. 77, Center for Information and Research on Civic Learning and Engagement (CIRCLE), viewed 14 July 2016, from http://goo.gl/zXzyUY.
The GRASS Project, 2016, Project description – The GRASS project, viewed 14 July 2016, from https://sites.google.com/site/llpgrassproject/project-description.
The Mozilla Foundation, 2012, The Mozilla Foundation, Peer 2 Peer University, The MacArthur Foundation: Open Badges for Lifelong Learning, viewed 16 June 2016, from https://wiki.mozilla.org/images/5/59/OpenBadges-Working-Paper_012312.pdf
Tran, C., Schenke, K. & Hickey, D.T., 2014, ‘Design principles for motivating learning with digital badges: Consideration of contextual factors of recognition and assessment’, Learning and Becoming in Practice Principles for Motivating Learning with Digital Badge (ICLS), 2, 1027.
Weeks, L.C. & Strudsholm, T., 2008, ‘A scoping review of research on complementary and alternative medicine (CAM) and the mass media: Looking back, moving forward’, BMC Complementary and Alternative Medicine 8(1), 43. https://doi.org/10.1186/1472-6882-8-43
White, A. & Schmidt, K., 2005, ‘Systematic literature reviews’, Complementary Therapies in Medicine 13(1), 54–60. https://doi.org/10.1016/j.ctim.2004.12.003
Footnotes
1. Used under CC-BY-ND License.
|
Hide
Show all
Email this article (Login required)
