Last week, we released the alpha version of the Mozilla Open Badge Infrastructure (OBI) with the first issuer, Peer 2 Peer University. 

What is the Open Badge Infrastructure? (earlier post and background here)

The Mozilla OBI is a technology solution to support the creation of a badge ecosystem. The OBI allows there to be many independent badge issuers and for a single individual to earn badges across issuers, collect them to a single collection that s/he has complete control over, and then decide how to share out various subcollections with display sites including social networks, career sites, personal sites and digital resumes/portfolios. The OBI is that central piece that includes a reference badge repository, management interface (the ‘Badge Backpack’) for each user and the necessary communication channels to support pushing badges in (issuers) and pulling badges out (displayers), as well as authenticating and verifying badges.

What is the alpha version? What does it include?

The alpha version is the first major release of the OBI. It includes:

• Core Hub (repository) technology
• Communication channels between issuer and Hub
• Issuer tech for P2PU (built for Django and available on Git Hub)

There are still a few things that need to happen on the P2PU side for the P2PU community to push their badges to the OBI, but all the groundwork is there. Here is a post on the latest updates on the P2PU side.

What’s next?

1) Beta - September 2011

The alpha version is the first step in many good things to come. Next up is the beta version which will be a a core feature-complete, fully functioning OBI that is launched with a small number of partners as issuers. Beta will also include the Badge Backpack for badge management and the first display widget for users to display badges outside of the OBI. We are aiming to launch beta in early September. That’s right folks, only a month away!

2) Public 1.0 Release - January 2012

After beta, next up is a public release of the spec/API and all documentation. In addition to feature additions and bug fixes/updates, this public 1.0 release will allow anyone to become a badge issuer or badge displayer with the OBI. The Public 1.0 release is scheduled for early January 2012.

So, needless to say, much more to come soon!

-E

We are currently in the midst of planning for the second phase of the School of Webcraft assessment and badge pilot and one of the key elements of this phase is the addition of more skill badges (and associated assessments). 

As I have previously detailed, last February we launched a pilot with 14 assessments and badges including skill badges (Javascript Basic/Expert, PHP Basic/Expert, Open Source Contributor), value badges (Accessibility Foundations/Expert), peer/community badges (Team Player, Peer Mentor, Good Communicator, Community Builder, etc) and some P2PU-specific badges (P2PU Veteran, Course Organizer). 

In this round, we are building out the skill badges significantly, adding at least 12 new skill badges to the mix:

1. HTML Basic
2. HTML Expert
3. CSS Basic
4. CSS Expert
5. Python Basic
6. Python Expert
7. JQuery Basic
8. JQuery Expert
9. HTML5
10. CSS3
11. Popcorn.js Demo
12. Popcorn.js Plug-In

The important part is to make sure that the assessments behind the badges are appropriate and effective at demonstrating the right skill. We don’t want something to easy or too hard, but that’s tough to tell since we aren’t experts in most (or all of this). So we need help. 

Here is an outline of the current thinking on the assessments. Please give us your feedback. 

3 key areas to focus your attention and feedback:

1. Filling in the blanks
2. Reviewing existing assessments
3. Reviewing rubrics 

1) FILLING IN THE BLANKS

You will see that there are two that are still pretty blank:

• Python Basic 
• JQuery Basic

What are some challenges, exercises or projects that demonstrate basic understanding of python or jquery? All ideas and resources are welcome and appreciated. 

2) REVIEWING EXISTING ASSESSMENTS

For the other badges, do these challenges/exercises make sense? Are they the right level? Do they sufficiently demonstrate the skill?

3) REVIEWING RUBRICS

For the rubrics, are these the right things to be looking for? What else should be in there to ensure quality work and skill? I am SURE that we have missed things since again, we are not experts in these technologies/approaches. 

Please have a look and give us your feedback either as a comment here or within the etherpad. As with everything we do, we are moving at warp speed but we want to make sure that we get this right so thanks in advance for all of your help! We are happy to give you acknowledgement for your contributions on the assessments and of course, there will be a badge for those who suggest ideas/feedback that gets incorporated. :)

-E

I have several posts that I have been meaning to do over the last few weeks but there has been so much going on that I have been remiss. So expect a flurry of posts (or a few at least) from me in the next few days.

But to kick things off - we have completed the first phase of the P2PU and Mozilla School of Webcraft Assessment and Badge Pilot. It’s a mouthful and rightly so, since it was full of a lot of very cool stuff. These previous posts here and here give some background on the pilot but to quickly summarize, the pilot consisted of new assessments and badges for skills, values, community interaction and participation in the School of Webcraft. These badges are meant to be an alternative pathway to accreditation and credentialing that SoW community members can earn to demonstrate skills and then share with stakeholders like peers, formal institutions or potential employers to network, progress careers and/or find jobs. 

This initial phase of the pilot included 14 pilot badges (ones designed by us and aligned with specific skills, values and community behaviors relevant to web development) and a bunch of participation badges that came with the core system we were using for the dedicated badge environment, OSQA. The latter were meant to encourage and guide participation in the site as a question and answer forum. Since we were not using it as a true Q&A system, but instead simply leveraging the functionality to support the assessments and badge issuing, many of the OSQA badges were not relevant or achievable by users but some were, such as First Responder, Popular Answer, Editor, etc.

The full evaluation report is available here, but for those that don’t want to read a (titallating) 17 page report, here are some highlights below:

Goals of the Pilot:

• Build proof of concept for a badge system for web development training 
• Create and roll out initial taxonomy for types of badges 
• Develop and roll out assessments that fit the peer and interest-driven learning environment
• Get initial feedback and reactions from the community
• Learn as much as possible that can be applied to later versions of the pilot or integrated solution
• Prototype and pilot the open badge infrastructure

Key Findings

Participation 

• Overall: Participation was lower than expected, with only 52 registered users (in the dedicated badge environment) and of those, 21 active users (earned a badge, assessed work, etc.) We feel there are a couple reasons for this low participation: 1) communication and 2) lack of integration. 
  • Communication: From a communication perspective, this pilot was intentionally tightly controlled, mostly because we wanted to make sure that course organizers were prepared and that we had assessments closely aligned with relevant courses to encourage more active participation and assessing. But this meant we only touched a small portion of the wider Webcraft audience and did so through course organizers who rightly passed the message along (if at all) on their own schedule, so traffic and attention was intermittent at best. We intend to communicate to participants more directly moving forward so that we can ensure that they are fully aware and have all of the information (including why these badges are worth their time). 
  • Integration: On the integration side, as mentioned before, we used an OSQA system that is separate from the P2PU platform and thus required learners to log into a separate site (we built it so that they could use their P2PU account to reduce this issue but it was still a separate action they had to actively take). We plan to integrate the assessments more directly into the learning environment and experience moving forward to make it more seamless.

Assessments

• Overall: Feedback on the assessments was very positive and it seems like we are on the right track with authentic, relevant challenge-based assessments.
• Types: Of the different types of assessments, we really only saw examples of peer assessment, which again were encouraging, with examples of constructive feedback and reworking of submitted work, as well as learners discussing how much they learned from the process of assessing peer work, but there was some struggle with ensuring that there were peers to assess submitted work. That incentive structure is still a gray area for us - we need to figure out how to attract quality people with the right skills to assess submitted work across the system. We will be exploring this more moving forward. We did not have any submissions for expert-level badges (see below) so we did not see any guru assessment, but hope to in subsequent rounds. There was some stealth assessment in the OSQA participation badges, but none of these were directly tied into the learning. 

Badges

• Overall: The main feedback was that people wanted more badges to cover more skills which we totally expected and plan to build out further as we move along. 
• Types: There was a good overall response to the types of badges we had and people felt it was important to have a mix of hard skills and soft skill badges, which we also know are important to badge consumers like potential employers, so we will continue down this path. 
• Levels: There were no submissions for the expert badges which makes some sense given that all of the courses were entry level with some pushing into intermediate for some skills. We do feel the expert level badges are important to have as a goal or benchmark for people to work towards, but we will need some more advanced courses and active advanced community members before we will get more traction on the expert badges. 

Infrastructure

• Prototype: We were planning to run the first phase of the pilot with a prototype of the open badge infrastructure (OBI) that would allow us to port the badges from OSQA into the infrastructure, and then display them on other sites including the P2PU profile. But due to development cycles on both the OBI and P2PU platform, we decided to push this to the end of the second cycle, which will be in late June. 

Summary

Overall, the initial phase of the badge pilot was a positive step in the right direction for our assessment and badge work. We had initially planned on starting with 2 badges and ended up with 14 badges which allowed us to explore more types of assessments and badges in this phase. While participation was low, we learned a lot that we will apply to the next rounds in terms of communication and outreach, and have identified areas that need dedicated focus like driving more peer assessors to be actively involved.

Revisiting our goals, we met most of them by building and launching a quality proof of concept badge system, which included a basic taxonomy for badge types and various assessments approaches built around peer learning. We got some great feedback and interest from the community, as well as other stakeholders, and have some solid direction around future versions of our efforts. The only goal that we were not able to meet was the prototype of the badge infrastructure, which again, was pushed because of delayed contingencies on the development sides, but is targeted to roll into the second phase of the pilot. This will allow us to port the badges out of the OSQA environment and into the P2PU profile to give learners more control over sharing and using the badges in other contexts.

Overall, we feel that we produced a good proof-of-concept to build off of moving forward, and initial responses and observations indicate that it is important and valuable to continue to move in this direction.

Phase 2

We are rolling all of the stuff that we learned from this pilot into the second phase of the pilot which will launch in early to midJune and run through July 2011. Look for another blog post shortly detailing the plans for that phase of the pilot.

Over and out,
E 

As we push forward with our badge pilot and other explorations of badges for learning, we are now facing the inevitable question of what is the badge itself? In some ways, its a much easier question than others we have been working through like how does one earn a badge, how does one use a badge, how much are badges valued, etc. But in other ways, its the toughest question to answer because the badge itself is the gateway, the alert, the attention-grabber, the story-teller. How does one little badge carry all that information?

Two ways. The up front story + the behind/underneath story.

1) What’s Up Front: DESIGN

The look and feel of the badge will tell some of the story at first glance (to humans…and maybe smart dogs like mine). And in fact, the more it can tell, the better, although that said, I much prefer something visual and simple than text-heavy

We are working with an awesome designer to design the badges for the pilot(s) and these are the most current iteration of the first subset:

A few things to note about the thought behind the designs:

Iconography: In our book, iconography can go a long way. We actually prefer icon treatment over text - when it works - but for the skill badges, we felt the text treatment was necessary because without it, we were getting too cute. And at the end of the day, we don’t want to make people guess what these badges are about. If I am putting my badges on a job application, I don’t want to confuse people or make them work hard to figure out what they represent.

 Shape: Shape can also add to the story and for the skill badges, we went with a gear shape to convey something along the lines of harder skills. May or may not be working - interested in feedback*.

Color: Some of this is design 101, but color can tell a story too. Red obviously pops and some felt was more aligned with skills than with community types of badges. (We were also driven by the colors that were automatically assigned to the different types of badges within the badge environment) I think at the very least, having some consistency in color treatments can help color add to the story.

Branding: This was a tricky one for us, and still is being worked out actually. Real estate is obviously an issue here - and while size is not prescribed (you can actually see two we were playing with), its doubtful anyone would display a badge that is 800 pixels wide. Most likely, they would resize it down so that they could display it alongside other badges and thus we would be back to square one. In addition to real estate, branding is tricky for us because we have so much of it :). These badges are being issued from the School of Webcraft, which are a set of P2PU courses that are backed by Mozilla. 

The current thinking is that carrying the Mozilla brand, especially on the skill badges, would have the most value in the marketplace, so that’s what we have now. But interested in feedback - is there more value in having SoW or P2PU? Having branding across all badges? What else?

That is actually a nice segue into the second aspect of badge’s story-telling power:

2) What’s Behind it: METADATA

What’s behind a badge can tell just as much of a story as what’s up front. In fact, it can tell more. Metadata gives us ways to extend the front end design and jam pack the badge with additional story-tellin’ information. For example, with simple html metadata, you may mouseover the badge to see more information like the title, description and branding. But beyond just that, we feel a badge itself should carry with it all of the information needed to understand the badge, value it, validate it (has it expired?), authenticate it (ensure it was issued to this person), etc. A badge is really (on the backend) just a blob of metadata that tells a story. This means that a learner can put that badge anywhere and it will still be able to tell the same, consistent story. 

Oh yeah, and computers don’t speak purty images, its the goods behind/underneath - the metadata - that they care about. And these badges are digital…aka, live, breath and thrive on and between computers…So obviously metadata is extremely important. 

As part of the open badge infrastructure project, we are working to define a metadata spec that each badge could adhere to and thus badges could be exchanged easily across sites. That spec is still in development* but here are some things we know it has to include:

CORE:

• Title
• Description (what does the badge represent, what did someone need to do to get it)
• Image file
• Issue Date
• Issuer
• Callback authentication link (link back to issuer to confirm that this badge was in fact issued on this date to this user)
• Expiration date (issuer can set a certain timeframe for the badge to be valid and require an update or new badge after a certain period)
• URL to evidence (learner work, endorsements, etc backing the badge)

OTHER:

• Group ID - to be able to say that x badge is from a certain group of issuers, this might be b/c you have a certain set of trusted issuers, etc.
• What else?

*How you can help:

• Help us help humans understand badges: Provide feedback and suggestions on the badge designs
• Help us help machines understand badges: Provide input and ideas around the metadata spec. What else will people/computers want to do with badges that should be captured in the metadata?

I *finally* got comments set up on this blog so the good news is that you can provide your feedback right here in line with the post! Brilliant! So, have at it!

-E

My badge paper (also discussed in a previous entry) continues to get more attention and feedback which is great. Keep it coming through the talk page or email. 

A recent post by a colleague pulls out the learner stories from the paper, (plus another one I recently added to show the employer perspective) to highlight the potential impact of badges for various types of learners and stakeholders. 

These are also open for feedback - we are definitely looking for suggestions on other perspectives or stories that are missing or would help add to the overall narrative around badges.

Here is the post that summarizes the stories: http://openmatt.wordpress.com/2011/03/17/badges-in-the-real-world/

Thanks,
E 

As you’ve read in my posts and those of colleagues, we’ve done a lot of thinking, talking and conceptualizing about badges as an alternative path to certification. We’ve talked about badges with a large number of people over the last year, including academics, researchers, youth developers, game designers, technologists, entrepreneurs, teachers, learners and more. We’ve accumulated a set of familiar faces, as well as many interacted with many new folks with new perspectives along the way. We’ve met in Stanford, Barcelona, Toronto, New York and virtually from all over the globe to discuss badges. Basically, it’s been an eat, sleep, drink, travel, talk badges kinda year.

From those discussions and meetings we have collected quite a significant amount of ideas, examples and questions. We’ve seen some trends emerge and have used them to refine our own thinking and approach to badges. 

We have been drafting a working badge paper to capture and express that thinking and approach, which we are now ready to put out for wider consumption. This paper has been through several rounds of edits and feedback from our own team, various working groups and leaders in the MacArthur Foundation. And now its your turn.

You can read the paper here: http://bit.ly/badgepaper4

The paper includes some conceptual foundations about badges for learning, as well as an experimental badge system framework that we have been developing to guide badge efforts and explorations. It also ends with a bunch of questions that are still unanswered and can only hope to be answered through on-the-ground badge prototypes and associated research studies, as well as through more discussions with more people. Thus the paper is intentionally approachable and less formal than an academic paper, but is meant to communicate our approach and support others thinking about badges system prototypes, as well as, perhaps most importantly, to put something on paper for people to start reacting to and to get more people involved in the discussions. 

So get involved! Read the paper. Give us feedback. Share your thoughts. There is a link within the g.doc to give feedback via a talk page or email. Looking forward to talkin’ badges with you further.

-E

That’s right, the time has finally come. The School of Webcraft assessment and badge pilot is live!  http://badges.p2pu.org

Learners can now log in (with their P2PU account) and:

Check out all the badges and assessments
There are a number of different types of badges and assessments represented in the pilot. Click around and check them out. (More on the pilot in case you missed it). 

Earn a badge
Submit to an assessment to earn a badge.

Assess submitted work
All the Basic or Foundation level badges are peer assessed. Peers can review the submitted work and vote Yes/No. At the defined threshold of Yes votes, the badge will be issued.

Give their peers community badges
There are a number of peer-to-peer badges that learners can give each other based on experiences in the courses and community. These include Good Teammate, Peer Mentor, Communicator, Innovator and Community Builder.

Give us feedback
Help us learn. Help us scale. Give us feedback on the pilot!

Badge on, friends, badge on.

-E

Drum roll please…it’s time for the great unveiling of our initial assessment and badge pilot!!

(Actually, being part of the open and transparent communities of P2PU and Mozilla, I’ve actually presented this many times, but this is my first official blog post about it)

Check out this wiki page for complete background information and pilot details.

Here’s the skinny, as I have discussed in previous posts, there is broad recognition of the need for alternative paths to certification and ways to legitimize the learning that is happening outside of formal channels, including informal and social learning. Both P2PU and Mozilla are committed to providing these open, free and personalized opportunities for learning, as well as making them ‘count’ in more formal contexts. We think badges (and associated assessments) can be a way to help capture, formalize and translate the learning that is occurring through these alternative (and awesome) channels. 

So that brings us to The Badge Pilot [emphasis added and intentional, its a big deal :)].  We are piloting a set of assessments and badges in the School of Webcraft, which is a joint venture between Mozilla and P2PU to offer peer-driven courses on open web development skills.

The pilot will is intended to explore different types of badges, as well as different types of assessments, and will include:

BADGES:

• Skill badges (Javascript Basic, Javascript Expert, PHP Basic, PHP Expert)
• Value badges (Accessibility Foundations, Accessibility Evangelist)
• Community-Oriented Badges (Peer Mentor, Good Communicator, Team Player, Innovator, Community Builder)
• Behavior/Participation Badges (within the community forum environment) (First Responder, Best Answer, etc.)

ASSESSMENTS:

• Peer Assessment - Basic badge work will be assessed and voted on by peers. At some threshold of Yes votes, the badge will be issued.  Peers will also assess the community-oriented badges and issue to peers directly based on their experiences within courses.
• Guru (Or Has-A Badge) Assessment - Expert/Evangelist badge work will be assessed by those community members who have already earned the badge.  This will ensure that we have expert work assessed by experts. We will seed the community with the first gurus and then they will assess the initial submissions. As they issue the badge, those learners will also be able to assess submitted work. 
• Stealth Assessment - All of the Behavior/Participation badges within the community forum will be issued by the system based on predefined logic. Badges will be awarded as a ‘surprise’ in many cases, and can help train and guide ideal community behaviors and types of participation. 

DELIVERY:

The assessments and badges will be delivered via an instance of the open source question and answer system, OSQA. This environment will help us manage the assessments (via voting) and badge issuing. 

INFRASTRUCTURE:

The badges issued within the OSQA environment will be ported to a prototype of the open badge infrastructure that is also currently in development as a parallel project. Badges will be collected within the infrastructure prototype and then displayed in the P2PU profile. 

And that’s about it! We are aiming to roll out the assessment/badge environment the week of Feb. 21st (coming right up!), to give learners a couple weeks to submit work for assessment, peers/gurus to assess the work and badges to be issued. And then we evaluate and see where to go from there!

Again, for lots more info, check out this wiki page and all the embedded links.

While this is definitely a low-fi, piece-meal prototype pilot, I think we have set it up in a way that we can learn a lot and inform our own future efforts, as well as the larger badge/certification community and efforts. I will report back!

-E

First post of 2011! What better way to ring in the New Year than a post about the 3rd (and final) piece of the badge system - the open badge infrastructure. I have already (briefly) talked about the assessments and badges, but there is a bigger piece that extends beyond our pilot and even our own definition of badges (hint: the badge infrastructure). As I have discussed before here and here, an alternative form of assessment and certification are necessary because learning is happening all around us, all across the Web and other experiences and yet none of that learning ‘counts’ or is transferable to other contexts. Assessments and associated badges can help us with this by providing a mechanism to demonstrate and capture the learning wherever it happens and then carry the evidence with us back to recruiters, formal institutions or our peer community.

Yeah yeah yeah, I have said all of this before, but the key part that I have not yet addressed is the ‘wherever it happens’ piece (hint: that’s where the open badge infrastructure comes in)… A lot of my day-to-day work lately has been mapping out an assessment/badge plan for the School of Webcraft, a set of P2PU courses on web development. And that’s really cool and important because it is a free, accessible and open path to learning and its also a peer learning environment - all of which are relatively unchartered territories as far as assessment and certification goes. And through these focused efforts we will learn a bunch, potentially (hopefully) provide more incentives for P2PU learners and even provide a model for other people to work from. All good and critical things, but they are still isolated. If we only build our system, we are not supporting learners much better than any individual institution does.  If someone chooses another perfectly legitimate path, it won’t ‘count’ because they can’t get the proof or evidence (degree, badges, etc).

So what are the options?  Well, we could work to design/vet/support badges that cover everyone for every type of learning and every skill/topic and manage all of the badges centrally… Hopefully that concept seems as ridiculous to you as it does to me. Who are we to try to do that? The beauty of the world we live in now is that again, learning is happening everywhere and that everywhere changes and grows constantly. So a truly valuable badge system is one that supports badges from that everywhere. It should support badges from any issuer, collect those badges to a persistent identity (for each individual) and allow the badges to be shared out back into the everywhere. It must be open so that every need and path can be captured and demonstrated and the learner remains in control. This is the open badge infrastructure. And Mozilla is building it.

The open badge infrastructure will support badges issued by anyone across the Web, and allow an individual learner to collect these badges (from those anyone), store them to a single identity and then carry them with them and share them across contexts. Said in plain(er) English, if I am taking a few courses at P2PU and I am also using a series of OER materials in another context that is issuing associated badges, I can collect badges from these independent issuers, have all of the earned badges connected with my open identity, and then I can take those badges with me to interviews, back to my formal institution or post on this blog or LinkedIn profile to demonstrate my learning and skills for various audiences.  This infrastructure is critical to truly support learning across the Web.

Now obviously this is idealized somewhat. In order for ‘every need and path’ to be supported, there would need to be badge issuers at every step. We can’t control who issues badges but we can provide the infrastructure to support anyone who wants to. So we are. And eventually, if and when the value is apparent, sites/providers/communities will want to have badges. And if it is truly open, learners could even create or suggest badges along the way.

Open scares a lot of people. I have heard a colleague say (paraphrasing): “Everyone loves open education until they consider education being truly open.”  Wait, ANYONE can issue badges? It could get messy! There might be a lot of badges?! There might be ‘bad’ badges! And people might game the system! True. All things to watch closely. But a centralized or closed system WILL NOT solve our problems, and in fact will simply recreate the ones we already have by only supporting a small subset of the learning that is occuring, putting the power to decide what ‘counts’ in a small number of hands, created prescribed learning paths, demotivating learners…and so on and so on.

Besides, I seem to remember a similar case in the early days of the Web. Wait, ANYONE can create a website? It could get messy! There might be a lot of websites?! There might be ‘bad’ websites! Then we had Google and various services that help us find, rate and share websites that are credible and/or are valuable/relevant/interesting to us.  Maybe we will need something like that for badges, we don’t know yet.  But just as a closed and controlled Web would have never resulted in the explosion of creativity, expression, transparency and access that we value and depend on today, a closed badge system will never reach full potential. Open badge infrastructure FTW!

It goes without saying that Mozilla, ambassador of the open web, is the right entity to be building this open badge infrastructure. There is a team already cranking away to open up badges and take this thing the the next level. They built a prototype in Barcelona and haven’t looked back. More to come over the next few months!

-E