Competency Matrix and Proof of Competence (Requirement Brainstorm)

# Overview

What if there was a competency matrix on-chain that described what information a person should know for a given expertise -- and when people completed courses for those competencies, they received an NFT as Proof of Competence?

# Competency Matrix

`Colin`: I'm still very keen on the idea of having a competency matrix defined on-chain, and then pointing an LMS towards that contract so that the modules in a course can be associated with each skill that makes up the competency. I imagine the LMS populating a dropdown with what's at the contract address.

`Brian`: What's an example of a competency matrix?

`Colin`: Here's one example: https://www.qualityze.com/build-effective-competency-matrix/ The best efforts to make this kind of data standardized have come out of DoD. The old model is SCORM, which is mostly used for making didactics and assessments more portable. The next iteration is xAPI: https://xapi.com/overview/

So like... if you're reading government solicitations or even RFPs from large orgs that already have multiple learning systems, it's pretty common to see requirements for an xAPI compliant LRS. A blockchain would be the ultimate LRS, but there are some serious considerations that should be given to privacy. Ideally, you could give someone permission to view whatever relevant portions of your transcript.

For our purposes, I would love to see a standard put forth for a JSON schema that could be used to publish a competency standard, such that an LMS could point to a contract and populate a dropdown like I described above.

[Here's an older graphic depicting the full publishing cycle]:
https://cdn.discordapp.com/attachments/906739946793078874/911672217128017940/unknown.png

The top two boxes are where the JSON object is published to a contract, and the curriculum dev points back to it

`Brian`: What could be the minimal prototype to prove out the concept?

Let's use our current Academy MVP content as the guinea pig.

Brainstorm of pieces/workflow:

- minimal JSON schema based on some xAPI for our existing content. Example: "a web3 developer, working on an NFT contract, needs to be able to store the image and JSON data"
- @Chiin is working on a quiz. Quiz somehow connects with JSON schema describing what person needs to learn. So in this example, the quiz would be picking 5 random questions on "What are storage options and how do you store that data in a decentralized way?".
- learner passes quiz
- certificate minted for that JSON schema -- or maybe many certifications -- once for each piece of the schema (e.g. "expert in distributed storage").

I could think of a million sub-specialties:
- an expert on web3 marketplaces and metadata; expert on royalties across marketplaces
- expert on deploying for xyz chain
- expert on testing solidity contracts

We want content to be real-world ready -- I like idea of a laundry list of specialities vs one NFT that says "web3 developer". Instead, I could find the person best for my project based on combo of skills.

I haven't been following Eden project, wonder how they're solving this? (since they're trying to map interests to projects)

`Colin`: Yeah that's basically it. The learner record is made up of any number of xAPI statements, some subset of which may count towards a credential.

# Initial brainstorms from Discord (2022/11/08)

@georgemac510: Is there any way to design assessment checkpoints? It might fold well into issuance of a POAP after passing a particular assessment. Only labor intensive part of it is a manual grading of the assessment on the team’s part.

@briangershon: It'll be hard to scale manual confirmations.

Some brainstorms for "checkpoints":

- we trust user to manually click a button that says "I completed this" checkpoint
- we verify automatically. Maybe each lesson has 5 questions, and they have to answer them all. Or they have to submit a final project, and we verify (scrape the site?) and see something running (completion of a whole track) -- or ping a contract on the blockchain they deployed to validate it works. They can all be cheated, but it sets some sort of minimal bar.
- or we just count impressions (website analytics) and the metric becomes "500 people saw the lesson"

POAPs for checkpoints could be a lower bar, but if we want to offer NFT certifications of completion, that seems a higher bar.

@georgemac510: Any chance to tighten assessment checkpoints where there is an assessment after each step of n? i.e. #1 - what is your output after ‘npx hardhat node”?

I know this will make lessons incredibly long, but seems that learners need a medial self-assessment to keep moving forward and gain confidence.

@briangershon: Almost like a REPL sort of system that could respond after each line.

Another approach that could work -- we supply intentionally failing unit tests -- and when they complete an exercise correctly, the tests will then all run. It's good exposure to automated testing culture -- which is a very useful skill as well. Could be automated too -- they paste their code in and we run unit tests in the cloud.

@georgemac510: I’m just concerned about building confidence through success. Students kick ass on assessments and they feel empowered to move forward.

@briangershon: Another idea -- say the first few lessons in a track are based on trust -- they get something running and feel confidence in their skills, and earn POAPs. Then later lessons in the track they can apply those skills to a bit more of a challenge (say through unit tests or questions or deploying something) and they get official credit for the track upon completion.

...

@briangershon: Ok, here’s [a brainstorm of] our first version of automated evaluation. User logs on with MetaMask. They deploy their contract at end of lesson. Our server then verifies they own the new contract, then exercises the contract via some method calls and POOF we mint them a POAP!

We'll have to flush out more, but maybe after they deploy, they could just submit the address of their new contract to our site.

We could then have a cloud service that calls methods on the contract, say mint or a custom one like retrieveGeneratedArt (depending on what the contract does). Methods that change the blockchain would cost us gas -- albeit DevDAO could fund through partnerships.