Idea impact measurement

Impact measurement approach for measuring the amount of impact that's generated by ideas that are executed

Overview

Idea impact measurement focuses on determining the amount of impact that gets generated when an idea is executed.

Moderate impact measurability (Score - 3)

• Total impact measuring complexity - An idea can often have a narrower set of potential outcomes and areas of impact than an ecosystem wide priority might have. Even though this might be the case there would still be many different areas for generating impact that an executed idea could help with. Executed ideas could also have secondary effects on other areas and generate impact for an ecosystem in a multitude of other ways. This makes it harder to determine what the total impact is for an executed idea.

• Ease of measurement - Most of the quantifiable areas of potential impact should be easy enough to track however there is still a need for interpreting these numbers as Web3 ecosystems can have many different ideas that are trying to address the same priorities. For any of the metrics that an idea is trying to help with, another factor to consider is how other initiatives could have also impacted those metrics in the same time frame.

• Comparability - The more targeted and specific a priority is the more easy it could be for ideas to be compared with each other when determining which of the executed ideas was most impactful for helping to address an existing priority. In other situations comparability is difficult as ideas can help with generating impact outside of any existing priorities and may cause other outcomes that generate more impact than the initially intended area. Ideas that have completely different execution outcomes, like an event and a software protocol, would be more difficult to compare as they have different outcomes and ways they would be measured for generating impact.

Moderate future impact opportunity (Score - 3)

• Usefulness for future decisions - If an executed idea helps to generate a large amount of impact there could be some correlation that incremental improvements to that same idea or similar ideas could also generate further impact as well. Another way that measuring impact could be useful is if it helps to identify the contribution approaches that generate the most high quality outcomes. Future ideas could learn from these previously impactful ideas which were executed in a certain way. Future decisions will involve selecting different and novel ideas which means there can be a limit in the usefulness of previous ideas that generated a certain amount of impact as these new ideas are trying to generate different outcomes. Some learnings can be gathered from these previous examples however it would be difficult to have any high certainty on what future ideas will definitely generate impact or not.

• Repeatability - Previous ideas can help with generating learnings about what quality of outcomes are needed to generate impact and how different ideas could be executed from a process perspective. What is less repeatable is the outcomes and impact generated as different ideas will generate different outcomes. This makes it difficult to accurately predict the impact that these outcomes will generate with any certainty. Some potential areas for repeatability could be identifying successful approaches that other ecosystems have adopted and integrating those approaches into the ecosystem. Another area could be extending an already successful idea by identifying the next logical steps of execution for that idea to improve and generate more impact.

High game theory risks (Score - 2)

• Manipulated outcomes - Ideas that are perceived to be more impactful than others could impact the disbursement process by influencing a community to select the contributors who were responsible for executing those ideas. Due to this there is an incentive for contributors to manipulate the metrics and feedback that emerges after an executed idea is measured so that they can increase the chances that they get selected for compensation in future disbursement decisions. Ideas can be more narrow and targeted in what they are trying to achieve. This smaller surface area of execution can help with making it easier to manipulate the numbers involved about how much a certain executed idea has been adopted or generated impact for the ecosystem.

• Exaggerated outcomes - Bad actors could target the most active participants that have used their executed idea to get testimonials and feedback that better align with their own narrative about how impactful their idea is for the ecosystem. Contributors can be expected to try and promote their own ideas positively to generate more interest and opportunity to receive future compensation. The risks around these efforts will be when that same targeted selection of narrative turns into excessive exaggeration and false observations and statements at the detriment of the ecosystem in being able to identify the most impactful opportunities to support in the future.

High impact verification time required (Score - 2)

• Automated verification - Much of the quantifiable metrics can be automatically verified due to the fact the information will be stored on immutable ledgers and blockchains. Tools can help with reducing the time it takes to get this data and format it into a digestible format. As ideas are often more targeted and focussed than priorities it should require less time to gather and verify this information.

• Manual verification - The qualitative areas of impact measuring will need a more organised and collective effort to analyse and verify the information gathered. Tools could eventually help with identifying biases, flaws and potential conflicts of interest that emerge from the contributors who participate in collecting this information. As ideas are often more targeted and focussed than priorities it should require less time to gather and verify this information.

Total score = 10 / 20