Problem & objective analysis

Let's start with an example: A dedicated team of Wikimedians

For the rest of the course we will be working alongside the example of a dedicated team of Wikimedians in their journey to develop their first education project. They will help us work on each topic that focuses on project management skills by providing examples we can analyze and use as a starting point for the course assignments. Let's meet them!

Meet Alex, Laura and Isaac! They are three Wikimedians who are embarking on their first education initiative.
For a few years now they have been volunteers in different Wikimedia projects. Alex is passionate about using Wikibooks as a platform to create OERs that represent local knowledge. Laura is an active contributor to Wikimedia Commons and likes to engage fellow students in photo walks around the countryside. Isaac spends his volunteer time adding content to Wikipedia and Wikidata in partnership with his librarian colleagues. They became friends after attending some local Wikimedia meet-ups and discovering their interest in working with teachers and students in their city.
They had a great idea for the first education project their team would develop with teachers from Alex's former high school: teaching them how to use Wikibooks so they can create their own books to use in the classroom and contribute to the OER community.
Alex talked to the headmaster and they got authorization to conduct a 3-hour introductory seminar to the Wikimedia projects in general and to Wikibooks in particular. The team worked hard in the planning and they invited some fellow Wikimedians to help in the activity. They prepared all the materials and made sure their seminar was dynamic, informative, and motivating. Many teachers attended and they were engaged and receptive.
However, the evaluation survey they conducted at the end showed them that teachers were already using other platforms to create their own books as OERs and didn't really need training on how to use a new one. In fact, the majority of teachers were more interested in learning about Wikidata and how to integrate it into their teaching practice.

Quiz: What should the team do next?

Bring Wikidata experts to train the teachers on the basics of the project.
Survey the teachers to see their level of familiarity with Wikidata and to understand their motivations.
Look for a new school where to conduct the training on Wikibooks.

Check the Discuss page for the answer.

Needs assessment as a first step to design for impact

Yes! The team can benefit from taking a step back and really getting to know their audience before designing a new seminar: What do they already know? What do they want to know?

Collecting and using this data to make decisions can improve the relevance of your work, and therefore your Wikimedia education projects can have a greater impact. We call this process "Needs Assessment". In the next sections you will learn more about what needs assessments are, the different types of needs assessments you can conduct, and the data collection tools you can use in this process.

Definition

For a definition of needs assessments let's turn to our favorite digital encyclopedia:

A needs assessment is a systematic process for determining and addressing needs, or "gaps" between current conditions and desired conditions or "wants". The discrepancy between the current condition and wanted condition must be measured to appropriately identify the need. The need can be a desire to improve current performance or to correct a deficiency. A needs assessment is a part of planning processes, often used for improvement in individuals, education/training, organizations, or communities. It can refine and improve a product such as a training or service a client receives. It can be an effective tool to clarify problems and identify appropriate interventions or solutions. By clearly identifying the problem, finite resources can be directed towards developing and implementing a feasible and applicable solution. Gathering appropriate and sufficient data informs the process of developing an effective product that will address the groups needs and wants. Needs assessments are only effective when they are ends-focused and provide concrete evidence that can be used to determine which of the possible means-to-the-ends are most effective and efficient for achieving the desired results.

Sample steps to conducting a needs assessment

In his article "Methods for conducting an educational needs assessment" , Paul F. McCawley (University of Idaho) introduces seven steps to develop a needs assessment.

Write objectives: What is it that you want to learn from the needs assessment?
Select audience: Who is the target audience? Whose needs are you measuring and to whom will you give the required information?
Collect data: How will you collect data that will tell you what you need to know? Will you collect data directly from the target audience or indirectly?
Select audience sample: How will you select a sample of respondents who represent the target audience?
Pick an instrument: What instruments and techniques will you use to collect data?
Analyze data: How will you analyze the data you collect?
Follow-up: What will you do with information that you gain? Data gathering methods by themselves are not a needs assessment. For the process to be complete, the needs assessment has to result in decision-making.

Types of needs assessments

A web search for "needs assessment" or "needs analysis" will show you a variety of methods, tools, and approaches to conducting this process. Below are a few types that can be particularly helpful when you are planning education activities.

SWOT Analysis^[1]

SWOT stands for:

Strengths - internal characteristics of your team/organization that can help you achieve the project's objectives.
Weaknesses - internal characteristics of your team/organization that can negatively impact the project.
Opportunities - external conditions that can help you to reach the project's objectives.
Threats - external conditions that can prevent the success of the project.

You can use a SWOT analysis to determine the internal and external factors that can impact your project's success. It is an exercise generally recommended to take place at the beginning of a project planning process, or when the internal or external conditions of your project have changed.

To do a SWOT analysis you can use the following template or a variation of it. (You can find the PDF version of this slide here)

In practice, you and your team would get together to brainstorm and list all of the elements that correspond to each field. Focus the process on things that are affecting or can affect the Wikimedia education project you are planning to develop. After that, you would reflect on some actions that you can take to improve your weaknesses, minimize the threats, and take advantage of the strengths and opportunities.

An example done by Alex's team looks like this:

Strengths	Weaknesses
Team is passionate about education Good knowledge of different Wikimedia projects Good relationships with school administrators Creative - many ideas for education projects	No experience conducting Wikimedia education projects Not much access to funding Team has little knowledge about the teaching profession in our country
Opportunities	Threats
Local Wikimedia affiliate can help us connect with other more experienced volunteers Wikimedia regional events are coming up - it's a chance to get more training and connect with other Wikimedians Networking events with education professionals happen frequently in our city	Schools tend to prefer trainings offered by old and well-established institutions Generally teachers have a bad impression on Wikipedia and other Wikimedia projects School administrators change quite often, this means there may not be consistent points of contact

Training Needs Assessment^[2]

It's a specific inquiry into the kind of capacity development opportunities needed by a team or organization. This is helpful to make decisions and allocate resources to address the gaps in knowledge and skills that the team/organization presents.

Linda Dausend^[3] breaks down the process of conducting TNA in 4 basic steps.

1. Start with the bigger organizational picture

In this step, Dausend recommends reflecting on the state of the organization/team: What has been done in the past? What is the vision for the future? In this stage it's important to ask questions that will help us understand what people want to achieve, how the new knowledge and skills will be implemented, and what the big goals of the training program will be. In the case of Alex's team, in this stage they would approach the school they want to work with and find out if there have been previous initiatives similar to what they conducted. They would explore what goals of the school can be aligned to the training they want to provide and identify the knowledge and skills that teachers could gain.

2. Understand the current performance

The next step is considering the skills levels of the participants in comparison to the desired level, this would help us identify the needs and define the learning objectives and audience. During this step it's important to define how these skills will be identified and measured, and how the trainers will guarantee reinforcement and sustainability after the training is over. Alex's team, for example, would approach the teachers and conduct surveys to identify the knowledge and skills they possess and the knowledge and skills they wish to gain. This would help them set a baseline with information of the current state of the teachers and set learning objectives to achieve during the training they will implement.

3. Get to know the ideal participants

Step 3 focuses on defining the number of participants that would attend the training and their characteristics. Identifying key leaders or emerging leaders will support the sustainability of the project. During this step, Alex's team would define some criteria to choose the ideal participants for their training and the number of teachers they have the capacity to train. They would also explore if including other actors from the school such as librarians or administrative staff can be beneficial.

4. Focus on the evaluation

The final stage is reflecting on the ways that the training can be evaluated: How will you know if the training is effective after you have developed it? Alex's team would decide if they want to use surveys, interviews, exams, etc. after the training to make sure that the participants have achieved the learning objectives established earlier.

One more consideration:

Dausend emphasizes that a TNA isn't a one time event - the needs of a training program change and evolve. She also states that this process should be analyzed, discussed, and verified among team members to ensure understanding of the findings. Making changes is perfectly OK. After the TNA is completed and we understand the skills and needs of the audience we will be working with, set learning objectives and an evaluation plan, we can create a better and more impactful training.

A final note:

Additionally, during the needs assessment process it's important to reflect on:

Who are we talking to? Think back to Module 2 in Unit 1 and the interview you did with a teacher. Who else can you talk to in the education sector to make sure you are designing an impactful project? Teachers, students, librarians, parents - Are you talking to people beyond your immediate circle? Are you including a variety of stakeholders in your needs analysis?

Who are we including? Is your needs assessment providing you with information that will help you include under-represented groups? What kind of questions do we need to ask to get this information? How do we learn more about the groups that are generally excluded or left out of this type of initiatives?

Tools and methods to conduct a needs assessment

In the previous section you saw the SWOT analysis matrix and you learned some basic stages of conducting a needs assessment. What other tools can we use in this process?

Here's a quick list from IDEO's Design Kit.

Recruiting tools to determine who to talk to.
Interviews for well-structured in-depth conversation with different participants and stakeholders. You can include activities such as the Five Whys or Card Sorting.
Group interviews to get a better understanding of a community or address specific groups of people.
Surveys which can be conducted online or face-to-face.
Peer observation to put the process in the hands of the people who will be impacted by your project.
Secondary research to explore and gather data already available about your participants or impact of similar projects.
Problem tree to understand the root causes and effects of an issue you're trying to address through your project.

Course Portfolio Assignment: Creating a problem tree for our project

Just like the team of fellow Wikimedians presented at the beginning of this module, it's your turn to start developing a Wikimedia education project of your own! Focus on one project idea you have for a Wikimedia education initiative. You can also choose a Wikimedia education project you have developed in the past or one you are currently working on. This course portfolio assignment and the ones after this will revolve around the Wikimedia education project you choose.

Follow the instructions in the slides below to create a problem tree for a Wikimedia education project. Include the result of this assignment in your Course Portfolio, it is one of the building blocks for your final course assignment!

PS: You don't have to do this process by yourself. If you're working on a Wikimedia education project idea with a team, meet up and work on this together! You can create your problem tree on a big piece of paper, take a picture and upload it to Commons (or another platform) to attach it to your Course Portfolio. Feel free to be creative!

Feel free to share your problem tree in the Discuss page as well for others to learn from it and provide feedback.

Are you looking for some inspiration? Check out some examples developed by participants of the first cohort of the Wikimedia Education Greenhouse online course: