Scientific Method for Wikimedians/Designing a Research Method

This is the eighth chapter in this course; it is entitled "designing a research method". it is located in the part 2 of this course "the research plan".
Previously, we have covered the first two steps of the scientific method in chapter 6; we talked about the research question; what is it, and how do we create it. We have also found that it is not a question in the common sense of the word; but rather a small documentof a page or two that describes the research motivations and objectives.
In Chapter 7, we have covered the state-of-the-art. It is the first part of the research document, where you need to understand the problem well and see what are the current achievements in the research domain.
Regarding the research question, you have defined in chapter 6; it is normal that by the end of the state-of-the-art you will reshape the research question based on the findings you have.
By arriving here, you need carefully to plan for the next steps in the research method; and the planning itself is the subject of this chapter. Chapter 8 includes five main points we need to cover:

First, a short reminder of the scientific method to show what we have done yet, and what we are expecting to do next. It is important to understand that this chapter does not cover a step which was the case in the last two chapters; instead, it covers a transition between steps that can be done in the office, on the disk, and the steps where you need to go to the field or Laboratory and collect data.
Next, we will explain what you need to do in terms of tasks; in fact you are expected to plan for the next few steps in the research method. In this section we will describe what to cover in the plan; and how shortly we will describe the elements of the research plan.
After that, we will shortly describe the importance of the research planing; how and why you need to plan carefully for your research.
The fourth part is the core of this chapter, in this section we will describe elements of the research plan described in the second point early in this chapter; thus we provide you with a solid background to design your research plan.
Finally as usual, we wrap up this chapter with general notes and findings on the research plan.

Reminder: Scientific method

Before starting, let me remind you what we have done so far regarding the scientific method; step one and two were covered in the previous chapters, and the next step is step 3 forming a hypothesis, so purely you are in between step two and three; and there is a huge difference between step one and two and step three and four, where the first two steps are done on the desk; you need to sit down read scientific Works written by the others and write down your own research question, making it as clear and narrow as possible.
On the other hand, the next two steps are to be done on the field; either a laboratory controlled natural environment or a human society; whatever it is, you cannot do it on your desk; and you need to move away to a new environment, understanding it well and then form your own hypothesis in a way that respect the environment conditions and restrictions. Keep in mind that after you do that you need to go back to the disk again to analyze the collected results; conclude your own research; and write down the final research document, an article, or a report. all phases you need also to keep in mind that you might not go to step seven; after you finish step six if the results are not valid for any reason; and you still have time you can jump back to step three; to make changes to your own hypothesis. Test it and collect new results, so as you can see your path. In the next steps, might not be linear from step 3 to step 8. Now here between step two and step 3, is where you have to plan well for your next moves.

What do you need to do?

In this section, what do you need to do here, is in brief, will instruct you, to the needing to explain; the procedure you will follow in the next steps; in a way, that if someone on the other side of the plant or 100 years later would like to redo your research again; he/she can simply follow your instruction and for sure have the same results. In fact, it's important to understand that the result at the end is the second part of your research work.
If you are writing a thesis or a research report, you need to write the second chapter in this step. If you are planning to produce a journal article, what you will do here is the second part of it the core of any scientific work can be divided into four main parts: the problem definition, the method to solve the problem, the presentation of the obtained results, and the analyzes of these results. Of course auxiliaries can be added before the core, or after it to make the work scientific and easy to verify; the references for example are added at the end of the work, it is not a part of the core but it is essential and cannot be discarded other auxiliaries includes abstract introduction and appendixes and the least varies based on the type of the scientific work you are planning to produce. I will briefly describe the previous structure because we will refer to these parts later in this chapter and discuss them in details:

first a problem definition, it is the first part of your scientific work, it corresponds directly to the research question and the state-of-the-art normally the output of these two steps serve the definition of the problem, I have talked about the two in details in chapter 6 and 7 in these course.
Second the method to solve the problem; it means how you will address the problem; what is your hypothesis how you will test it and what exactly you are looking to collect and why you need clearly to explain that. So if someone want to do the eighth step of the scientific method on your work, which is the retest he can simply do it. In this part you need also to explain how you will show the results; will you use tables? will you use graphics? how you will present huge samples of data? will you use appendixes? you need to explain that here, some researchers prefer to do it in the third part, and you can do that but it is not recommended.
It's highly important that you focus in the third part on just showing the results, so the third part is focusing on results, and here you have to present the data you have collected in your tests; following the way you have defined previously in section two. Remember that you need not to analyze your finding here you have just to present it.
The fourth part is the analysis the results. It is where you need to explain your findings, and add value to your research. The final answer to the research question you have raised in section one should be presented in this section. Even if you could not answer the question; you need to explain why you have failed, and suggest new solution or modifications for your research methods, so others can try it later. We will discuss all of these steps indetails in this chapter.

Now; we understand what are the main parts of the core of any scientific work, we can dive deeper into the second part of this work, which is the research method. To define your research method well, you need to cover four main points I will cover them shortly here but later in this chapter we will discuss them in details:

first, the hypothesis you need to test here based on your research question; you need to form a hypothesis that is a possible answer for the research question we have talked about hypothesis in chapter 5 of this course. shortly, you need to form a relationship between two parameters or morea nd try by test to see if this relationship is true. For example if we talk about the gendergap in English Wikipedia; you can think about the age of the people; who had articles in English Wikipedia to be your first valuable; then the number of the articles for people in this age to be your second variable; then the relationship is to be formed. For example, I will say "biography for women aged more than 60 are half of those of men." So here I defined a hypothesis; it is a claim and I need to prove it later in the next steps. Again this is just an example to show you how you form a hypothesis.
after that you need to test the hypothesis; which is the second part to cover in this section. You can also have several hypothesis but it is preferable to have one hypothesis or Central hypothesis. thus, it is easier to test as we said the second point, you need to cover in this part is the test; How you will verify your hypothesis? What will be the sources of the data? Why did you choose them? and how you will collect them? you need to explain that in details. Thus, if someone is doing the test again, he/she will be capable of doing it.
The third point you need to cover is how you will analyze the collected data. You need to describe your research method, is it going to be inductive or deductive? Will you use quantitative style or qualitative style? Here, you need to describe these in details, we have covered these concepts before in chapter 4, of this course, and you need to refer back to it for more details.
Additionally, later in this course; in part 4, we will focus on analyzing data, so do not worry we will cover these points in details later.
Fourth, the final point you need to cover is: how you will present your results? sometimes you have a choice to go for a journal article, and sometimes you are limited to a research report; whatever the case here, what you need to understand here is that the choice will affect how you will present the data. So you need to understand that well and reduce the exact type of information that is suitable to the media you aregoing to produce. The types of the scientific work will be discussed in details in chapter 9 of this course.

Importance of the method

let me try to answer these questionswhy it is important to describe yourresearch methodwellthere are two answers for these questions:

first, all scientific knowledge are systematicwe have discussed that beforesystematic means they are createdor obtained in a well-defined wayand this way is called the scientificmethodso to show that your work is scientificyou need to tell people what is yourmethodunfortunatelysome researchers does not respect thatand follows some non-scientific wayswithout making attentionand these leads to the direct reject oftheir work in the peer review process wehave covered in chapter fivein simple wordsshowing the method prove that you havefollowed a scientific wayso you need to do that the second answerif you remember the scientific methodshown previously in the second slide ofthis chapterstep 8 is the retestwell if your method is not clear anddetailedother scientists cannot retest your workand validate itcausing the whole scientific method tobe incompleteand this led us to an important questionwe know that old works created free tothe Renaissance did not follow this waynormally methods are not shown in theworks themselvesand the question is does that make ituselesswell this is a complex questionbecause the absence of the method doesnot mean it does not existsimply it was not important to write itdownwe have agreed before that in most ofthe casesin the pre-renaissance erapeople focused more on resultsthan on the methodthey were interested with the finalproduct without making attentionhow it is produced in the first place having that in mind we can easilyunderstandwhy does ancient work lack the methodparthowever in many cases it can beconsidered scientific if we canreproduce it systematically and test itso how you can handle this old workswellnormallyall old Works have been analyzed manytimes beforeso it is safe to check what other peoplewho studied the original text wroteabout it and how did they explain itthen you need to verify that yourselfif for any reasonyou were not able to find any documentthat explains the old textyou have to handle it with doubtand never to accept anything from itwithout verifyingkeep in mind that this does not meanthat the work itself is wrongnor it's truesimply its status it's indetermined andyou need to do your own research toverify its claims

Elements of the method

now let's start with the elements of the method one by one:

The first one is forming a hypothesis; a hypothesisis a claim to be testedit is a possible answer for yourresearch question and it needs to becreatedhoweverin many cases you can work on othershypothesisor simply enhance itif you go back to the scientific methodwe had discussed in chapter fourstep six is the conclusion of thescientific workit is where the researcher put hisfuture aspects in form of hypothesis tobe testedso searching for hypothesis created bythe others is a fine and a correct wayto gothe other way is to create your ownhypothesisthat relates directly to the researchquestion you have posed in the previouschapterin this caseyou need to do the followingaDefine two or more variables directlyrelated to your research questiona variable is a simple scientific factthat can be measuredthis means that it needs to be welldefined with a clear way to measure itfor example if we go back to the gendergap in English Wikipediaif possible variable can bethe total number of female biographiesthe age of person who has an article andthe size of the articleboth the number of the articles ismeasured in articleswhile the age of person is measured inyearsand the size of the article is measuredin bytesI think the idea is clear nowbi you need to create a functionthat describe your valuablefor example let's take the total numberof the male articleand see it in the table shown on thescreenyou will haveone row for the year and one row for thenumber of the articleplease note that the number of thearticles are just for the sake ofexamplethe same goes for the total number ofbiographies for femaleand it's shown on the screen as a secondtable below the table of the mainbiographiesand let me just show you what you havegot in a graphical waythe blue line is the main articles as afunction of timeand the red line are the number of womanbiography as a function of time as welland the variables we have created in the first stepis a point that can move on thesecolumnsand now let's go to the third pointafter you have created your variablesyou need to create a relationshipbetween themand test itfor example the number of biographies ofmenis three times more than the number ofArticles of womenand this ratio is fixed and here is yourhypothesis and now you need to prove itagain the numbers I shown and thehypothesis I createdare for the sake of example and are notbased on any real datathe second point you need to cover inthe method sectionisthe sources of the resultsand this is a large and very importantpartand before we advance I want tohighlight the classification of thesourcesbecause we will use it later in thischapterand we will use it again in the in thenext chapterimagine that the dark point you areseeing on the screenis the studied subjectwellpeople or thingswho get their knowledge directly fromthe subjectwith no intermediate mediumare called primary sourcesand we will refer to them with a darkblue circlethat is drawn around the subjectprimary sources include people that areusing the subjectthus their information or storytellingmight be basedand should be treated carefullythis is true with weaknesses onsomethingespecially if it is shocking or violencelike a natural disaster or a murderprimary sources includes also scientificwork provided by scientists working itlaws to the subjectand studying itthese Works should be treated carefullyas wellbecause they might not be peer reviewedand they might include errors ormistakesif you remember the covid-19 crisisthere were several papers that weresuspendedafter being publishedwell they included errorsand when they were receivedby the scientific communityother scientists find that they shouldnot be publishedthe second type of the sources are thesecondary sourcesas their name suggeststhey came in the second levelshown here as a circle with light bluearound the primary sourcesthe secondary sources are created bypeople who did not contacted directlywith the subject itselfbut rather with people or Works writtenby peoplepeople who did thatso technically they are one step awayfrom the subjectand this is two-edged swordon one handit might include not real informationbecause a second person will interpretthe information he heard or read orsimply he might hear it or read it wrongthuswhen dealing with secondary sourcesyou need to check this alwayson the other handthis step might give the personwho is studying the subject a wideroverview of the subject itselfand might help him remove any shocks orinacceptable values from the data setand this is good and fine for youif you remember the surveys we havetalked aboutin the previous chapterwell the surveys is a good example ofsecondary sourceshere a scientistspicks up and collects finding of otherscientistswhich are primary sourcesas they are touching the subjectand then he reform his own overviewof the works created by the othersthe third type is called territorysourcesit is far away from the studied subjectwe will refer to the territory sourcesas a circle with no color aroundeverything we had so farand if the secondary sources is anaccount for somethingit is highly probably that it isinaccuratebecause it passed by two differentpersoneach had to understand it and torecreated in his own worldregarding the written worksterritory sources includes encyclopediasand dictionariesthey are useful to provide a largeoverview on the studied subjectnothing more specificwellthat is nicewhy do we address it herethe answer for this question is thatit's important for you to provideinformationon what kind of sources you will useprimary secondary or treatedyou have to use the informationdiscussed above to differentiate thetype of the sources you are going to useand to mention that in your memoryand after you detected what type ofsources you will useyou need to detectwhere you will find your sourcesand let me explain that a little bitmoreif data sources are peopleyou need to detect exactly who are theythis meanshow many age education sexwhere do they livein a city suburb and so onthis is very important to avoid anypossible piousin samples lateradditionally this step helps people whoare evaluating your workto better understand what you have andwhat you are trying to do if you areworking following a descriptive researchmethodyou might need to use specific documentsor specific edition of booksor evenrecords from public our Chiefs if thisis the caseyou need to mention that in detailsso if someone want to do your researchagainhe can simply follow your steps and hewill arrive to the same resultsin some cases you might need to accesssensitive data or rare manuscriptsthat are well preservedin this case you need to have a specialpermission to do that if this is thecase you need to explain how and whyas well as providing a photocopy of anyofficial documentsyou might have in this regardsagain this is essential if someone wantto redo your researchand it will help a lot saving time andeffort the third thing you need to coveris to write down the operationaldefinitionsyou need to keep in mind when you areconducting a research that it is highlyprobably that you will use specificterminologythat includes keywords with meanings faraway from the common use of the worldif you have any keywords from this kindyou can add their definitions hereof course you need to check territorysources such as dictionaries andencyclopediathey are a great sources of operationaldefinitionsforget not to cite them correctlyin your scientific workyou can also add your definitions forspecific termsfor example if you need to Define A NewConcept or add a specific meaning to agiven word that is already used to meansomething elseyou can do that hereof course forget not to be detailed andspecific as much as possiblefourthyou need alsoto detect in detail how you will getdata out from its sourcesif you are planning to do surveysyou need to explain the structure of thesurvey herewhy you have asked these questions andwhat kind of data you are planning tocollect in some cases you might collectdata by letter exchanging all emailsand you need to show how why and whatkind of data you are looking to collectin this case alsofeel free also to write down surveysquestionsletters and emails you have usedin specific appendixes add it to the endof your research workwe will talk about the appendixes in thefirst in the fifth part of this coursewhere we will explain the format of theresearch workthe fifth point you need to cover in thedata source element is only applicableif you are following an inductiveresearch methodin this case you need to build a test toverify the claim you have proposed in anhypothesis formyou have to explain the experimentsettings in a way if someone want toredo the same test againhe will be able to do that with noproblem at allyou need also to explain the variablesyou want to testwhy howand which unit you will use to measureitadditionallyyou need to explain the way the testwill be conductingfor example when you do the test twiceone without the existing of the studiedvariableand then you insert the studied variablein the secondthus you can see how the variable affectthe resultsanother way is to measure the valueof the variable in a controlledenvironmentseveral timesand thenstudy how this value is a changing overtimethere can be several ways to do the testI won't be able to cover them all butwhat is importantis that you explain in details how andwhyyou choose to do the test this wayso the test itself can be repeatedthe sixth and the last point in the datasources elementyou need to cover is directly connectedto the studieson uncountable values such as theknowledgewe have talked about thatin chapter one of these courseand if you remember we used the ladderdiagramshown on the screenwe have study knowledgeon five different aspects such as theskills needed credibility and detailswhen dealing with uncountable ornon-measurable concepts you have tocreate scalesthe scales varies on a linear X betweenfor example low and highshort and longcheap and expensiveand you need carefully to Define each ofthese Conceptsso they can be used with no ambiguitykeep in mindthat although these scales are widelyused in social studiesthey are considered non-scientificbecause they are not reproduciblefor exampleif we take the cheap and expensive scaleit is relative and subjectivewhat it is cheap for someone might beexpensive for another and sothese does not mean that you cannot usethese toolsyou canyou need to define the scale very wellor for example make a large samples toavoid any possible biaswe will cover that in details in partthree of these scores when we will talkabout creating surveys using direct andindirect questionsthe second element you need to cover isthe analysisit means to do three actionsand they should be treated as a pyramidat the bottom of the pyramid you havethe pure data that you need to collectclassify and sort to generate usefulinformation the three actions can bedescribed as following:
First, you need to collect all piece of data related to the studied subject positive or negative; simply collect everything you can get on the studied subject.
The second action is to classify the piece of data you havecollectedinto heterogeneous groups; where all databases that share common properties are grouped together.
The third action you have to do, is taking place inside each group; where you need to sort the piece of data, in a specific way. for example chronologically the objective is that they have a clear meaning, in general the analysis part must cover the following points:

First you need to create rules for whenyou collect the piece of data and whenyou discard themyou need to think of this as you arecreating a filterwhere you will let the data you need topass onlyand all other data not relative to yourresearch will be automatically discardedyou need also to create rules to decidehow to classify the data you havecollected into well-defined classesthis means that you should have alreadydefined the classes that are ready tousewhat you need to do here is to createrulesthat clearly Define with no ambiguitywhen a piece of data is classified intoclass Aand when it is classified under Class Bthirdyou need to create rules to decide howyou will sort the data within each groupafter they are classifiedfor example you can choose to sortchronologicallyor ultraticallybased on the second name of the creatorof the dataand so onthere is no limit for what you can doherebut what it is important is that youmake the sorting system clearso if two different persons try to useit on the same sample of the data theywill get the same results alwaysyou also need to explain why did youchoose to sort your data this way andhow this can have effects on yourresultsand the last point I want to coverin the analyze elementis only applicable if you are using aquantitative Styleyou need to decide what statistical toolyou need to use in order to finishanalyzing the sorted dataof course you need to explain why didyou choose ithow does this tool serves your researchthis means you need to Define two things; first the statistical functions you want touseand we will talk about these functionsin detailsin part four of these scoresso I will keep them for that partbut for now just you need to know thatyou need to mention the functions youare going to use such as average ordeviationthe second thing you need to Define isthe tool you are usingfor exampleyou will use Microsoft Excel or pythonalthough the functions are the same frommathematical point of viewthey differ slightly in theimplementationand these differences can change theresults of your research completelyso what you need to do here is to sayfor exampleI will use Excelfrom Microsoft Office2023 version one two three four fivepoint six sevenat the last point you need to coverin the element of the research method ishow you will present the results you have got let me first tell you what you should avoid and not doin simple wordsyou do not needto present every single resultyou have collectedyou need to understand that this is notthe objective of the result parthowever if you have resultsand you think it is very important toadd it you can go for appendixesand add one or two appendixeswith tables that includes all resultsyou want to showthe second thing you need to dowhen presenting the results is that youdo not show the same results in twodifferent waysfor example if you think that you needto show a table with some valuesthis is a presentation of the results ina text formit is completely okaywhat it is not okay that after thatdirectly you draw a curve thatrepresents the same results graphicallyand add it after the tablein this way you are repeating the sameresults twice and you need to avoid thatin all possible waysthe third thing is to try to do as muchdata as possible in one figure or onetableor even explain them in one paragraphbut this does not mean that you puteverything you have in one place this isnot correct toowhat you have to do is to search fordata that is compatibleand that the compaction has a meaningfor your own scientific workthen you need to present this data onthe same scale or the same table ifpossiblethe more you do that is the easier theanalysis will becomeand the more coherent your findings willbethe fourth point you need to be aware ofis that you need to focus only on whatis directly related to your researchfor exampleif you record the resultsfor a testin during 10 minutesand the event you are looking for ishappening in the seventh minutefor three seconds onlywell you do not have to present theentire 10 minutesit's important to say that the testlasted for 10 minutesand that the event happened in theseventh minutebut when you need to present resultsit's enough to show 10 seconds onlythree seconds before the event the eventitself which is three seconds and threeseconds after the eventthis is almost 1.5 percent of the totaltime which is 10 minutesyou need to do this because here iswhere the real results are the fifthpointis related to coloringand we will cover that in details inpart 5. but I would like to mention ithere because it's very important andunderrated in many casessimply do not color if it is notnecessaryif you are coloring because this makeyour work beautifulyou need to stop it directlythe colors are only used when you haveto differentiate between two quantitiesor values and there is no other way todo thatnever use the colors to draw attentionthis is not a scientific practiceyou have to be objective and do not tryto convince people in any way justpresent what you have and let resultstalk themselvesadditionally if you need to colortable or imageyou cannot simply use any color you likeconsider always that the work might beprinted in black and whitewhich is a possible way to goif you did not choose your colorscarefullyyou might have areas or lines that areindistinguishableso if you want to colorand you cannot avoid that it's a goodpractice to print what you have in blackand white and verify that you have itallso with all that in mindwhat do you needto do here welllet me put that in simple several points:

first, you need to show how you will present your results are you going to use tablesgraphicstext onlyyou have to choose the format and youwill need to explain why you have chosenthe second thingyou have to do is to decide what to showexactlyso you put aside data that is out fromthese categoryeven the data you want to show you haveto divide it into two partsa what goes into result section directlybe what goes into appendixes at the endof your researchif you have appendixes you need also todecide how manyand what to present inside each of themthe third point you need to cover is topresent the data you havein an effective waytry to merge the set of data you havethus compatible data are put togetherin one figure or tableso instead of having four or fivereplicated graphsyou can show that in one pieceand one last point before we finish thislong sectionwe have used the word explain andjustify a lot in this courseI want to highlight them hereif your work is to be evaluated themethod part is where the evaluationprocess will startthat is why you need to focus on thispartwhen we say explainyou have to imagine that the person whois reading your work has no idea whatyou will doso if you have any assumptions orconcepts that you think they are basicand clear for you they might not be forthe others so keep in mind that you neednot to Omit or skip explaining any stepof what we have covered of our justbecause you think it is clearon the other hand forget not that youare addressing researchers and not highschool studentsso it is safe to assume that they arefamiliar with the research domainyour work fits inso there is no need to explaintheoretical Conceptsyou can simply refer to other referencesthat do thatso in simple words explain your methodand the steps you need to follow nextclearlyin justify it logically and avoidexplaining any theoretical aspects andfacts

General notes on designing a method

let me finish this chapterputting everything we have learned sofar togetherany scientific workshould include four main partsin it is called they are the definitionof the problemexplaining how to address itpresenting results and analyzing whatyou have got the first part was covered in theprevious chapter the state of the artwhile the second part how to address theproblem or simply presenting yourresearch method is a subject of thischapterthe research method is normally thesecond part of the journal articleor the second chapter of the researchreport or thesisin all cases it follows the state of theart and should be followed by theresultsa good method section should include four main parts:

First the hypothesisover which the research is builtthis section should include thedefinition of the variablesfunctions and the relationships betweenthemthis means if the value of unvaluable ischanging in a given way how does thatwill affect the other variablesthis should be presented as hypothesisa claim to be testedso you need to avoid presenting yourhypothesis as a result or a factthe second point you need to cover isthe test you need to doin order to collect data to prove yourhypothesisyou need to explain what is the test exactly so if someone in other place or in other time want to redo the same test againhe can rebuild it based on your notesyou need also to precisely define whatare the data you need to collect and how to collect or measure themand why you choose to do that in the first place.
The third point is to explain how youwill analyze the collected dataanalyzing means to decide what to collect from the obtained datahow to sort that into heterogeneous groups where similarpieces of data are grouped together andfinally to sort the data within eachgroup in a specific wayso you need to explain what to collectand howhow to classify and how to sortthe fourth point you need to cover isthe way you will present your resultshere you need to explain what to showand what to omit and of course you needto say why you choose to do thatyou need also to say what will go to theresults part and what will go to theappendixes at the end of your scientificworkthus if you have several app indexesyou need also to explain why you chooseto have these appendixes and what arethe contents of each of themkeep in mind that what you will produce is the first version of the work and it can be simply modified laterbased on how the research Will Go Onso you can have your final definitiveversion before the publishingafter adapting all changes you needand here you have itdesigning a research methodif you are live here this means you havefinished the theoretical part of chapter8. Bravo forget not to check thePractical part of this chaptersee you soon