Scientific Method for Wikimedians/Scientific Methods

This is chapter 4 where we will describe the scientific method. The structure of this chapter it includes five main sections:

First we will start by a small reminder from previous chapters on what is the science and what is the method, we will define the two terms based on the findings in the previous chapters.
Second we will put the two words together to form the meaning of the title of this chapter "Scientific method" and this is the subject of the second section of this chapter.
Thirdly we will address the structure of the scientific method, a generic scientific method includes eight steps we will talk about each one of them explaining why do we need it and how do we perform it.
In the fourth section we will talk about the properties of the scientific method this is very useful if we need to distinguish what consider a science from what is not.
Finally as usual we finish with a conclusion where we put what we have found together to create one large image on a solid background.

Reminder: Science & Method

in chapter 2 we have defined science as follows:

“

It is the organized systematic Enterprise that gathers the knowledge about the world and confidences into testable laws and principles.

”

We highlighted the importance of the words systematic and testable. For a thing to be systematic, this means it has a clear order and a well-defined steps that can be followed always with no exceptions.

On the other hand, testable means that a subject can be always tested in order to verify its validity.

from a scientific perspective, we need to always be capable of test defects. It's not enough to be tested and proven valid or invalid once or twice. NO! we should be able to test this validity always, and if you remember the example of the amphitheater. back from chapter one, in this example, the question was: how many seats are there in the amphitheater?

A non-scientific answer was a guess of 500 seats or maybe 700 seats. It is not scientific, because we do not know how the answer was created. please note that the answer itself might be right but because that we don't know the system used to create it, we consider it non-scientific. On the other hand, an answer like the number of the rows and then count the number of the seats in one row and multiply the two together is considered a scientific answer.

first it is systematic we have a clear method that includes well-defined steps
second it is testable if you have the time you can simply go and perform it, please note that for a fact to be scientific does not mean that it is valid you can simply get a wrong answer that is simply because the method might not be valid or simply corrupted but even in this case the method still scientific there is nothing wrong in arriving to wrong results.

For example, in this scenario the suggested method will not give an accurate answer because the number of the seats is not fixed in each row look to the image and you can see the more we go away of the center of the theater the row will have more seats so technically the suggested method will not create their items the solution.

In this case, it is to adjust your method to have more accurate results, we have agreed that science is a continuous process always oriented by scientific facts. In Chapter 3, we have clearly defined what is a research method, where we found that it is a strategic process, or technique used in a collection of data or evidence for analysis, in order to uncover new information or to create a better understanding of the studied topic.

Clearly, the objective of the research method is to create new knowledge. More accurately, a new scientific facts we have also given an example of a research method we have been using in this course so far this is the method helping create new knowledge about the meaning of a word:

first we search in general dictionaries
second we search for the etymology of the word
third research in a specific domain dictionaries then we use what we have learned to construct new knowledge so we have set of steps that are well defined we also talked about the order of the steps

The essential point here is to have a set of well-defined steps that might follow a strict order or might not. Turning to start defining the scientific method. From previous, we have:

science is a way or a tool, it's not a Doctrine nor an ideology

if you say that you follow the scientific method, you are not referring to a belief or points at all. The only thing you are referring to, is that you follow a specific way to obtain knowledge. Thus, the objective of the science is to create new knowledge, and the most important point that this creation is systematic.

This means that it can be achieved following a set of well-defined steps, and what is important here is the validity of the way, because valid way will always lead you to valid risks. Following this way, you have results to test and by following a valid way. The test will always lead to the same results again and again.

Briefly, science is more about the way of creation of the results, more than the results themselves. Having all that in mind, we can ask the following question:

What to consider a science?

This is a very tricky question, in fact from philosophical perspective, science is much more complicated. a paper published in 2013 in the Scientific American Journal, suggested that a science must have six properties, if it does not satisfy all of them, then it's not considered a science by the definition.

Briefly I will address the six points:

A science should have a clear and well-defined terms, each term should refer to a specific concept. It can be only interpreted in one way, this is why we see a specific domain dictionary for every branch of the knowledge.
Quantifiability, means that the science must handle measurable quantities, to do so, there should be a clear standard system of units and these units need to be clearly defined and that's why we see all Sciences always focusing on the quantities and the measuring units.

There is a huge debate on this point, however the mainstream consensus which scientists today agrees upon is that psychology is not a science. By the definition a science need to be based on expertise, and this expertise need to be well controlled. In fact this condition is related to the reason why the article we are addressing was written psychology.

For example, deal with human behavior and instincts such as fear, this is not a quantity that can be measured, and this is why psychology is not considered a science.

In fact, this condition make all branch of knowledge that are fully based on a descriptive method not scientific, and that is why social science, such as history and religious study struggle with the question of the science is asked.

Reproducibility means that anyone who follows this way or implement these steps should have the same exact results. You might think that all branches of knowledge has this property and the answer for sure is no take.

For example, Is the economy reproductible?

The answer clearly is no, because it includes human factors and these factors cannot be predicted 100 percent and that is why economy is not considered as a science as well, however the question will be about is it a science, or not? is still open and there is a huge debate about it, but the mainstream consensus is that it is not a science, the predictability means that a science must be able to predict a set of data based on current and past sets or calculations. Please note that this is not a fortune telling because predictions need to be testable

For example astronomy can tell the exact position of the moon in the sky tomorrow in your city at the midnight. And it works based on calculation and it is not a Lucky Strike

6. The sixth property is that a science must be testable it means that all claims or hypothesis provided by science should be testable sometimes we do not have the way to test it and that is completely okay.

for example when Albert Einstein predicted that huge mass can distort time this is a testable claim theoretically at least.

However, we did not have tools to do that when the idea was first suggested but a decade ago, after almost a century of this claim scientists were able to test it, and it was correct so to sum all of this clearly the word science is much more oriented towards inductive research methods, especially in the applied domain, such as physics, chemistry, and engineering.

The definition of the word has sophisticatedly grown much more beyond the simple definition we have drawn, going deep in the structure of the scientific method, it is an eight-step research method, it is shown on the left side of the screen the first note we can have on the figure is that it is cyclic and that includes two facts:

First it is a continuous way and that is coherent with what we have learned so far about science we have already found that science is continuous.
Second a circle has no starting point this is partly true here we will see in the following slides that you can start from different places in this method especially steps eight and one and let me Define the steps first before we discuss each of them in details the first step is the definition of the problem you need to have a problem or an open question and the research aims to answer it we will discuss the research question in details in chapter 5 and 6 in this course after you detect the problem you need to see what is the current achievement in this regard thus you need to start searching in the available sources this step ends usually with reshaping of the research question and by the end of this step you will have a more specific research question this process is called the state of the art and we will discuss it in details in the chapter 7 of this course we will also discuss what are the reliable sources that you can use in your research in chapter 9 of this course
The third step is to form a hypothesis, we talked in the previous chapter about what is an hypothesis. In simple words a hypothesis is "a claim needs to be validated"
The test is the fourth step, you need to design a test to validate the suggested hypothesis, you will use other theories to do that. You need also to detect what is the data to be collected and to collect it by the end of the test. In many cases you might repeat the same test under slightly changed circumstances or parameters. This is done to see what is the effect of a selected parameter on the studied case.
Even if the test has failed or the results are corrupted, you still need to write the finding down, as the invalid results are also considered results, after the test is finished, You need to analyze the collected results, this means classifying and sorting them. Part 4 of this course will completely focus on this issue.
The results of the analyzers will help you to have a large overview, on the studied case and this will help you to draw a solid conclusion. Either you go for the publishing, if the results are fine, or you go to form a new hypothesis.
Publishing is one of the most important steps, and it is often underestimated. When you publish your results, you share findings with the others.
It is a great way of communication between researchers, the more the Publications are free and open, the more the human knowledge grow faster.
However publishing itself is a complex process, and we will address it in details in the following slides.
Finally the retest is usually conducted by other researchers, who retest the hypothesis and validate the results.

Scientific Method: Definition

Defining the research question, the result of this step will be a question, and the objective of the research is to answer it.

The research question need to be clear and specified as much as possible keep in mind that you cannot solve all the problem in one research work so you need to pick up one specific problem as narrow as possible and try to answer it the research question can be open or closed as well being narrow has no effect of being opened the openness here means that the set of answers are large

For example, if you are doing a research on dark matter, or on the life on the other planets. well, we have not a lot of data about that. so having an open question might be a good strategy.

On the other hand, if you make a research on one specific property of a chemical element, this set of possible answers are normally specified and limited based on our current knowledge.

The research question must also be based on the other's work, you must not start from zero, instead, you need to start from the point where others have stopped. That is why the real starting point might be in the retest step, or in the sources step, where the research question might be significantly changed as the experience of the researcher plays a major role here.

In fact defining a research question is the hardest step in the method, and it has a large impact on the all other steps. That is why young researchers need consoles from expert researchers and experience. Knowing the domain well is vital here, there is a journalic concept called "5W1H" it can help formulating the research question, "5W" stands for: What, Where, Why, When, and Who, "1H" stands for "how".

Please note that the research question should not have all these keywords, normally, a research question can have none of them.

the state of the art

The second step, is the state of the art, this is a technical term, it means The Cutting Edge or the Leading Edge. It is where the current human knowledge have stopped regarding the studied problem.
It is essential that a researcher is aware of the elitist achievement in his domain, so he do not work on a problem that has been solved already. It's very important to avoid replication of the same research. This step is completely based on reliable sources, the researcher entered this step with a research question, based on his current knowledge. Then he needs to look for others works published in reliable sources. Normally, he will get into details in this step, and the research question will be changed and reformulated to reflect this new obtained knowledge. The most important thing to keep in mind is that you will handle a large quantity of this similar information and you need to organize and analyze the collected data in order to make out the best of it.

Normally the finding of this step occupy the first chapter or part of your final report or thesis, we do not start the report showing results instead we show the problem. thus, at the end of this step you will have a well-defined reformulated research question.

forming a hypothesis

The third step in the scientific method is to form a hypothesis, it is a conjecture to be tested. we have talked about it in details in Chapter 2 of this course. Briefly, any problem has possible solutions or answers, normally due to time and cost limitations, you cannot test all possible answers so you need to pick up one or to merge two answers together into one more generic and then try to test it, so the selection is not random. In fact, an expert researcher can avoid corrupted answers or those with dead ends based on his previous experiences. Again this is why it's important for the young researchers to have a director or a supervisor that follow, orient, and validate the research step by step.

When we compared hypothesis to Theory we have also said that a hypothesis is close to physical reality, and deal with objects that can be measured. a hypothesis should not handle abstract concepts, this is essential for the next step, The test.

Physical object can be tested easily, but an abstract cannot. The example of the gravity we have shown before is a good hypothesis, it is something like if relatable or a physical object freely in the earth atmosphere it will fall down due to gravity clearly this is testable the abstract form can be something like gravity is the force that attract objects to each other based on their mass and the distance between them. The force For example is an abstract concept. Finally forget not that you need to test the hypothesis using experiments based on other theories.

test building

The fourth step where we need to build a test or construct a query to collect data about the suggested hypothesis, here you need to Define exactly, what are the parameters you are looking for, how you will create them, and what are the expected values.

Normally the expected values can be drawn from other theories, it's important to precisely define what do you expect to collect. Now you need to perform the test and collect the data and here you might get different output:

first the results can be exactly what you have expected and this is rare and far to happen in this case I highly recommend you to redo all the previous steps again to avoid any possible orientation or manipulation of the results sometimes unconsciously you Orient your test to have the exact results you are expecting this is not research and this is not good and you need to be objective and avoid that by all possible means.
second who might have possible results but far away from what you expected and from my experience this is a good sign because you need to analyze what you have correct the method adjust the test and re-perform it again.
third you might have corrupted results something out of the scope completely and cannot fit anyway in the current research well this means that your method and tests are invalid, and that you need to make a major changes to it.

In all the cases you need to go on to analyze and conclude. this is very important step. So you can learn from your mistakes and understand what is the problem.

Remember that science is more about how to get the results, and not about the results themselves. Sometimes it's recommended that you go and publish invalid results, but you need to show that they are invalid, and to explain why, providing suggestions for the corrections.
By doing so, you are helping other researchers not to do the same mistakes you did. Sometimes they can avoid a dead end unfortunately not all researcherbrs do this.

Analyzing of the collected data

the fifth step, the analyzing of the collected data:

1. first you need to classify and sort the collected data. classify means:

to group data together in heterogeneous groups.

heterogeneous means not similar, sorting means to create an order within each group. For example, you can classify data into dates, names, links and so on..
Then, inside each group, you need to sort the data, for example, in descending order, or in ascending order, or chronologically. You might also have repeated or corrupted values, and you have to remove them. we will discuss how you prepare data for analyzing in specific chapter in part 4 of this course

2. second you need to ask what kind of data you have collected. if it is quantitative, you need to apply statical tools to analyze it. if it is a qualitative you need to represent it on scales so you can search for patterns. We will discuss how we handle the two types in part four of this course.
Before starting analyzing the data, keep in mind that you must be objective, you must not have your personal opinion get into your research. Never try to orient results in one way or another, you must hold a natural point of view regarding your own research. The main objective of the analysis is not to compare collected data to the expected values, but rather to explain why it is closed, far or identical with it.
Again you need to be suspicious about identical results, try to find out why the values went in one way, or another, did you missed something, Is there any parameter that is you need to answer?
All these questions are in the analyzing step, the most important point, that you do not jump directly to the results, this is a bad sign. You need to work slowly and inductively, building on facts, numbers and evidence. In the conclusion, you will be drawn in

conclusions

6. the sixth step you need to be honest when drawing conclusions, don't try to manipulate them. If you don't have the expected value, it is not a personal fail. Remember, you hold a natural point of view, and this is not personal. after you have conclusion in hand you need to decide what is next and here you need to consider factors like time and budget with all that in mind you have two possible ways either you go back to step two or three redefining better the research question again or looking for new possible answers to test again this is limited by the available time you may also go to step seven where you publish results again there is no problem publishing about invalid hypothesis this is useful the most important is that you publish it in this perspective you have tested something and it did not work out for the following reasons and you need also to suggest new solution to enhance the hypothesis for the people who wants to build on your work and if you choose to go for step 7. the publishing you need to be aware of several things.

Scientific publication

1. first the scientific publishing include a process called peer review, it is where other scientists working in the same domain review the work and validate it. That if you process includes if a final decision, either the paper got accepted and published as it is, or it is accepted with modification requested from the author, or it is rejected.

If your paper got rejected you should not take it personally, there should be a problem with your work, and you will get reviewers comments. So you can understand the problem, and work to correct it. Thus having a paper rejected is completely fine.

2. Second in order to publish your results you need to write them down in a systematic way:

first the problem and the state of the art
then the proposed solution and how you are going to test it, and
finally the results conclusion and the future work.

All scientific Works follow this structure, and in order to publish, you need to follow it. After that, you need to decide where do you want to publish your results. is it going to be a workshop, or a meeting on the studied subject? is it going to be a conference on the studied subject? or is it a scientific journal?

Each has its own requirements, and style of writing, where you need to know and respect it.

Retest

Finally the last step is the retest. After your work is completed, and published, Other scientists will retest and validate it. this could be step one, for the others. Based on your work, they will identify new problems, and start their own research.

It is a continuous process, in fact, when people read your work, you are indirectly communicating with them. In several cases, this communication might become real, as scientists connect each other, using by the emails written down at the top of the published work.

A connection like this normally includes idea exchanges, or consultations. Finally, you can also do step 8 on others work, this is a good way to stay aware of the current achievements in a specific domain.

Scientific Method: Properties

Unfortunately, I will not be able to talk about all the properties of the scientific method, briefly, I will highlight what I think is the most important at least from my perspective the properties I am going to Highlights are:

first reproducibility it is the fact that the scientific experiment will always produce the same results whenever you repeat it under the same circumstances the reproducibility can be described as the consistency of measurements
the second property is the predictability it means the ability to forecast results and predict expected values for hypothesis or theories you are suggesting please note that this is not a random process and the way you use to predict the results should be click
the third property is the systemacity it means that this method follows a system that includes several well-defined steps and the process flow is clear during all of these steps
the fourth property is the skepticism it means that the method has always doubts regarding any untested piece of knowledge strictly following the scientific method you need to test and validate every fact you are working with at least once and there is no confidence in the scientific method.
the last property is the external review it is when results obtained by this method are being reviewed by other scientists working in the same domain.

I would also like to honorably mention objectivity, there is no property more important than others at least from my perspective, all properties are connected to each other, and I will start with:

Reproducibility

1. Reproducibility, it's also called replicability or repeatability and as you can see in all the terms the prefix 3 is used this prefix is added to a given word to reflect the fact, that the action will happen again, this is essential here because following the scientific method, you should have the same results every time.

You do the tests under the same circumstances, if you have a subject in an initial state, and you are doing an experiment observational study, or a statistical analysis, you should always arrive to the same results, if not, then, your method is not scientific.

A great example is power, pausing a decade ago, there was a claim, you will have a mental aid if you stand in front of a poster of a superhero, while giving a speech. You might think that this is not serious, in fact, it was. There was dozens of studies to prove that this technique is valid scientifically, and it was not why. Because it fails to produce the same results, which means that it did not pass the reproducibility test, thus, it's considered a non-scientific claim

Predictability

2. the second property is predictability, it means the ability to forecast results based on current and old sets of data

For example when Dimitri mandelieve proposed his famous periodic table in chemistry. He predicted that there will be undiscovered elements to fill empty places in his table. He also predicted some properties of them, and discovered elements.

This is no astrology, nor fortune telling because this prediction was made on solid background. mandelieve did not predict the undiscovered elements based on his personal beliefs, or because he had a spiritual experience. In fact, he studied well the chemical elements and discovered that they can be sorted in a specific periodic way, where properties of the elements can be detected based on the position in the table.

In all the cases the scientific production or hypothesis and must be versioned. For example, in the case of mandelieve, scientists start searching for the chemical elements of these properties, and they were able to prove his claim, that sometimes there can be difficulties verifying claims due to lack of Technologies.

For example when Albert Einstein predicted based on his calculation, that if the mass is big enough, it can change time flow and thus, time is not constant, but relative. When he suggested that, we had a very limited knowledge, or space, and there were no way to prove his claims, if they were right or wrong.

It took scientists more than 100 years, to prove by test, that an object with a large mass, such as a black hole, can do that.

Systematicity

3. The third property is systematicity, it means methodical or simply following a clear method, with detailed well-defined steps, with no ambiguity, if you use the scientific method, you need to create order, in a reasonable way, you should be always able to justify your choices, telling why you have selected this way, and not.

That simply your choices must not be random, or based on luck, and that is why we have said well-defined steps several times so far. You always need to define the concepts you create, the processes you follow, and the units you use to measure quantities.

In fact without systematicity, there can be no reproduction, nor prediction. As both need a clear system to follow, in order to work.

A nice example of the systemacity of the scientific method is the classical impossibles of the geometry constructions. Before explaining that, you need to keep in mind that this is a geometrical Construction, so you are allowed only to use a campus to draw circles, and a ruler to draw straight lines, and not to measure distance, you see how the system is clear, and well defined well.

Having all that said:

1. You cannot divide an angle into three equal angles, or simply trisecting an angle.
2. You cannot draw a square that has the same area of a known Circle, or simply you cannot squaring a circle.
3.You cannot double a cube this means that if you have a cube of a known value, you cannot create geometrically another Cube that has double the size

We know with our current knowledge that this is not possible to do in classical geometry, and because geometry is a science, and it follows a clear and well-defined system. The ancients were not able to solve these problems, or to cheat around them.

Skepticism

4. the fourth property of the scientific method is the skepticism,

it means to examine all the claims and theories at least once before accepting.

The Skeptics or the people who practice spectacism, have created an association, and defined skepticism as:

“

The application of reason to any and all ideas

”

Skepticism is an evaluation of ideas based on the verifiability, and false viability. so if you prove that an idea is valid, you can accept it, and for sure, personal beliefs or unreliable sources or evidence are not counted. We have already talked about Descartes in chapter one, well the Cartesian doubt has its name suggests, was renamed after him.

You can find on the fig. x the method he suggested to practice this doubt, and it is a scientific method. If you look to Step One, you can see the following, accepting only information you know, is true. this is a direct use of skepticism.

Peer review

5. the last property is the external revie or simply the peer review:

it means that the scientific work needs to be reviewed by an entity that is completely independent from the scientists who created the work

In this method we have three entities, the author or the scientist, the Entity where the work will be reviewed, and the audience that will read the final product, which are normally other scientists.

The author write down his findings in an article and submit it to the entity, the main objective is to produce a scientific work. The paper then passed through a test for its validity by technical editors. if it did not pass, It will be rejected. if it passes, it will be passed to
a second round of peer review, where scientists working on the same domain will review the work, and give their opinion. If they agree to publish the work, then the paper goes for the publication step. The work might also be not valid strictly, and simply it will get rejected. the reviewer might also accept the work, and ask the author to adapt several modifications, or to make some corrections on the original work, where they send their comments back to him. The author needs to adopt the feedback into his own work, and will send the answer back to the entity again. then they will be an editorial board of entity, where they have to decide based on the answer of the author to the suggested modifications. If the author did not satisfy the comments, the work will be rejected. If he is, the work will be accepted and will be passed to
The publication step and finally the paper passed to the public audience and become available for everyone.

This is how the peer review mechanism works, dependent process that require a high scale, and organized entity to do it. The objective is to be sure that the published works are valid. In fact, this is a long process, it can take in average between three to nine months, from the submission to publication, depending on the path.

Clearly it's also an expensive process, normally the author pay the entity only if the paper is accepted. The expenses is needed because the system is complex, and needs money to work, and today there is a huge debate about that. Because this system does not provide equal opportunities to the countries, or to the research institutions. The cost at this level can be a challenge. There were several attempts to create new models, unfortunately none is successful enough to replace this current model.

predatory journals

finally I need to highlight the concept of the predatory journals. In recent time, it is a danger threatening the scientific method.

A predatory Journal is a journal claiming that it follows the model presented before, but simply they are not. they care about fees only, publishing almost everything they received, even if it is not valid. So. before sending your paper to any Journal, take a moment and make a fast check to verify if the journal is following this predatory model.

Notes on the Scientific Method

Finally let us put all things we have learned together so we can have a general overview:

First we found that the scientific method is a complex concept much more beyond what we have found before. a method to be scientific, it should not only be systematic and testable, but also to have clear definitions. Also be able to reproduce the same results under the same circumstances again and again, and to predict new results based on current and past sets of results and calculations.
If one property is not satisfied, well the method is simply not scientific. some knowledge branches such as economy and theology are excluded by definition, and are not considered scientific.

We also talked about the steps of the scientific method, it includes eight steps on the right side of the screen I am showing another linear representation of this method, but forget not that this is showing only one iteration, and the method has a cyclic natural.

Finally we talked about the properties of the scientific method, where we focus on the peer review process, where a scientific work is being reviewed by independent entity to verify its validity.