Good/Bad Research Practices

This is Applicable to all research, not just Psychology

Good Research Practices

Open Materials

Being open about what materials are being used, and how. This includes being open about the setup of the study rooms, what questions are on a questionaire, and basically everything that was used to complete your study. While people who use Open Materials vary on how much detail they give, there needs to be enough information that without asking for help, another researcher would be able to do a direct replication of your study. This allows for others to replicate your research and provide more empirical data to prove your results were accurate.

Open Data

Being open about data that was collected. The purest form of this would include sensitive information, so generally with names and identifying information removed, data will be shared in its raw form for researchers to be able to create their own graphs from it. This prevents researchers from (purposefully or not) changing their data to artificially create a significant response without others being able to catch them.

Preregistry

Preregistering your thesis before doing a study holds researchers accountable for what they are researching. Sometimes, in order to create significant findings, researchers will change their thesis so their study proves it correct rather than incorrect. This is bad because the researchers are being dishonest and the information that this type of study doesn't show a significant change specifically for their original thesis.

Bad Research Practices

HARKing

HARKing stands for Hypothesizing After Results are Known and does exactly what tha name suggests. Changing ones hypethesis after results have been found causes problems with not only honesty but also problems with the results found. It implies that the researcher knew this information all along when that was not the case. Proving your hypothesis wrong is good. It means we know with much more certainty that the results are true because people like to prove themselves right. Admitting your hypothesis was wrong and what the actual results were also allows for growth in the field as ideas may need to be updated and more research should be done to make sure this study was not a fluke. The best way to mitigate HARKing is through Prerestering the hypotheses to prove that the researchers did not change them after the research was conducted.

P-Hacking

This results from questionable data useage to create a significant P-value. This can be acheived by cherry picking data, whether maliciously or simply on accident trying to figure out how the data is working. This create false significant data which can lead to problems with replication and false conclusions being drawn. The best way to combat this is with Open Data practices.

Underreporting Null Findings

Also known as the File-Drawer problem, this is simply removing all subjects who do not demonstrate what the researchers is looking for and either literally or metaphorically hiding the data in a file drawer and never opening it again. Inaccurate data is poor for replication and just like P-hacking, creates false conclusions. The best way to cobat this is once again with Open Data practices.

Why do bad research practices happen?

Researchers need to eat. They need grant money. They need to show their studies are adding to the literature. If a researcher is struggling to pay bills, they are much more likely to fall into these bad research practices because otherwise it is hard to get published. Research journals don't like to report on studies that either confirm what we already know, or provide no significant information. If a researcher is looking at their data and realizing their data doesn't show a significant change, the need to get published makes them much more likely to provide false information, or to omit data that is important to fabricate a significant change.

There is also a large emphasis on new information. This emphasis on new information discourages the replication of past studies, especially direct replication. Without direct replication to expose bad research practices, we cannot know for sure whether or not a researcher is being honest. Statistical validity is incredibly important and yet often forgotten in the chase for new informaiton and for money to support the researcher's livlihoods.

Statistics are hard. Even when a researcher means no harm, sometimes our expectations for results clouds our judgement and we just are dumb with our data. The desire for results to be significant can cause researchers to remove or keep in outliers that they normally would not change. Open data is especially good for solving this as other researchers can come in and check the data being shown.

Within psychology, there was a crisis in the mid 2010's because most research could not be replicated due to the prevalanece of these problems. Things have gotten better since then, especially since 2020, however these problems still exist. Unless the emphasis on innovation, and the importance of money are solved, these problems will continue to be a large problem, not only in Psychology but in all fields of science.

How can we make sure good practices are being followed?

The Open Science Framework was created for the purpose of allowing for good research practices. It allows for open data and open materials to be more easily acessible to everyone, whether you are a researcher or not. In addition, Preregistering your hypothesis prevents HARKing. Both of these resource help keep researchers honest and are making research more acessible to the average person.

Another good research practice to make research availible to the public is Sci-Hub. Looking up where Sci-Hub is now and pasting the DOI link to an article you cannot access is a good research practice. Science should be accessible for everyone. Always.