Conducting High Impact Research with Limited Financial Resources (While Working From Home)

The Covid-19 pandemic has far-reaching implications for researchers. For example, many researchers cannot access their labs anymore and are hit by budget-cuts from their institutions. Luckily, there are a range of ways how high-quality research can be conducted without funding and face-to-face interactions. In the present paper, I discuss nine such possibilities, including meta-analyses, secondary data analyses, web-scrapping, scientometrics, or sharing one’s expert knowledge (e.g., writing tutorials). Most of these possibilities can be done from home, as they require only access to a computer, the internet, and time; but no state-of-the art equipment or funding to pay for participants. Thus, they are particularly relevant for researchers with limited financial resources beyond pandemics and quarantines.


Conducting High Impact Research with Limited Financial Resources (While Working from Home)
Lower student numbers and a general economic recession caused by global quarantine measures to control the Covid-19 pandemic are putting a lot of pressure on universities and researchers (Adams, 2020). For example, lab access as well as research budgets are suspended, and recruitment of diverse samples, even online, might be more difficult (Lourenco & Tasimi, 2020). The lack of funding can hamper the quantity and quality of research output and cause numerous issues. Indeed, early career researchers identified having few resources as a major reason they struggle with publishing and therefore advancing their careers (e.g., Lennon, 2019;Urbanska, 2019). Furthermore, a lack of resources and funding can have a detrimental effect on the mental health of PhD-students (Levecque et al., 2017) and academic staff (Gillespie et al., 2001). However, while having substantial resources arguably facilitates primary research (i.e., researchers collecting their own data), it is possible to conduct high-impact and high-quality research with little or no funding as well while working remotely from home.
In this paper, I provide nine examples of how high-impact research in biomedical and social sciences can be conducted with limited materialistic resources. That is research which is published in prestigious journals (e.g., journals that are among the top 25% in a given field according to Scopus). The list of examples presented is neither meant to be exhaustive nor representative. Nevertheless, I am hoping that the examples provided can inspire researchers to think of new research questions or methods and allow them to take some pressure off themselves. I discuss how people can conduct high-impact research using information provided within published work, with data collected by others (secondary data analysis), with researcher's expertise and interests (e.g., tutorials), as well as with simulation studies. Table 1 provides an overview of the nine approaches which are discussed in detail below. Table 1 How to conduct high impact research with limited resources: An overview Of course, whether it will be easy or difficult to acquire the necessary skills to write a paper within any of the nine approaches discussed below depends on a range of factors such as previous experience, complexity of the research question, and data availability to answer a specific research question. That is, it can be easier to publish a paper using for example secondary data because no data collection is required, but if a researcher is unfamiliar with specific statistical analyses such as multi-level modeling and the relevant literature, it might take longer than collecting primary data and writing a paper up.

Meta-analyses
A meta-analysis is a quantitative review of the literature on a specific topic. The main aims are to estimate the strength of an effect across studies, test for moderators, publication bias, and to identify gaps in the literature (Borenstein et al., 2009;Simonsohn et al., 2015).
For example, researchers might be interested in testing which emotion regulation strategy works best (Webb et al., 2012) or whether psychotherapy is better than pharmacotherapy in treating depressive and anxiety disorders (Cuijpers et al., 2013).
To perform a meta-analysis, researchers tend to start with a systematic literature review 1 , identify relevant articles and ideally unpublished studies, extract descriptive statistics (e.g., sample size, descriptive statistics) and information of relevant moderators (e.g., country of origin, sample type), and finally meta-analyse across samples (Cheung & Vijayakumar, 2016). Thus, researchers need only a computer and access to the internet to perform a metaanalysis 2 . Nevertheless, a meta-analysis is hard work and a range of pitfalls, such as an unsystematic literature review, must be avoided. Luckily, guidelines exist which help to overcome pitfalls (e.g., PRISMA guidelines) (Moher et al., 2009) as well as how to reduce publication bias (Stanley & Doucouliagos, 2014), and powerful software can facilitate the 1 A systematic review alone without a quantitative synthesis can be useful as well. For example, when there are only a few or too diverse papers published in a specific topical area, a qualitative summary only can be informative.
2 Many research projects in general and meta-analyses in particular can benefit from collaborations. For example, any coding of studies is ideally done by at least two researchers. Finding reliable collaborators can be an issue for people with a smaller research network, especially in times when labs are closed, conferences cancelled, and home office is encouraged. There are many ways how potential collaborators can be identified (Sparks, 2019). One is to first identify researchers who already published relevant articles or graduate students that are listed on the lab pages of more senior researchers and start follow them on social media to get an impression of their views and beliefs on various issues. Then reach out to them via email to gauge their general interest and, in case of a positive reply, schedule a video chat. If this is going well, it might be useful to discuss early on who contributes what and authorships. Who does what? Who gets to be first author? It is worth keeping in mind that shared (first) authorships are possible. statistical analysis and visualisations (e.g., the R-package metafor) (Viechtbauer, 2010). Also, pre-registration of meta-analyses is possible (Quintana, 2015;Stewart et al., 2012).
Meta-analyses are useful for many disciplines because they provide a robust effect size estimate of a specific research question. Also, meta-analyses typically attract more citations than empirical studies (Patsopoulos et al., 2005). Meta-analyses that identify moderators or develop new taxonomies based on the literature can be especially influential (Webb et al., 2012). If meta-analyses already exist in a given subfield, researchers can consider performing a second-order meta-analysis: A meta-analysis across meta-analyses to get even more robust effect size estimates (Hyde, 2005) or to test for moderators such as cultural factors (Fischer et al., 2019).
Additionally, meta-analyses come with secondary benefits for meta-analysts themselves. Everyone who has performed a meta-analysis knows that identifying the relevant information such as descriptive statistics or effect sizes in empirical articles can easily get frustrating because authors often do not report sufficient information. This can mean that otherwise perfectly suitable studies cannot be included in a meta-analysis. Thus, every PhDstudent in biomedical and social sciences working on a quantitative research question, might want to consider performing a meta-analysis at the beginning of their program to teach them the importance of reporting detailed results and ideally also of sharing the (anonymised) data openly.
One objection against the claim that every researcher with a computer and internet access can perform a meta-analysis, might be that particularly less affluent institutions can not pay the high subscription fees for many scientific journals. However, as the number of preprints and open access journals are increasing, paywalls become less of an issue. Further, researchers from less affluent institutions can collaborate with colleagues from institutions with access to the required journals. Finally, while legally questionable, researchers have found a way to bypass the paywall of most scientific publishers (Bohannon, 2016).

Scientometrics
Scientometrics is an interdisciplinary scientific field that analyses scientific publication trends using various statistical methods. There are countless ways that publications can be analysed. I will discuss a few of them in this and the next section. For example, one line of publications investigates how often so-called statistically significant findings occur: Are 'positive' results increasing "down the hierarchy of the sciences" (Fanelli, 2010a), does publication pressure increases scientists' bias (Fanelli, 2010b), or are p-values just below .05 occurring more frequently than one would expect assuming no publication bias (Simonsohn et al., 2015)?
A prominent example of scientometrics is citation analysis. For example, what predicts whether a scientific article gets cited? Is it whether it is published open access (McKiernan et al., 2016) or whether sample sizes are large (Hanel & Haase, 2017)? All relevant information to address these questions can be extracted from the articles of a specific scientific (sub-)field and sometimes even from meta-analyses (Hanel & Haase, 2017).
Typically, questions such as these are investigated separately in each subfield such as internal medicine ( Van der Veer et al., 2015).
Similar research questions can be tested with citations aggregated on a journal level.
The amount of citations articles published in the last 2, 3, or 5 years in a specific journal are averaged and used as quality indicator of that journal (i.e., the so-called Journal Impact Factor or, more recently, Cite Score) (Teixeira da Silva & Memon, 2017). However, it is an empirical question in its own right whether these quality indicators are associated with other quality indicators of empirical studies (Brembs, 2018), and whether there are unintended consequences of ranking journals based on alleged quality (Brembs et al., 2013). Research questions such as these, and others like them, can again be tested with limited resources as they often only require the coding of published articles (e.g., on some quality indicators).
Furthermore, some journals are asking reviewers to assess the quality of a manuscript quantitatively when providing their review. If one has access to how reviewers evaluate manuscripts it is possible to assess whether reviewers agree on the quality of the manuscript (Bornmann et al., 2010), or whether reviewers (Reinhart, 2009) can predict how well a paper or researchers get cited in the following years.

Network and Cluster Analyses of the Published Literature
Yet another way to perform research at low costs is to perform network and cluster analyses. A network "is an abstract representation of a system of entities or variables (i.e., nodes) that have some form of connection with each other (i.e., edges)" (Dalege et al., 2017, p. 528). Nodes can represent a variety of things including people, journals, or keywords. In short, network analyses typically reveal how strongly objects are associated. For example, from combing through keywords, journal names, citation counts, or country of origin of authors from hundreds or thousands of articles, it is possible to identify emerging themes and track a disciplines evolution. This can show which keywords are more frequently used together, which journals cite each other (journal citation network analysis), or researchers from which countries collaborate together more frequently (Cipresso et al., 2018). In addition to this, these analyses also allow researchers to identify potential gaps in the literature (e.g., if two or more keywords are not linked in a keyword network analysis, this might indicate a potential gap in the literature). Finally, moving beyond network analysis, extracting the full text of scientific articles can be used to analyse their readability (Plavén-Sigray et al., 2017) or to estimate the accuracy of the reported statistical information (Nuijten et al., 2015), for instance.

Data Made Available by (Research) Organisations
Over the past decades, the number of large, openly available surveys relevant to the social sciences and researchers interested in the mental health of people has grown rapidly.  (Gonzales et al., 2003). Basabe and Valencia (2007) correlated the country averages of Hofstede's (2001) cultural dimensions, Inglehart's (Inglehart & Baker, 2000) values as measured by the World Values Survey, and Schwartz's (2006) cultural value dimensions, with indices of human development provided in the United Nations Report (e.g., 2014) and De Riviera's (2004) culture of peace dimensions. Such analyses allow to identify, for example, what predicts whether a country is more likely to engage in wars and supress its own population. As all the prior mentioned datasets are openly available, it is relatively easy to reproduce all analyses and come up with new research questions that can be answered with these datasets. Further, it is possible to pre-register secondary data analysis (Van den Akker et al., 2019), The complexity of statistical analysis depends on the research question and data. For example, testing hypothesis with large (N > 40,000) datasets containing data from various countries typically requires multi-level modeling, because participants are nested within countries (for an example paper see Rudnev & Vauclair, 2018). In contrast, when two or more datasets have been combined, and, for example, only country-level data is available, researchers typically rely on correlation and regression analyses (e.g., Basabe & Valencia, 2007;Inman et al., 2017). Recommendations for performing secondary data analyses exist, for example, for social studies (Fitchett & Heafner, 2017), medical sciences (Cheng & Phillips, 2014), human biology (Rosinger & Ice, 2019), and qualitative research (Sherif, 2018).

Reusing Data
This point is similar to the one above, except that it focuses solely on reusing data collected by either researchers' own lab-group or that were shared by other researchers.
Typically, the data were collected to answer some pre-defined research question, but not for the additional analyses someone thought about only after data collection. Further, if one has access to several similar datasets that also included some demographic information which may have been reported but were not the focus of the main paper(s), the datasets can be combined and reanalysed to test for differences and similarities between the demographic groups on several of the primary variables (assuming this analysis has not been reported in the primary papers). In a similar refrain, if several primary studies included a scale with, as a rule-of-thumb, more than eight-items per dimension, it is worth considering to test whether only some of the items of each dimension is as reliable and valid as the original scale (Coelho et al., 2018).
Both types of research questions (comparisons across demographic groups and scale validation) along with several other ones, can also entirely be addressed with datasets openly shared by researchers. Google has created a search engine that searches for open datasets (https://datasetsearch.research.google.com/; see also https://dataverse.harvard.edu/) which can be directly used or combined with other datasets. To the best of my knowledge, the number of articles based on re-using data collected by other researchers is still very limited. However, since more and more researchers are sharing their data and search engines allow to identify potentially relevant datasets, the number of papers based on other researchers' data is likely increasing.
An additional way to reuse data is to verify the results of an already published article with the data collected by the original authors. The necessity of this is illustrated by an attempt to replicate 59 macroeconomic papers using the original data (Chang & Li, 2017).
Only 29 papers were replicated, even with the help of the original authors. Such an initiative would be very useful in other scientific fields too. But also replicating the results from single papers has been encouraged. For example, the journal Cortex has recently announced a new article type "Verification Reports" which reports independent replication of the research findings of a published article through repeating the original analyses. This is to "provide scientists with professional credit for evaluating one of the most fundamental forms of credibility: whether the claims in previous studies are justified by their own data" (Chambers, 2020, p. A1).

Web-Scraping
When people use social media or use a search engine, they produce data. Some of the traces people leave online can be relatively easy scrapped (i.e., extracted or harvested) and allow us to answer research questions we would not be able to answer with traditional approaches (cf. Paxton & Griffiths, 2017). Webpages from which data can relatively easily obtained include: Twitter, Reddit, as well as Google Ngram Viewer, and Google Trends. For example, researchers used Twitter to test whether survey responses of social media use are accurate (Guess et al., 2019), and predictors of solidarity expressions with refugees (Smith et al., 2018). Further, Google Trends -which analyses how often people searched using Google for specific terms in one or all countries on a specific date -was used to test whether online health-seeking behaviour predicts influenza-like symptoms (Ginsberg et al., 2009) and whether Google searches predict stock market moves (Preis et al., 2013).

Tutorials
To conduct high quality primary research, researchers often need to acquire specific skills. Examples of expert knowledge and skills that researchers may have, include recruiting participants from hard to reach populations, setting up testing equipment which often includes programming skills (e.g., in cognitive psychology or neuroscience), and analysing the data.
Without a good mentor, helpful peers, or informative tutorials, acquiring such skills can be cumbersome. Sharing this knowledge by writing blogposts or peer-reviewed articles (e.g., tutorials) can therefore be very useful once we acquired some specialist expert knowledge.
For example, what recruitment methods work well to get couples to participate in unpaid online or lab-studies (e.g., flyer distribution on places where people are waiting anyway such as train stations, schools, on campus, or targeting specific groups on social media)? What are best practices for writing reproducible code? How should data of a specific format be analysed? Writing a step-by-step tutorial (assuming this does not yet exist), ideally with some concrete examples, may be often cited and help to establish a reputation as an expert.
Expert knowledge also allows researchers more easily to write commentaries on various topics. Popular commentaries include topics such as the scientific publication system (Lawrence, 2003) or cargo cult science (Feynman, 1974).

Theoretical Papers
Related to tutorials, scientists can integrate and advance research in theoretical papers.
Theories are important because they help us to see "the coherent structures in seemingly chaotic phenomena and make inroads into previously uncharted domains, thus affording progress in the way we understand the world around us" (Van Lange, 2013, p. 40). In contrast to tutorials which typically focus on solving specific problems such as conducting a specific analysis, theoretical papers can both solve problems through integrating apparently contradictory findings into one broader framework, but can also 'cause' problems through making novel predictions and is therefore crucial for new empirical discoveries (Higgins, 2004).
Prominent examples include the theory of planned behavior (Ajzen, 1985(Ajzen, , 1991 which aims to explain planned human behaviour or cognitive dissonance theory (Festinger, 1957) which aims to explain how people deal with internal inconsistencies. However, developing a formalised and testable theory can be challenging. For example, van Lange (2013) argues that good theories should contain "truth, abstraction, progress, and applicability as standards" (p. 40) and provides recommendations how this can be done. Smaldino (2020) discusses various options how verbal theories can be translated into formal models.

Simulation Studies
Another way to get data without needing to conduct a study, is to simulate data from hundreds or often even thousands of studies using specialised statistical software such as the freely available program R (Feinberg & Rubright, 2016). In a simulation study, data are generated that may or may not reflect real data. Thanks to the advances in processing capacities, many simulation studies can be done nowadays without needing to access a supercomputer. Simulation studies have been used to answer a range of questions, such as which mediation test best balances type-I error and statistical power (MacKinnon et al., 2002) and the pitfalls in specifying fixed and random effects in multilevel models (Schmidt-Catran & Fairbrother, 2016).
The first step in a simulation study is typically to define the problem. For example, a researcher might be interested in exploring which, out of multiple tests that serve the same purpose, has the lowest type-I and type-II error rates. Other steps include making assumptions, simulating the data, evaluating the output, and finally disseminating the findings (for tutorials see Beaujean, 2018;Feinberg & Rubright, 2016). In short, simulation studies are an effective way to conduct cheap research, but require advanced programming skills.

Conclusion
In the present paper, I provide suggestions of how impactful research can be conducted with limited resources and while working remotely. The above list is not meant to be exhaustive but will hopefully provide some examples that might inspire researchers to consider alternative ways to research phenomena they find interesting. Importantly, encouraging researchers to conduct research using more secondary data analysis does not disregard primary empirical research. However, it is sometimes not feasible for everyone to conduct well-powered empirical studies because of a limited amount of resources. Thus, being aware of alternative ways to conduct research can help researchers in this situation, to get to a point in which they can compete with researchers who have access to more resources (cf. Lepori et al., 2019). Ultimately, it might make science more egalitarian, because it also allows researchers from financially less well-situated institutions to publish in prestigious journals.