Is DeSci the Answer to Science's Problem? Part 5 - Replicability and reproducibility

Reproducibility and Replicability

While certain scientific disciplines are resistant to replication, many scientific fields rely on the ability to replicate and reproduce results, which are considered indicators of quality scientific discoveries. Reproducible results can be achieved multiple times by the same team using the same methodology, while replicable results can be achieved by a different group using the same experimental setup. However, the ability to replicate and reproduce results has become a major issue across many science disciplines. Funding initiatives and journals often prefer to support new and groundbreaking discoveries, as replication studies “lack novelty.” (Shilina, 2023). Even when scientists attempt to replicate studies, they often find it difficult to do so. This is increasingly referred to as the "crisis of irreproducibility" (Wesler, 2018). This happens if the original study's methods are too opaque, or if there were too few participants to produce a replicable answer, if the replication requires specific materials or resources e.g. a particle accelerator or a costly, powerful computer, or if the study was simply poorly designed or outright wrong. As a result, science in some fields is suffering from increasingly poor reproducibility. This runs the risk of loss of public credibility and scarcity of public funding (Ioannidis, 2005; Siebert, 2015).

‍

Replication is oftentimes a foundational concept in science, as researchers take an older study that they want to test and try to reproduce it to see if the findings hold up. However, the lack of incentives to engage in tedious replication and the difficulty of replicating studies have led to the crisis of irreproducibility.

‍

With the use of an innovative technological stack, incentivized mechanisms, and transparency, Web3 claims to provide opportunities for open-source data sharing and collaboration, which enables easy replication. Blockchain consensus systems provide a network where members are responsible for reproducing calculations and validating results. Additionally, supplementing academic papers with detailed procedural information can facilitate the replication of experiments. Incentivizing scientists is also essential for reproducibility. Blockchain funding mechanisms like as described above may publicly incentivize replication studies, while academic journals should publish "negative" results. All scientific results should be valued, not just the paradigm-shifting ones.

‍

While these points remain valid, it can be argued that replicability has intrinsic limits and is specific to certain scientific disciplines. For example, it is not clear how Web3 tools could improve the replicability of a sociological, medieval, or ethnological paper. How can replicability be applied to the study of an extinct language? Therefore the crisis of irreproducibility does not affect every scientific discipline. Secondly, even if a study is replicable by design, how can it be replicated if the researchers lack the resources, or requires a significant amount of computing power?

‍

Information Access

‍

The inaccessibility of scientific information is a major concern in the field of research. While science is widely considered a global public good, a significant amount of scientific knowledge is restricted to private databases and paid journals. The flow of scientific value is impeded by the isolation of information. The value of knowledge and its ownership are largely controlled by profit-seeking intermediaries, such as publishing groups (Aspesi & Brand, 2020), (García‐Peñalvo et al., 2010). Conversely, without adequate protection of knowledge ownership, researchers are hesitant to share their data and leverage it as a potential competitive advantage in the future (Zuiderwijk et al., 2020). This, in turn, results in duplicated data production and wasteful expenditure of scientific resources (Ding et al., 2022).

‍

DeSci believes it offers a solution that doesn't rely on centralized gatekeepers for funding or distribution. By allowing scientists to own their data and make it accessible, it is said that decentralized storage can help hedge against catastrophic data loss.

‍

However, even after a study has been funded, conducted, and peer-reviewed, there is still the question of how to make its results accessible to others. Many scientists are dissatisfied with the current system of scientific research dissemination, which is often slow and expensive. The issue of access is particularly concerning, as a significant amount of research is locked away behind expensive paywalls. Some scientists argue that academic research should be freely accessible to all. They criticize the current model in which for-profit publishers charge exorbitant fees for access to journals.

‍

‍Conclusion

‍

Ultimately, the issue of access to scientific information is a complex one, with no easy solutions. The DeSci movement seeks to disrupt the traditional science model by creating an ecosystem where researchers are recognized and incentivized to share their work, making research more accessible and diverse. DeSci proposes a decentralized approach using Web3 technologies, promoting open and citizen science, and providing more funding options. By applying the aforementioned Web3 concepts, DeSci aims to address issues with the current science model and create a more equitable and productive research landscape.

‍

While the DeSci movement has undeniably fostered a culture of innovation and disruption, it seems to lack a clear set of shared values - which is essential for the emergence of a new culture within science. What is the movement united around? Rather than being united around common principles, different segments of the movement are defined by the problems they aim to solve. Therefore, the movement would benefit from rallying behind shared values for a more cohesive approach. Additionally, as modern science serves a mix of industries, it is necessary to evaluate who and what the public good of science should ultimately serve, and how blockchain technologies could aid in achieving this goal. The question also arises of how decentralized the movement should be, and whether DeSci should face these questions as a unified movement or independently. As the DeSci movement grows, so too does the need for open public discussion.