Chatting with Chatbase over the rationality issue of the cost of science

168

AISDL Team
Email: aisdl_team@mindsponge.info

March 11, 2023

In this article, we present the outcome of our first experiment with Chatbase, a chatbot built on chatGPT’s functioning model(s). Our idea is to try instructing Chatbase to perform a reading, digesting, and summarizing task for a specifically formatted academic document.

The task was performed at the instance: https://www.chatbase.co/chatbot/s41562-017-0281-4-pdf-bz0bqm2-b. The instance was created by uploading [1] to the Chatbase server because it did not seem to work well with the URL (at least for this particular article). The rationale behind the first try with [1] is that the paper represents one of the key principles for operating this scholarly community [2] alongside [3-4].

The results of this experiment follow.

Prompt 1: What do you learn about the rationality of the cost of science in this article?

Chatbase: The article discusses the dilemma between the demand for better cost consideration by scientists and appreciation of the value that the scientific enterprise contributes to society. The public has a right to question the value of their tax contribution to the state’s budget, but there is something not quite right about the public’s irrationality in their consideration of cost justification. The article argues that the cost of science must be put into perspective, especially in developing countries such as Vietnam where the gross domestic product per capita is low.

Prompt 2: What does the article tell about the cost of policy failure when putting the cost of science in perspective?

Chatbase: The article argues that policy failures can cause huge and long-lasting costs to societies. For instance, in Vietnam, people still recall the price–currency–wage innovation that almost brought the whole economy to its knees in 1984–1985, necessitating the extensive reform in 1986 amid a three-digit inflation tsunami. When implementation of science policy fails to deliver scientific knowledge to the public, that failure usually reflects several problems: weak planning and management of resources; inadequate capacity of the science institutions — scientists included — in addressing problems faced by the society, from science, technology, engineering and mathematics education to productivity at work, and to emerging healthcare issues, and so on; and missed opportunities to bring about timely and valuable change, not just in science and technology but in the development of humankind. These failures are genuinely expensive, far exceeding funding for basic research studies, and are well known to hard-working researchers in numerous disciplines.

Prompt 3: Do you agree with the article’s arguments?

Chatbase: As an AI language model, I do not have personal opinions or beliefs. However, the article presents a well-reasoned argument for the value of science and the need to put the cost of science into perspective, especially in developing countries. It highlights the importance of basic research and the potential long-term benefits it can bring to society. The article also emphasizes the need for better policy planning and management of resources to ensure that scientific knowledge is effectively delivered to the public.



Illustration: Chatbase’s disclaimer.

Prompt 4: Find similar articles from trustworthy sources available on the Internet.

Chatbase: Here are some similar articles from trustworthy sources available on the internet:


We offer the following evaluations:

First, Chatbase can read a specifically formatted article quite well, in this situation, presented in multiple columns. It understands the content and can present what it learns in an intelligible manner. The summary can be considered readable and meaningful.

Second, for a more focused query like Prompt 2, requiring it to perform a more complex task of dissecting the text to select appropriate and related pieces, Chatbase can still perform reliably. But it is noteworthy that this AI tends to repeat well-structured phrases, perhaps to avoid mistakes. This risk avoidance may cause parts of the outcome to look like a verbatim quote. (This represents a caveat for practical usage.)

Third, the machine seems to be modeled based on some pre-set ethical ground rules so that the response to the third prompt starts with a standard text disclaiming its responsibility regarding “what it thinks.” (It is not supposed to think or believe?) But right after this disclaimer, the AI provides an excellent answer you can examine above.

Finally, Chatbase does not seem to be able to present meaningful reading materials because all five reference items it gives turn out non-existent.
We hope these initial insights can enable further experiments to learn about Chatbase’s capabilities in aiding researchers to search the huge academic literature.

References

[1] Vuong QH. (2018). The (ir)rational consideration of the cost of science in transition economies. Nature Human Behaviour, 2(1), 5.

[2] Jin R, et al. (2023). An analytical framework-based pedagogical method for scholarly community coaching: A proof of concept. MethodsX, 102082.

[3] Vuong QH. (2019). Breaking barriers in publishing demands a proactive attitude. Nature Human Behaviour, 3(10), 1034.

[4] Vuong QH. (2020). Reform retractions to make them more transparent. Nature, 582(7811), 149.


tags:   Chatbasedigesting an academic document