News
Can ChatGPT do my literature search for me?
Since its launch in November 2022, ChatGPT has become increasingly popular. To help you make informed decisions about its use, we're taking a closer look at the usefulness of ChatGPT in discovering, evaluating and citing scientific information.
ChatGPT, next level AI
ChatGPT is a chatbot based on artificial intelligence using a large language model. It is trained on large datasets of human language. Language models have been around for quite some time and most of us use them regularly. These models range from predicting the words you’ll type to more advanced Siri or Google Home assistants. What is new about ChatGPT is its ability to have strikingly natural conversations and to produce fluid texts on a huge range of topics.
General issues with ChatGPT
It is important to keep the following general shortcomings in mind:
- ChatGPT is not a search engine. It has been trained with a large amount of information, but the chatbot doesn’t understand the meaning of the information itself.
- ChatGPT is developed and distributed by OpenAI, an American corporation. This company defines what are acceptable answers and what knowledge is worth including in the language model.
- As of this writing, OpenAI refuses to release information on the data used to train ChatGPT.
- Artificial intelligence models are notoriously known to be biased.
- The quality of the answers heavily depends on the quality of the input (prompts) that are entered into the chatbot.
ChatGPT & scientific literature search
Before rushing to ChatGPT, always ask yourself if this is the most appropriate tool for your task. We’ll discuss the use of ChatGPT in the different phases of searching for literature.
Orientating on a topic
ChatGPT is not able to help you define your information need. ChatGPT has only been trained on data until 2021. It does not have any information on the most recent research output. However, you can use the discussion tool to get some inspiration when formulating your research question. Be aware that ChatGPT is not trained to formulate search questions for searching scientific articles in bibliographic databases. You’ll need to edit and reformulate the answers it gives. You also need to be wary of any biases that ChatGPT might incorporate into your discussion.
Preparing your search
ChatGPT can help you identify appropriate aids or databases to a certain degree. Be aware that it doesn’t always provide complete or exhaustive overviews of available domain-specific resources. Moreover, it can’t tell you if the library subscribes to a specific database. For a complete overview of multi-discipline, discipline-specific databases, resources and subscription information, please use the WUR Library Databases & Collections tool.
ChatGPT is great to help you find good search terms for your search query. Do not hesitate to push ChatGPT to add more terms than the ones in its initial answer. This does not replace the need to consult additional sources, such as a synonyms dictionary or a thesaurus, or to check the jargon used by authors in your field.
Performing your search
ChatGPT cannot help you at this stage. ChatGPT is unable to determine the best search strategy for your information need. What’s more, it can’t search for information systematically and transparently, nor is it good at building search queries.
ChatGPT may provide you with some information on how a database works. However, we advise you to always directly check the help menu of your chosen bibliographic database. Functionalities may have changed since 2021, and the ChatGPT answer may well be outdated.
Evaluating the results you found
You can’t use ChatGPT to evaluate your search results. Evaluating information is a human skill based on obvious and latent criteria that you deem relevant. As a researcher or student, you must decide whether a source is relevant to your work.
You can, however, currently use ChatGPT to clarify difficult texts since it is good at summarizing or reformulating a text.
Citing and referencing
ChatGPT cannot or will not provide sources for the texts it writes. Moreover, something odd happens when you ask ChatGPT to provide a bibliography on a specific topic. It will come up with a list of sources that initially seems accurate. However, upon closer look, these sources are often fake and totally made up.
Moreover, ChatGPT is unable to create reference lists following specific guidelines. Reference management software, such as EndNote or Mendeley, support countless reference styles and are a much better choice.
Writing and communicating
The use of ChatGPT to write scientific articles is problematic. An increasing number of publishers explicitly forbid its use as a writing assistant. For a comprehensive discussion, see this article.
Should I cite and refer to ChatGPT?
Yes, if you've used ChatGPT in your work, you should explicitly mention any use of it. There is currently an ongoing discussion on how this is best done. Reference style guidelines have not yet been updated.
We advise you to cite and refer to ChatGPT as a personal communication or as a computer program. In any case, you need to explicitly inform the reader about your use of ChatGPT, for example, in your materials and methods, including the prompt you used. If the publication allows it, you can also include your chat as an appendix.
Sources
ChatGPT. (2023, March 10). In Wikipedia. https://en.wikipedia.org/wiki/ChatGPT
Parikh, R. B., Teeple, S., & Navathe, A. S. (2019). Addressing bias in artificial intelligence in health care. Jama,322(24), 2377-2378. https://doi.org/10.1001/jama.2019.18058
Roselli, D., Matthews, J., & Talagala, N. (2019, May). Managing bias in AI. In Companion Proceedings of The 2019 World Wide Web Conference (pp. 539-544). https://doi.org/10.1145/3308560.3317590
Blog post The APA-Team - Chat GPT