Application of ChatGPT in Information Literacy Instructional Design

Round 1

Reviewer 1 Report

I had the opportunity to review your manuscript on the use of ChatGPT in higher education, particularly focusing on instructional design for information literacy courses in academic libraries. I will be presenting my review in two major sections: (a) an overall evaluation; and (b) specific areas.

OVERALL THOUGHTS

It's clear that you've undertaken a comprehensive exploration of a highly relevant topic, balancing potential benefits with challenges and enhancing the practical relevance of your research with real-world applications. Your manuscript systematically progresses from providing a broad context of ChatGPT's growing significance in higher education to delving into specific research objectives and methodologies, presenting insightful findings, and concluding with meaningful implications and suggestions for future research. This logical flow ensures readers are well-guided through your arguments.

In terms of content, the depth and breadth of your exploration are commendable. You've managed to cover the essential aspects of ChatGPT's application, making a significant contribution to the field. However, to further enhance clarity and reader engagement, consider simplifying complex sentences and ensuring that technical details are accessible to a broader audience. Tools like Grammarly or professional editing services can be invaluable in refining grammar and syntax, ensuring your final manuscript is polished and free of errors.

Your methodology section effectively outlines your innovative approach, though providing more justification for your choices could strengthen it. Likewise, in the results and discussion, explicitly connecting your findings back to the initial research questions and the broader context can offer readers a clearer understanding of the significance of your work.

Then, your conclusion succinctly encapsulates your study's contributions while thoughtfully acknowledging limitations and paving the way for future inquiries. This thoughtful reflection underscores the importance of continued exploration in this rapidly evolving field.

As for writing and clarity, I’d recommend that the authors ensure consistency in terminology and formatting. For example, when referring to "ChatGPT" or "AI," it would be necessary to maintain the same style throughout the document.  Also, I’d encourage the authors to review the use of commas for complex sentences to ensure they aid in readability rather than hinder it. For instance, check if clauses are correctly separated to enhance understanding.  As we have it, while the overall quality of writing is just fine, the paper gets tiresome to read in some places.

SPECIFIC THOUGHTS

Abstract: Your abstract concisely outlines the study's purpose, methodology, and findings. Consider mentioning the significance of your findings and their implications for future research or practice in information literacy.  But please consider simplifying complex sentences for clarity. For example, the sentence "The purpose of this paper is to investigate the application of ChatGPT in a small-scale environment and a specific field: instructional design for information literacy courses in academic libraries" could be made clearer by breaking it into two sentences if the sentence structure becomes too convoluted. It gets tiresome to read run-on sentences. With this said, "Generative Pre-Trained Transformer (ChatGPT) is a statistical model for generating new data based on variable relationships in a dataset" might confuse readers unfamiliar with AI terminology. Consider explaining briefly that ChatGPT generates text that mimics human writing styles based on the vast amount of data it was trained on.

Introduction: You've set a strong foundation for your study by highlighting the relevance of ChatGPT and AI in higher education. However, perhaps you could consider clarifying early on how your study contributes to the existing body of knowledge rather than leave readers wondering what sets this paper apart from others.

Materials and Methods: Please consider providing more detailed justification for your methodological choices to strengthen this section. For example, why was ChatGPT chosen over other AI tools? What makes the specific setting of your study particularly suitable or interesting for this research?  I may have missed these nuances, but including such information would be critical in papers of this nature.  Also, while the paper outlines the process of model training and developing a custom chatbot, more detailed discussion on the technical challenges, the dataset's comprehensiveness, and the customization process might enhance understanding and replicability.

Results as well as Discussion: Pls ensure to connect your findings back to your research questions explicitly. Discuss not only what your results are but also what they mean in the broader context of AI in education. How do they compare with existing literature?  I feel that more in-dept discussion is necessary in this context because, upon reading the Discussion section, I was left a bit dissatisfied, feeling that perhaps this section only “scratches the surface” rather than delve into the issues with rigor, referencing back to the research questions and existing literature. Line 230 (in Results), "Compared to the results of the Eager and Brunton case study..." might benefit from briefly summarizing the findings of the referenced case study for context, ensuring readers understand the comparison with ease—rather than have them go back and review. Finally, the paper presents immediate findings from the implementation of ChatGPT and a custom chatbot model. However, while I may have missed this, an exploration of the long-term impact on student learning outcomes, academic integrity, and library services would provide a more comprehensive view of AI's role in higher education.

Conclusion: Here, perhaps you could consider summarizing the key findings and their implications a bit more thoughtfully. It is nice that limitations of the work is introduced; however, as we have it, it does not necessarily help the readers appreciate WHY those two points are problematic, in what ways, etc.  I say this because, while this is a short paragraph, without some explanations, readers may wonder, “Wait, so the study isn’t that meaningful unless some paid version is used?  Then why are we reading this?”  You see, if the reasons outlining WHY/HOW these shortcomings matter and WHAT implications they have on the results and future research, this short paragraph can cause great skepticism.  So, I do belive it would be in the authors’ best interest to be clear about the nature of the limitations AND its implications.

With this all said, the problem I have here is that little to no attention seems to be paid to discussing/exploring the ethical dimensions in this section.  While the paper touches ever-so-briefly on proper attribution and responsible use in the beginning, a deeper dive into the ethical implications of AI in education—including privacy concerns, potential biases in AI models, and the impact on employment in academic libraries—would add a critical dimension to the discussion in concluding this paper.

All in all, I think the paper is probably in good shape, but a second round of reviews is definitely warranted, in my opinion.  This is because the paper appears to be missing some key information and considerations, which will greatly impact the overall significance of this work.  I wish the authors best of luck, and I will be looking forward to reading the revision.

Please see above for my feedback no writing.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

1. This paper makes a contribution to the evolving discourse on the integration of ChatGPT, within the context of instructional design for information literacy in academic libraries. Few things should be improved: 1. Comparative study must be included. 2. Explain novelty om custom chat model development. Why low-code platform flowise-ai is used? ChatGPT API is costly. Authors should use open-source free API services.

3. Result section is poorly framed. Data centric approach is needed.

4. Conclusion is too large, it can be shortened.

5. Use most recently more articles into the references.

Minor changes needed.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Thanks for the opportunity to review the revised manuscript.  It appears to reflect relatively substantial improvements in response to my feedback, particularly in clarifying complex ideas, enhancing the logical flow, and providing detailed justifications for the methodological choices made. The revised paper also attempts to address ethical considerations although, in my opinion, there's room for more extensive discussion on this front in the current context.  But it is a minor concerns in the bigger scheme of things. The paper now better communicates its contributions to the field and the practical implications of its findings, aligning closely with my recommendations.  I find the revision to be adequate.

Reviewer 2 Report

The authors have modified the article than earlier version. 

1. They have improved the abstract.

2. Two research questions are included.

3. Materials and methods section is improved.

4. Discussion section is also modified.

Minor changes needed.