Can LLMs Provide References for Their Work? An Experiment

Phil Windley's article "Can LLMs Provide References for Their Work?" explores whether Large Language Models (LLMs) like ChatGPT can offer valid references. Windley clarifies that LLMs do not inherently generate references as they create text based on a vast corpus of pre-ingested data, not from active research. An experiment on providing references highlighted that while LLMs can generate hyperlinked citations, many can be erroneous or outdated. Windley suggests that interrogating LLMs through follow-up questions may yield better-supported content, akin to consulting a librarian for dependable sources. Despite some success, ensuring accuracy in AI-generated references remains challenging, as emphasized by examples of broken or shallow links.

Opens on Phil Windley's Technometria