Wikipedia talk:Reliable sources

AI-written citations?

I was adding an event to an article (Special:Diff/1220193358) when I noticed that the article I was reading as a source, and planning to cite, was tagged as being written by AI on the news company's website. I've looked around a bit, skimmed Wikipedia: Using neural network language models on Wikipedia, WP:LLM, WP:AI, WP:RS and this Wikimedia post, but couldn't find anything directly addressing whether it's ok to cite articles written by AI. Closest I could find is here on WP:RS tentatively saying "ML generation in itself does not necessarily disqualify a source that is properly checked by the person using it" and here on WP:LLM, which clearly states "LLMs do not follow Wikipedia's policies on verifiability and reliable sourcing.", but in a slightly different context, so I'm getting mixed signals. I also asked Copilot and GPT3.5, which both said AI-written citations neither explicitly banned nor permitted, with varying levels of vaguery.

For my specific example, I submitted it but put "(AI)" after the name, but I wanted to raise this more broadly because I'm not sure what to do. My proposal is what I did, use them but tag them as AI in the link, but I'm curious to hear other suggestions.

I've put this on the talk pages in Wikipedia:Using neural network language models on Wikipedia and Wikipedia:Reliable sources. SqueakSquawk4 (talk) 11:36, 22 April 2024 (UTC)

For me it comes down to a case by case basis. If AI is being used as part of the process, but ultimately the article is from a real person and editor then it's probably fine. The issue comes from articles completely written by AI with little or no oversight.
The site has an AI disclaimer[1] where they say they only use AI in the first way, not the latter. So on that point I would think it should be ok. -- LCU ActivelyDisinterested «@» °∆t° 13:02, 22 April 2024 (UTC)

@SqueakSquawk4, do you absolutely need that source? If you can find a better one, then I suggest using the better one instead. WhatamIdoing (talk) 02:03, 24 April 2024 (UTC)

A) I kinda do, it's the only citation I found with everything in the same place. If I took it out I'd have to put in 2 or 3 seperate citations to not leave something uncited.

B) I was going trying to ask more generally, with the one I found as just an example rather than really the focus of what I was asking.

C) @ActivelyDisinterested Thanks, didn't spot that. SqueakSquawk4 (talk) 12:32, 25 April 2024 (UTC)

• AI = NO Considering the 'hallucination" issue that LLMs have, and, in fact, considering how they are constructed at a base logic level, I would categorically treat any "AI" source as intrinsically non-reliable. If a news agency is found to be using "AI" constructed articles on a regular basis then that source should be deprecated. Simonm223 (talk) 12:42, 25 April 2024 (UTC)

  Simon, I think black-and-white rules are easy to understand, but hallucination is only an issue when it appears. AI sometimes generates false claims. If it's writing something you know to be true and non-hallucinated (e.g., because you've read the same claim in other sources, or because it's the kind of general, non-controversial knowledge that the Wikipedia:No original research says doesn't require a citation, like "The capital of France is Paris"), then that problem is irrelevant.

  @SqueakSquawk4, editors might accept this source, especially in light of what AD says. However, if the content is important to you, you might consider using the three other sources instead of (or in addition to) this one, to make it harder for someone to remove it on simplistic "all AI is wrong and bad" grounds.

  As a tangent, we've never defined reliable sources. Unlike an article, which would doubtless begin with a sentence like "A reliable source is...", this guideline begins with "Wikipedia articles should be based on reliable, published sources". I suggest that the actual definition, in practice, is "A reliable source is a published source that experienced Wikipedia editors accept as supporting the material it is cited for". Some editors strongly oppose AI-generated sources, and we can usually expect that some editors won't take time to understand the nuances behind using AI as a convenience vs using AI unsupervised to generate content wholesale.[*] Therefore, I'm uncertain whether it would considered reliable if it were ever seriously disputed.

  [*] This is happening in the real world, with a student accused of plagiarism without any evidence except Turnitin thinking it was AI-generated,[2] so it'll happen on wiki, too. WhatamIdoing (talk) 17:10, 25 April 2024 (UTC)

  I read on some AI-test tool I tried a caveat, something like "don't use this to punish students." Gråbergs Gråa Sång (talk) 18:22, 25 April 2024 (UTC)

  IF you have double checked the AI generated source, and it a) actually exists, b) is reliable and c) directly supports the information in the article… then it doesn’t really matter how the source was “generated”. The key is that a human has checked it. Blueboar (talk) 12:40, 3 July 2024 (UTC)

Two possible additions

Under Scholarship, the bullet point Reliable scholarship should consider that books on academic subjects are often reviewed in journals covering the appropriate academic discipline. These can often highlight the value of any particular book. Since some academic publishers seem to be less reliable on the quality of their output than they once were, this is a useful verification of the content of a book (versus a properly peer-reviewed paper).

In rare instances, a review may be so damning that we would probably all see the book in question as not being a suitable source. (See [3] for an example of such a review). Other reviews actually turn out to add to the content of an academic book by giving a second supporting opinion on some content. (See [4] for an example – search for "observations that may not be widely-understood and accepted, but are nevertheless accurate" to see this in action. This example also shows how a review might highlight the strengths and weaknesses of a work, so further helping the editor in how to use a source.)

Therefore I suggest the "Reliable scholarship" paragraph should have added:

• Books are often reviewed in academic journals that cover their subject – these reviews may help an editor understand the strengths and weaknesses of the work in question.

The second suggestion is more concise. The last sentence of Citation counts should be expanded to say

• The number of citations may be misleading if an author cites themselves often, or if a work is frequently cited by those who disagree with or disprove it.

Generally, to disagree with the work of others, you have to cite them. This obviously increases the citation count, especially if a lot of other authors publish in disagreement. ThoughtIdRetired _TIR 15:14, 8 June 2024 (UTC)

Hiding conflict of interest? You've been unilaterally removing Schaffer from Wikipedia for a very very long time based on that single review, in your ongoing WP:TENDENTIOUS edits against a very specific topic that you decided you do not like. - OBSIDIAN†SOUL 02:39, 21 June 2024 (UTC)

ThoughtIdRetired on 15 June 2024 added "Some books on academic subjects are reviewed in peer-reviewed journals, so giving additional information on the reliability of their contents." Obsidian Soul on 21 June 2024 reverted. ThoughtIdRetired on 29 June 2024 re-inserted. I believe that re-inserting reverted PAG insertions, without getting consensus, should generally be opposed. Peter Gulutzan (talk) 14:08, 29 June 2024 (UTC)

My own interpretation this, but the edit by Obsidian Soul (OS) was part of a rather unpleasant dispute which has resulted in them, for the present, retiring from Wikipedia. I have encountered ample support for the example raised (Shaffer) not being an RS, for instance[5], together with at least one instance of thanks when I removed Shaffer as a reference (with a full explanation and links to the review in the edit summary) on some articles not of interest to OS.

The more important point on the sentence added is that reviews in academic journals can highlight the strengths of a potential source. This can be particularly useful. This really uses the concept mentioned in WP:HISTRS (or more precisely, WP:HSC. The real example of usefulness is the second example I give, where not only the book but the review itself could be used as a source for the comparative merits of square rig and fore and aft rig in the late 18th and 19th centuries.

Do you think that the added sentence is bad advice in any way? If so, please explain your thinking. ThoughtIdRetired _TIR 15:53, 30 June 2024 (UTC)

I don't think that this is limited to academic publishers or to reviews in academic journals. It's no different than any other source. You have found a source, and you want to know whether it's any good. How can you do this?

'Source' means three things on wiki: author, publisher, and document. Consider the author. Consider the publisher. Consider the document. How do you do that last step? If it's a book, look for book reviews. If it's a peer-reviewed paper, look for citations and commentary/letters about it. If it's in a non-scholarly periodical, look for the kinds of sources we would use to write about their reputation and scope. And so forth. This is just ordinary "introduction to evaluating your source" work. It is not special to the academic press. WhatamIdoing (talk) 16:38, 1 July 2024 (UTC)

Thanks User:WhatamIdoing. For the specific example of Shaffer, see[6] On the point raised here, whether the added sentence should or should not be included, do you think it is helpful? For an editor who, perhaps, is not trained in assessing sources, it may not occur to them that academic books are reviewed in academic journals. As above, the concept comes from WP:HISTRS, but, in my view, the advice of checking for reviews in academic journals deserves greater prominence. It is certainly a practice I try and follow. ThoughtIdRetired _TIR 07:35, 2 July 2024 (UTC)

I don't think that most editors explicitly research the sources they're using. At least twice in the last year, I have accidentally cited a Wikipedia mirror (some company makes a business of printing books of Wikipedia articles and selling them), which I'd never have done if I researched the book. All I was looking at was the source's contents, not its provenance. I had a reasonable belief that the Wikipedia article was already correct.

I think that review work primarily happens when a source has been contested on a talk page or at RSN. That's when it's useful to know how to determine a source's (or in my case, a publisher's) reputation. WhatamIdoing (talk) 20:08, 2 July 2024 (UTC)

Wow, I didn't realise that I was that unusual. I often check for academic reviews of books before using them as a source. That may be because I live a long way from a decent library, so it is often cheaper to buy a book rather than travel to a library – but I wish to avoid spending money on something that is not worth having. (Strange that the same argument would apply to the cost of travelling to a library to use the same source, but it does not have the same emotional impact.) I find some reviews enormously useful as they can add to the content of the reviewed book. I also keep an eye on book reviews in academic journals to see if any cover subjects of potential interest – often finding things that you would otherwise never discover. Incidentally, I sometimes check the sources in sources, which brought to light an academic paper that took information and the sources from a Wikipedia article (uncredited) where the sources should have been flagged as "failed verification". (The basic fact was wrong.) Another example of circularity, like the Wikipedia mirror case. ThoughtIdRetired _TIR 22:50, 2 July 2024 (UTC)

I think it depends on what kind of editing you're doing. If you're deeply invested in an article, it's worth finding the best sources you can, and it would be disappointing indeed to put a lot of time, effort, or money in a source that you ultimately had to discard. But if you're just trying to get rid of a {{fact}} for an uncontroversial claim, then (almost) any old source will do.

BTW, I hope you have been mining Wikipedia:The Wikipedia Library. I put some time into looking around in it a couple of months ago, and there are literally so many (tens of thousands!) of university press and similar books available for free that I was having trouble figuring out which ones to use. Look under Brill, Perlego, Wiley for starters. Several other publishers are in their systems, so it's not just the ones with their own imprint. WhatamIdoing (talk) 03:51, 3 July 2024 (UTC)

Yes books are often reviewed, yes reviews may help an editor, but an editor is likely to know that already. So I maintain that the sentence is useless, but alas I can't think of a rule against uselessness. Peter Gulutzan (talk) 12:51, 4 July 2024 (UTC)

That rule is at Wikipedia:Avoid instruction creep. WhatamIdoing (talk) 16:35, 4 July 2024 (UTC)

Thanks but unfortunately that's an essay. I have, though, finally seen that WP:PAG says "Avoid needless words." and "Expect editors to use common sense. If the spirit of the rule is clear, say no more." On that basis I have reverted the re-insertion. Peter Gulutzan (talk) 14:01, 16 July 2024 (UTC)

Clarification on reliability and sponsored / promotional content

WP:SPONSORED notes that such sources are "generally unacceptable", but could they still be used for statements of basic fact? For instance could a promotional piece about an individual be used to verify their date and place of birth, and other such non-promotional details? The same for sources that use superlatives, for instance could "They were the greatest artist of all time" be used to verify that they were an artist? I've instinctively thought of the answer to these questions as being 'yes', but I unsure that necessarily matches policy/guidance. -- LCU ActivelyDisinterested «@» °∆t° 10:58, 20 July 2024 (UTC)

In the first instance I'd say maybe, depending on the specific case. WP:ABOUTSELF can be argued to apply, but would it be good enough for WP:DOB? IMO, not necessarily. In the second instance I'd say no. If a sponsored source is the best source calling someone an artist, that's not good enough reason WP should. Gråbergs Gråa Sång (talk) 11:23, 20 July 2024 (UTC)

In the second instance what if the source wasn't sponsored but did contain very promotional language? -- LCU ActivelyDisinterested «@» °∆t° 11:30, 20 July 2024 (UTC)

Well for example, while looking for sources for a draft yesterday, a review in The Stage said of my subject "some of the fastest comic juggling you are ever likely to see." I'd absolutely take that as a RS that he's a juggler, if I needed it. Gråbergs Gråa Sång (talk) 11:38, 20 July 2024 (UTC)

Yes that the sort of source and issue I meant. -- LCU ActivelyDisinterested «@» °∆t° 11:51, 20 July 2024 (UTC)

I'm not sure that our actual rules are clear in that section. For example, the first sentence says: "Sponsored content is generally unacceptable as a source, because it is paid for by advertisers and bypasses the publication's editorial process."

The problem identified (i.e., bypasses editorial processes) is real, but our rule ought to be closer to "Treat it like an advertisement...because that's what it is". It'd be silly to say that you could support a claim to the subject's self-published and non-independent Twitter account, or to a full-page ad placed by the subject in a magazine, but if the subject pays for an ad in the form of sponsored content, then that's suddenly beyond the pale. WhatamIdoing (talk) 23:40, 27 July 2024 (UTC)

You've expressed the issue far better than I could. -- LCU ActivelyDisinterested «@» °∆t° 00:07, 28 July 2024 (UTC)

Let's change the first sentence in that section, then.

• Current: Sponsored content is generally unacceptable as a source, because it is paid for by advertisers and bypasses the publication's editorial process.
• Idea #1: Sponsored content is a type of paid advertisement and should be treated like any other paid advertisement.
• Idea #2: Sponsored content, like other paid advertisements, are paid for by advertisers and are not subject to the publication's editorial process. Advertisements can be cited, but they are non-independent and should be treated as self-published and primary sources in articles.
• Idea #3: Sponsored content is a paid advertisement that is formatted to look like an article, but whose content and decision to publish is controlled by the sponsor, rather than the publication's editors.

Do you have some ideas? WhatamIdoing (talk) 01:10, 28 July 2024 (UTC)

I like #3 it direct, defines what sponsored content is, and why that's an issue. But #2 is more explicit, which will likely help stop editors quibbling over it at a later date.

The main thing I like about #3 is "is a paid advertisement that is formatted to look like an article", it closes off silly questions of why sponsored content is an advert (e.g. "How is it an advert when it's a review of the product?").

The second half of #2 ("Advertisements can be...") is also good as it clarifies how they can be used and directs readers to the relevant policies.

So a combination of #3, with the second half of #2 maybe. -- LCU ActivelyDisinterested «@» °∆t° 12:09, 28 July 2024 (UTC)

The thing that worries me about "whose content and decision to publish is controlled by the sponsor, rather than the publication's editors" is that someone's going to say "But the influencer promises that the content is her own honest opinion, so that's not controlled by the sponsor!"

• Idea #2+#3: "Sponsored content is a paid advertisement that is formatted to look like an article or other piece of typical content for that outlet. Advertisements can be cited, but they are non-independent and should be treated as self-published and primary sources in articles."
• Idea #2+#4: Sponsored content is a paid advertisement that is formatted to look like an article or other piece of typical content for that outlet. The content may be directly controlled by the sponsor, or the advertiser may pay an author to create the content (e.g., influencer marketing). Advertisements can be cited, but they are non-independent and should be treated as self-published and primary sources in articles."

WhatamIdoing (talk) 19:07, 28 July 2024 (UTC)

Yes #2+#4 is an improvement, and you're right about the influencer issue. -- LCU ActivelyDisinterested «@» °∆t° 22:26, 28 July 2024 (UTC)

I think that would be an improvement. Let's wait until tomorrow, just in case anyone has any objections? WhatamIdoing (talk) 23:50, 28 July 2024 (UTC)

Fine with me. -- LCU ActivelyDisinterested «@» °∆t° 13:27, 29 July 2024 (UTC)

Done. WhatamIdoing (talk) 19:20, 30 July 2024 (UTC)

Gerontology Research Group

I do not think that the grg should be considered Reliable they have been "validating" hundreds of supercentenarians that have been convenientlly years before serveal other organizations existed so they can claim to have validated them beforehand also some of the "vaildations" have little to 0 actual documentation Wwew345t (talk) 23:06, 26 July 2024 (UTC)

Furthermore almost every important person in this group has been banned on wikipedia and almost all have tried using sockpuppet afterwards is this really reliable when they arent doing through work and have a history of breaking rules Wwew345t (talk) 22:11, 27 July 2024 (UTC)

And do you have any content related arguments about why they are unreliable? The Banner talk 23:26, 27 July 2024 (UTC)

They have been adding hundreds of "retroactive" validations that conveniently predates the founding of longeviquest who they are feuding with these validations have also included several cases that have no documentation/proof such as the two 114 year old Brazilian woman they validated recently they also credit people who weren't even in the group at the time of the supposed "Vaildation" again in reference to several latin american cases Wwew345t (talk) 23:32, 27 July 2024 (UTC)

@Wwew345t, did you see the big box at the very top of the page that says:

This is the wrong page for this discussion. That's why I closed this discussion earlier. You are allowed to have a discussion about whether Gerontology Research Group is a useful source at Wikipedia talk:WikiProject Longevity#Grg if you would like to. You are allowed to have a discussion about Gerontology Research Group at Wikipedia:Reliable sources/Noticeboard if you would like to. You are even allowed to have a discussion about Gerontology Research Group on the talk page of any article that is citing them.

What is not allowed is:

• more than one discussion on the same subject at the same time (WP:MULTI), or
• a discussion about whether a source is reliable on this page.

If you have questions about where you should take your question, please ask for help. WhatamIdoing (talk) 00:59, 28 July 2024 (UTC)

So where do you want me to put this? Project longevity hasnt been exactly active recently so I though I had to put it here Wwew345t (talk) 01:01, 28 July 2024 (UTC)

I don't actually care, so long as it is not here.

However, if you are interested in the biggest 'audience', then Wikipedia:Reliable sources/Noticeboard is the highest traffic option. WhatamIdoing (talk) 01:22, 28 July 2024 (UTC)

I will move it there Wwew345t (talk) 01:29, 28 July 2024 (UTC)

Where is the list of consensus of which websites are reliable?

Where is the list of consensus of which websites are reliable? Personally I find it to be extremely hard to find. Please make it easier to find. NamelessLameless (talk) 06:01, 5 August 2024 (UTC)

Are you perhaps asking about WP:Reliable sources/Perennial sources (shortcut WP:RSP) - which lists those sources we have discussed multiple times? Blueboar (talk) 10:27, 5 August 2024 (UTC)

Yeah, that's what I was trying to find. NamelessLameless (talk) 22:50, 9 August 2024 (UTC)

Note that we don't have (and can't have) either an exhaustive list of 'reliable' sources, or of 'unreliable' ones. Instead, we have policy describing the types of sources that are likely to be considered reliable, and mechanisms for discussing whether a particular source should be considered reliable for particular content. WP:RSNP consists of a list of repeatedly discussed sources only. Generally speaking, these tend to be edge cases of one sort or another. AndyTheGrump (talk) 10:54, 5 August 2024 (UTC)

The list itself is at Sources Mcljlm (talk) 14:53, 5 August 2024 (UTC)

That is only a list of sources that have been discussed regularly at RSN, it isn't close to being a full list of consensus of which sources are reliable. As well as discussions on RSN that don't appear on the list many project maintain lists related to their areas. -- LCU ActivelyDisinterested «@» °∆t° 16:44, 5 August 2024 (UTC)

How can the various lists be found? Mcljlm (talk) 06:36, 6 August 2024 (UTC)

Prior discussions on RSN can be found by searching the archives, there's a search block in the RSN header. I don't know of any easy way of finding all the project lists. NPP maintain a quite big list, Wikipedia:New page patrol source guide, but it still won't be a complete list and they have their own reasons for maintaining it. Ultimately the reason there isn't a single list is that editors should be looking to the relevant policy and guideline, and using their own good judgement. The consensus lists are meant to help editors when disagreement exists about verification of article content, so the same discussions don't have to happen repeatedly. -- LCU ActivelyDisinterested «@» °∆t° 10:10, 6 August 2024 (UTC)

@NamelessLameless and @Mcljlm, I am curious why you expect a list to exist. Did another editor perhaps claim that a source you wanted to use wasn't on an approved list?

There are somewhere around 1,500,000,000 websites. If an editor spent just one minute looking at assessing each of them, it would take 3,000 years of round-the-clock work – 24 hours a day, 365 days a year, for 40 lifetimes – to make such a list. Also, because websites spring up and then get removed, the list would be seriously out of date even after a few years. It is impossible. There is no list, and there never will be any such list. WhatamIdoing (talk) 21:23, 7 August 2024 (UTC)

WP:RS : One more essay, or updating present ones?

May be you know, here @ WP I keep taking some constructive initiatives to fill information and knowledge gap areas. Also have started doing little bit of content level of mentoring. Since last couple of months I am contemplating to take initiative to get couple of essays written from other experienced users.

One essay, I would like to take initiative, which would give glimpse of meticulous selection and application of academic scholarly sources that would have better chance to stand at GA, FA, CTOP and during any intense level of content negotiation. Some essay similar to WP:TIERS, but with more practical examples and guidance may be like WP:RSVETTING.

Idk from where to begin whom all to request. I know as of now already there are good number of essays exist and still I do think there is scope for reviewing present essays finding and discussing gap areas and promoting one more essay as said above.

• Currently available sources: WP:RS#Essays, WP:ESSAYDIR#Verifiability and sources, WP:APPLYRS

Requesting inputs. Bookku (talk) 10:16, 8 August 2024 (UTC)

Do reliable sources have to be informed of their use in a Wikipedia article?

Do reliable sources have to be informed of their use in a Wikipedia article? Howie Marx (talk) 07:01, 21 August 2024 (UTC)

Why would/should they be? Headbomb {t · c · p · b} 07:04, 21 August 2024 (UTC)

I didn`t know if we needed their permission or not to include them as a reliable source in an article ....I can`t think of any reason why they would object to this ... surely it`s good for them too? (as long as it`s factual)

Do you think they should be informed Headbomb? Howie Marx (talk) 07:27, 21 August 2024 (UTC)

No they don't need to be informed, and their objections wouldn't matter if they had any. -- LCU ActivelyDisinterested «@» °∆t° 10:03, 21 August 2024 (UTC)

Reliability and Time

How much does time factor into the reliability of sources and the accuracy of information? For example, say an article has multiple sources - enough to pass WP:N and WP:V. However, although the sources are regarded as reliable - as in Generally or Marginally - the info from them may be outdated. Perhaps the article was of an older topic that was notable but wasn't created until after a long time has passed. Yet, the only evidence that proves such info is out of date comes from primary and/or potentially unreliable sources. Since the site requires that articles are based on secondary sources, what would an editor do in this situation? Is it better to leave the article intact until a new reliable secondary source is found? Or should the article be updated with current information, even if that info is from dubious sourcing (or even none at all)? I was under the impression that WP:VNT and WP:NTEMP applies, but is it concerning to leave it even if no other sources ever emerge? Thanks, PantheonRadiance (talk) 22:25, 22 August 2024 (UTC)

To me, the distinction between primary and secondary sources is prescient: a source becomes primary when the context in which it was written has been sufficiently diverged from by "our own", that it no longer suffices to transparently verify claims and communicate information to the reader. That is, if the reader attempted to make deductions while intuitively applying the understanding of the world presently around them onto the text, they would get crucial facts wrong. A primary source requires an additional layer of interpretation and expertise between it and transparent, verifiable claims we can cite. Remsense ‥ 论 22:43, 22 August 2024 (UTC)

To directly answer your question: I guess we'll have to find out when it happens? It seems like it would depend entirely on what the article is specifically about. Remsense ‥ 论 22:43, 22 August 2024 (UTC)

Thanks for your reply! I think I get your point, but just to clarify: unless the info is inherently obvious, most of the time the info from primary sources needs analysis and expertise before we can use it on Wikipedia? And such analysis can generally only be done through a reliable secondary source? That's my line of thinking too. I believe that when it comes to the ease of spreading misinformation online, secondary reliable sources should definitely be used to combat that. PantheonRadiance (talk) 23:15, 22 August 2024 (UTC)

Also for a specific example, my impetus to discuss this was based on Smosh Games. The article became a redirect ten years ago due to a lack of notability. Recently I uncovered multiple sources found since then that proved notability per WP:WEB. However, there was recent contention regarding the info in the article, namely whether much of it was inaccurate because it was outdated - due to the ten years since the AfD. While I believed in sustained notability, another editor claimed its inaccuracy, and continuously added info attempting to update it, without verifying that the info came from secondary sources. I objected to that due to failing WP:V and WP:OR among other MOS guidelines. Needless to say it's a messy debate. PantheonRadiance (talk) 23:20, 22 August 2024 (UTC)