I don’t think I disagree with your general point, but this is fairly different from what the comment above was looking for—confidence values that we can put next to our outputs.
I mean, I don’t think such a value (it is definitely possible I’m reading it overly-specifically), like, a numerical value, can generally be assigned to the truthiness of a snippet of general prose.
I mean, in your “7 peer reviewed papers” example, part of the point of research (a big part!) is to eventually overturn previous consensus views. So, if we have 6 peer reviewed papers with lots of citations, and one that conclusively debunks the rest of them, there is not a 6/7 chance that any random sentiment pulled out of the pile of text is “true” in terms of physical reality.
I mean, I don’t think such a value (it is definitely possible I’m reading it overly-specifically), like, a numerical value, can generally be assigned to the truthiness of a snippet of general prose.
I mean, in your “7 peer reviewed papers” example, part of the point of research (a big part!) is to eventually overturn previous consensus views. So, if we have 6 peer reviewed papers with lots of citations, and one that conclusively debunks the rest of them, there is not a 6/7 chance that any random sentiment pulled out of the pile of text is “true” in terms of physical reality.