Say a fashion expert, a instructor at the Fashion Institute goes to a cafe and has a coffee. It turns out another instructor students displaying a fall project. The clothes are pretty ugly, poorly made. You weren’t part of the instructional team and the professor isn’t there....
I read the first page of their notes, which were good. I missed the other pages the first time around.
You have good points. The dog piling was not good.
"Hi, I'm a 10 year faculty over at XYZ (I have instructor on my scubaboard badge), you had some interesting elements in your show, you might consider alternate elements ABC, many people have been going in that direction. Are you a student of professor Z, they've been a hold out against this movement. It has been a bit of a debate in the community. Glad you're part of the fashion scene, I hope you're enjoying it."
Maybe. That seems good for a one off with the student, with you and the student at a cafe, no one else around.
What if it is at a convention, in the hall where people discuss, and others have questions, How short do you cut the 'Well professor Z has not been moving in the direction of most on this, let's just leave it there." Or do you explain, and debate with your fellow fashion experts/novices/interested from various institutions/public?
"I'm not sure why Z was doing it that way. I'd likely have done it this way. I think many might."
"We might ask Z, I do not see them around."
Much I think like was done in laying out a different, much longer, syllabus.
I think there is the difference in context. Dog piling no. But I think the message is still as clearly expressed.
I'm a junior trainee in academics, in my 6th year, completing my Ph.D this semester, I think we might try to have the same conversation. Just, yes, more carefully, a little indirectly.
"I'm not sure why A got those results. I work in that area. When I worked at replicating their experiments, I got abc, over n repetitions, which are <pause> significantly outside what they report. It's possible their results were not over a few repetitions, it wasn't clear in the paper."
(ETA: In as "These are the results I found when I looked, I'm modestly skilled. I don't know why they differ." Neutral, these were my results, description. Because I do not know why they differ. I have reasonable confidence in my results. I only know the details of their methods as reported in their paper, from which I was not able to reproduce their results. And, in this example, I have not followed up with them on why.)
This is a version of saying that A faked their results (a cardinal sin) or were sloppy in their methods or reporting.
(ETA: or I do not have much of a clue what I am doing. A possibility...)
ETA2: Side bar on science. Feel free to skip.
As context for the above “I got different results”. Not that anyone asked.
The goal is solid results backed up by others reproducing them. That produces trusted 'facts' about the world.
Often, if the methodology looks good/careful/though, the results are trusted for now, even without others reporting that they explicitly reproduced them. In my general field. Particularly as replication is not as publishable, if publishable much at all. A reason to do your experiment n times reporting the average and variation and not just once. Now, if things that try to build on those results fail, then that winds up explicitly invalidating the results, if they are at fault. And those that reported the initial results get egg on their face. Possibly a lot of egg, depending on how it all turns out.
So, a good attempt at explicitly replicating their exact experiment, that fails, (my 'I got different results' above) means we really need to reexamine the initial results, they do not look as good as we assumed. No matter what the reason. There may be a variable we did not realize. Their lab may have a different magnetic signature, that affects the results, that we need to know about. What ever. Their results, that A+B+C produces D, get put in the 'not sure' bin. As when we did A+B+C we got E. Part of their job in reporting the results was to tell others all the conditions to reproduce them. They reported. We reproduced the steps they reported (hopefully), but got a different answer, (we think).
That is either:
- interesting, as in cool, there is something new that could increase our understanding of the world, or
- bad in various ways, with their or our methods or reporting.