Read on Twitter
My new paper, "Causality and Continuity Close the Gaps in Event Representations", with Lewis Baker, Frank Keil, and @stricklandbrent has just been published in Memory and Cognition! Short summary in thread, link here: https://rdcu.be/b8bt1 ">https://rdcu.be/b8bt1&quo... (1/9)
This is a project that& #39;s easier to show than tell. Watch this video:
https://osf.io/xsajz/
Did">https://osf.io/xsajz/&qu... you see this frame? https://osf.io/ps2ev/ ">https://osf.io/ps2ev/&qu... (2/9)
https://osf.io/xsajz/
Did">https://osf.io/xsajz/&qu... you see this frame? https://osf.io/ps2ev/ ">https://osf.io/ps2ev/&qu... (2/9)
You probably thought you did, but in fact the video cut before then. Your mind completed the event representation by filling in the missing moment of release. My coauthors @stricklandbrent and Frank Keil first reported this "filling-in" effect in a 2011 paper (3/9)
The goal of this paper was to figure out *how* we fill in these missing causal links. Do we rely on a specific semantic schema like "shooting a basketball"? Do we match the object before and after the cut? Or do we only rely on cues to spatiotemporal continuity? (4/9)
Experiments 1 and 2 used completely artificial CG animations with unusual causal interactions to see if this happened without clear semantic schemas to rely on. Example: https://osf.io/j5kxs/ ">https://osf.io/j5kxs/&qu... (s/o to Lewis for making these, they were a ton of work!) (5/9)
With these unfamiliar events, Exp. 1 and 2 found that people filled in the moment of impact/release as long as the ball& #39;s trajectory was visible in the second half of the video, regardless of anything else that happened. No reliance on schemas like "shooting a basketball" (6/9)
Experiment 3 used realistic videos and manipulated two features of the second half of the video: Whether the object transformed (basketball becomes soccer ball), or whether it appeared "too far along" in its trajectory after the cut (7/9)
Changing the object identity had no detectable effect, but disrupting spatiotemporal continuity reduced filling-in.
Event representations are constructed using "low-level" cues like spatiotemporal continuity. Seems a bit like the "tunnel effect" https://en.wikipedia.org/wiki/Tunnel_effect">https://en.wikipedia.org/wiki/Tunn... (8/9)
Event representations are constructed using "low-level" cues like spatiotemporal continuity. Seems a bit like the "tunnel effect" https://en.wikipedia.org/wiki/Tunnel_effect">https://en.wikipedia.org/wiki/Tunn... (8/9)
Moral of the story: Creating coherent event representations
relies on perceptual object-tracking cues, particularly spatiotemporal continuity, rather than semantic knowledge. (9/9)
But there& #39;s a little wrinkle hidden here for future work (up for grabs if you want it...) (0/5)
relies on perceptual object-tracking cues, particularly spatiotemporal continuity, rather than semantic knowledge. (9/9)
But there& #39;s a little wrinkle hidden here for future work (up for grabs if you want it...) (0/5)
The cut from the first to second half of these videos involves a change in perspective. We have the impression of spatiotemporal continuity across that cut, as long as the ball appears in the "right place" after the cut.
That& #39;s actually pretty strange. (1/5)
That& #39;s actually pretty strange. (1/5)
In multiple-object-tracking tasks, perspective shifts like that absolutely destroy performance. Yet, in this case, people are effortlessly linking the two halves of this event together to the degree that they falsely remember seeing parts of the event that didn& #39;t happen! (2/5)
For a single object, we can readily track spatiotemporal continuity across a change in perspective, even when the features of the object change. As far as we can tell, nobody& #39;s tried to figure out how that works, even though TV and movies rely on it all the time. (3/5)
So open challenge to my vision science friends: Changing the background, object identity, and perspective doesn& #39;t disrupt single-object tracking. How? Could a computer vision system pull that off? And why does it break when the ball is "too far" along? (4/5)
