Loading ...
Sorry, an error occurred while loading the content.

"Foveated" gigapans

Expand Messages
  • panovrx
    News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where
    Message 1 of 14 , Nov 23, 2011
    • 0 Attachment
      News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
      http://live.ece.utexas.edu/research/realtime_foveation/index.php
      http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")

      This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years

      PeterM
    • Christian Bloch
      Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip:
      Message 2 of 14 , Nov 24, 2011
      • 0 Attachment
        Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip: http://vimeo.com/9013786

        Happy Thanksgiving,
        Blochi


        On Nov 23, 2011, at 2:53 PM, panovrx wrote:

         

        News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
        http://live.ece.utexas.edu/research/realtime_foveation/index.php
        http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")

        This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years

        PeterM


      • panovrx
        This might make you think a foveated sharpness display would work the same -- ie as you looked around it just got sharp locally where you looked -- which is
        Message 3 of 14 , Nov 24, 2011
        • 0 Attachment
          This might make you think a foveated sharpness display would work the same -- ie as you looked around it just got sharp locally where you looked -- which is what is does to a third party observer watching the subject. But the idea is that to the subject, the person whose gaze is being tracked, the whole screen is blurred (because, for example, in a gigapan situation, that only the gaze point data has been transmitted in detail), but they see the whole screen as sharp. Similar how we see the real world in real life (the whole world looks sharp even though our brain is only getting sharp detail from a the central area of fixation where their fovea is being stimulated.) I read somewhere the human eye is optically worse than the cheapest disposable camera but the world looks sharp all over. This is what the tech emulates. (
          Trivia question of the day, which kind of creature has more than one fovea in each eye?)

          PeterM

          --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@...> wrote:
          >
          > Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip: http://vimeo.com/9013786
          >
          > Happy Thanksgiving,
          > Blochi
          >
          >
          > On Nov 23, 2011, at 2:53 PM, panovrx wrote:
          >
          > > News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
          > > http://live.ece.utexas.edu/research/realtime_foveation/index.php
          > > http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")
          > >
          > > This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years
          > >
          > > PeterM
          > >
          > >
          >
        • Christian Bloch
          I see what you re getting at. Somehow transmitting the giga-tiles currently under observation. But I would hold up concerns that A) the eye darts around quite
          Message 4 of 14 , Nov 24, 2011
          • 0 Attachment
            I see what you're getting at. Somehow transmitting the giga-tiles currently under observation. But I would hold up concerns that A) the eye darts around quite a bit, and B) just the time between recognizing a looked-at area and transmission of that tile being completed will be enough to kill the illusion the entire pano would be already be loaded. It would need to be instantaneous, otherwise you'll have an irritating look, pop to highres, look, pop, look, pop.

            But I can see a gaze thingie quite useful for highlighting hotspots. Some sexy glint, or maybe even contextual info appearing… 

            Blochi

            On Nov 24, 2011, at 1:04 PM, panovrx wrote:

             

            This might make you think a foveated sharpness display would work the same -- ie as you looked around it just got sharp locally where you looked -- which is what is does to a third party observer watching the subject. But the idea is that to the subject, the person whose gaze is being tracked, the whole screen is blurred (because, for example, in a gigapan situation, that only the gaze point data has been transmitted in detail), but they see the whole screen as sharp. Similar how we see the real world in real life (the whole world looks sharp even though our brain is only getting sharp detail from a the central area of fixation where their fovea is being stimulated.) I read somewhere the human eye is optically worse than the cheapest disposable camera but the world looks sharp all over. This is what the tech emulates. (
            Trivia question of the day, which kind of creature has more than one fovea in each eye?)

            PeterM

            --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@...> wrote:
            >
            > Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip: http://vimeo.com/9013786
            >
            > Happy Thanksgiving,
            > Blochi
            >
            >
            > On Nov 23, 2011, at 2:53 PM, panovrx wrote:
            >
            > > News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
            > > http://live.ece.utexas.edu/research/realtime_foveation/index.php
            > > http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")
            > >
            > > This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years
            > >
            > > PeterM
            > >
            > >
            >


          • panovrx
            ... not instantaneous -- a finite (very short presumably) time ^ Loschky, L.C. & Wolverton, G.S. (2007). How Late Can You Update Gaze-contingent
            Message 5 of 14 , Nov 24, 2011
            • 0 Attachment
              --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@...> wrote:
              >
              > I see what you're getting at. Somehow transmitting the giga-tiles currently under observation. But I would hold up concerns that A) the eye darts around quite a bit, and B) just the time between recognizing a looked-at area and transmission of that tile being completed will be enough to kill the illusion the entire pano would be already be loaded. It would need to be instantaneous, otherwise you'll have an irritating look, pop to highres, look, pop, look, pop.
              >
              not instantaneous -- a finite (very short presumably) time
              ^ Loschky, L.C. & Wolverton, G.S. (2007). How Late Can You Update Gaze-contingent Multi-resolutional Displays Without Detection? ACM Transactions on Multimedia Computing, Communications and Applications, 3(4):25, 1-10.
            • Roger D Williams
              ... That, too, of course. But it won t be too many years before communication speeds and computer response times are fast enough to make the original
              Message 6 of 14 , Nov 24, 2011
              • 0 Attachment
                On Nov 25, 2011, at 9:02 AM, Christian Bloch <Blochi@...> wrote:

                I see what you're getting at. Somehow transmitting the giga-tiles currently under observation. But I would hold up concerns that A) the eye darts around quite a bit, and B) just the time between recognizing a looked-at area and transmission of that tile being completed will be enough to kill the illusion the entire pano would be already be loaded. It would need to be instantaneous, otherwise you'll have an irritating look, pop to highres, look, pop, look, pop.

                But I can see a gaze thingie quite useful for highlighting hotspots. Some sexy glint, or maybe even contextual info appearing…

                That, too, of course. But it won't be too many years before communication speeds and computer response times are fast enough to make the original concept--megapixel pictures that are sharp wherever you look, but not elsewhere--viable. I certainly hope to live to see it! (I just hope that's not tempting fate: I am 74 already).

                Roger W._,_._,___
              • ddraig
                2011/11/25 Roger D Williams ... There s massive latency issues involved in transmitting data over long distances. The speed of light is
                Message 7 of 14 , Nov 24, 2011
                • 0 Attachment
                  2011/11/25 Roger D Williams <roger@...>

                   

                  That, too, of course. But it won't be too many years before communication speeds and computer response times are fast enough to make the original concept--megapixel pictures that are sharp wherever you look, but not elsewhere--viable. I certainly hope to live to see it! (I just hope that's not tempting fate: I am 74 already).

                  There's massive latency issues involved in transmitting data over long distances.  The speed of light is fast, but when transmitting across the surface of the globe from the other side of it, the delay is noticeable. It's a major problem in computer gaming, and there's no way around it.  So while this technique might work well locally, I doubt it would be viable for intercontinental or trans-global communications.

                  Dwayne
                  --
                    ddraig@... irc.bluesphereweb.com #dna
                           ...r.e.t.u.r.n....t.o....t.h.e....s.o.u.r.c.e...
                          http://tinyurl.com/he-is-right-you-know-jpg
                  our aim is wakefulness,  our enemy is dreamless sleep
                • Roger D. Williams
                  ... Yes, it s quite noticeable that on-the-scene reporters often appear to take rather a long time responding to questions from the studio. This is usually
                  Message 8 of 14 , Nov 25, 2011
                  • 0 Attachment
                    On Fri, 25 Nov 2011 11:41:03 +0900, ddraig <dwayne@...> wrote:

                    > 2011/11/25 Roger D Williams <roger@...>
                    >
                    >> That, too, of course. But it won't be too many years before
                    >> communication
                    >> speeds and computer response times are fast enough to make the original
                    >> concept--megapixel pictures that are sharp wherever you look, but not
                    >> elsewhere--viable. I certainly hope to live to see it! (I just hope
                    >> that's
                    >> not tempting fate: I am 74 already).
                    >>
                    >>
                    >
                    > There's massive latency issues involved in transmitting data over long
                    > distances. The speed of light is fast, but when transmitting across the
                    > surface of the globe from the other side of it, the delay is noticeable.
                    > It's a major problem in computer gaming, and there's no way around it.
                    > So
                    > while this technique might work well locally, I doubt it would be viable
                    > for intercontinental or trans-global communications.

                    Yes, it's quite noticeable that on-the-scene reporters often appear to
                    take rather a long time responding to questions from the studio. This is
                    usually because the signals are being bounced off a satellite, and this
                    takes a sizable fraction of a second.

                    Roger W.

                    --
                    Using Opera's revolutionary email client: http://www.opera.com/mail/
                  • enridp
                    That s not true, the human eye has much less cones (photoreceptor cell) than a common digital camera. So, every camera has more megapixels than the eye, but
                    Message 9 of 14 , Nov 25, 2011
                    • 0 Attachment
                      That's not true, the human eye has much less cones (photoreceptor cell) than a common digital camera.
                      So, every camera has more "megapixels" than the eye, but they don't have more resolution, because our cones are concentrated in a very very small area (fovea), so we can see a very poor range with precision (4 to 12 degree), but our eye is moving continuously (even when you don't note it) because our eyes are scanning the entire FoV.
                      If you think about it, our brain is stitching panoramas all the time (stereo, with HDR, and in real time).


                      --- In PanoToolsNG@yahoogroups.com, "panovrx" <panovrx@...> wrote:
                      >
                      > This might make you think a foveated sharpness display would work the same -- ie as you looked around it just got sharp locally where you looked -- which is what is does to a third party observer watching the subject. But the idea is that to the subject, the person whose gaze is being tracked, the whole screen is blurred (because, for example, in a gigapan situation, that only the gaze point data has been transmitted in detail), but they see the whole screen as sharp. Similar how we see the real world in real life (the whole world looks sharp even though our brain is only getting sharp detail from a the central area of fixation where their fovea is being stimulated.) I read somewhere the human eye is optically worse than the cheapest disposable camera but the world looks sharp all over. This is what the tech emulates. (
                      > Trivia question of the day, which kind of creature has more than one fovea in each eye?)
                      >
                      > PeterM
                      >
                      > --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@> wrote:
                      > >
                      > > Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip: http://vimeo.com/9013786
                      > >
                      > > Happy Thanksgiving,
                      > > Blochi
                      > >
                      > >
                      > > On Nov 23, 2011, at 2:53 PM, panovrx wrote:
                      > >
                      > > > News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
                      > > > http://live.ece.utexas.edu/research/realtime_foveation/index.php
                      > > > http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")
                      > > >
                      > > > This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years
                      > > >
                      > > > PeterM
                      > > >
                      > > >
                      > >
                      >
                    • enridp
                      I think I m not catching the idea. Even with instantaneous speed I can t see how this could make a good ilusion (not without proper lenses covering all our
                      Message 10 of 14 , Nov 25, 2011
                      • 0 Attachment
                        I think I'm not catching the idea.
                        Even with instantaneous speed I can't see how this could make a good ilusion (not without proper lenses covering all our FoV).
                        If you blur some area in my screen I will see it with blur.

                        --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@...> wrote:
                        >
                        > I see what you're getting at. Somehow transmitting the giga-tiles currently under observation. But I would hold up concerns that A) the eye darts around quite a bit, and B) just the time between recognizing a looked-at area and transmission of that tile being completed will be enough to kill the illusion the entire pano would be already be loaded. It would need to be instantaneous, otherwise you'll have an irritating look, pop to highres, look, pop, look, pop.
                        >
                        > But I can see a gaze thingie quite useful for highlighting hotspots. Some sexy glint, or maybe even contextual info appearing…
                        >
                        > Blochi
                        >
                        > On Nov 24, 2011, at 1:04 PM, panovrx wrote:
                        >
                        > > This might make you think a foveated sharpness display would work the same -- ie as you looked around it just got sharp locally where you looked -- which is what is does to a third party observer watching the subject. But the idea is that to the subject, the person whose gaze is being tracked, the whole screen is blurred (because, for example, in a gigapan situation, that only the gaze point data has been transmitted in detail), but they see the whole screen as sharp. Similar how we see the real world in real life (the whole world looks sharp even though our brain is only getting sharp detail from a the central area of fixation where their fovea is being stimulated.) I read somewhere the human eye is optically worse than the cheapest disposable camera but the world looks sharp all over. This is what the tech emulates. (
                        > > Trivia question of the day, which kind of creature has more than one fovea in each eye?)
                        > >
                        > > PeterM
                        > >
                        > > --- In PanoToolsNG@yahoogroups.com, Christian Bloch <Blochi@> wrote:
                        > > >
                        > > > Yeah, saw a similar thing at Siggraph Asia 2009, but it was changing exposure in an HDR image depending on where you look. Shot this clip: http://vimeo.com/9013786
                        > > >
                        > > > Happy Thanksgiving,
                        > > > Blochi
                        > > >
                        > > >
                        > > > On Nov 23, 2011, at 2:53 PM, panovrx wrote:
                        > > >
                        > > > > News to me was that it is possible to have an gaze-tracking display system that in real time shows a high resolution video scene that is blurred except where the person is looking yet is indistinguishable in user experience from a display that is sharp all over.
                        > > > > http://live.ece.utexas.edu/research/realtime_foveation/index.php
                        > > > > http://en.wikipedia.org/wiki/Foveated_imaging (whence a new phrase to reckon with -- "gaze contingent imaging")
                        > > > >
                        > > > > This has implications for super high res gigapan display one would think -- if every pc has fast gaze tracking in a few years
                        > > > >
                        > > > > PeterM
                        > > > >
                        > > > >
                        > > >
                        > >
                        > >
                        >
                      • ddraig
                        ... The theory is that you will not notice the blur, as the bit you are actually looking at will be focussed, the rest blurred. Dwayne -- ddraig@pobox.com
                        Message 11 of 14 , Nov 25, 2011
                        • 0 Attachment
                          On 25 November 2011 23:56, enridp <enridp@...> wrote:
                           

                          I think I'm not catching the idea.
                          Even with instantaneous speed I can't see how this could make a good ilusion (not without proper lenses covering all our FoV).
                          If you blur some area in my screen I will see it with blur.



                          The theory is that you will not notice the blur, as the bit you are actually looking at will be focussed, the rest blurred.

                          Dwayne
                          --
                            ddraig@... irc.bluesphereweb.com #dna
                                   ...r.e.t.u.r.n....t.o....t.h.e....s.o.u.r.c.e...
                                  http://tinyurl.com/he-is-right-you-know-jpg
                          our aim is wakefulness,  our enemy is dreamless sleep
                        • Keith Davison
                          ... You won t, if you are human. Our area of sharp vision is the size of a UK 50p coin ( 28mm dia. ) held at arms length, everything outside of this is
                          Message 12 of 14 , Nov 25, 2011
                          • 0 Attachment
                            > I think I'm not catching the idea.
                            > Even with instantaneous speed I can't see how this could make a good ilusion (not without proper
                            > lenses covering all our FoV).
                            > If you blur some area in my screen I will see it with blur.

                            You won't, if you are human.

                            Our area of sharp vision is the size of a UK 50p coin ( 28mm dia. ) held at arms length,
                            everything outside of this is relatively blurred. Your eyes "scan" this sharp area across a
                            scene and your wetware builds up a compressed composite of the scene in memory. This
                            model favours edges and colour/contrast changes, which is why camouflage is effective at
                            confusing it.

                            Your brain can also intentionally selectively process visual input to concentrate on a
                            particular object class, which is how inattentive blindness occurs.

                            The blurred area outside of the sharp foveal area is, however, very sensitive to change and
                            movement, especially on the periphery. Very handy if you're a primate with large predators
                            creeping up on you in the savannah.

                            I don't think our brains have the processing power to deal with a foveal level sharpness
                            over our whole visual field.

                            KeithD
                          • ddraig
                            ... I think they probably do, but obviously it d be wasted processing power which could be used for things more suited to keeping us alive, otherwise that s
                            Message 13 of 14 , Nov 25, 2011
                            • 0 Attachment
                              On 26 November 2011 01:47, Keith Davison <megakd@...> wrote:
                               

                              I don't think our brains have the processing power to deal with a foveal level sharpness
                              over our whole visual field.

                              I think they probably do, but obviously it'd be wasted processing power which could be used for things more suited to keeping us alive, otherwise that's how our visual system would work.   I actually suspect most of our brain's processing power is kept as a reserve for emergency use.

                              Dwayne
                              --
                                ddraig@... irc.bluesphereweb.com #dna
                                       ...r.e.t.u.r.n....t.o....t.h.e....s.o.u.r.c.e...
                                      http://tinyurl.com/he-is-right-you-know-jpg
                              our aim is wakefulness,  our enemy is dreamless sleep
                            • enridp
                              Yes, but that works in real life (with full FoV), but does it work in a screen? When I see an image like this:
                              Message 14 of 14 , Nov 26, 2011
                              • 0 Attachment
                                Yes, but that works in real life (with full FoV), but does it work in a screen?
                                When I see an image like this:
                                http://en.wikipedia.org/wiki/File:Texas_state_cemetery_foveated1.png
                                even when I'm focusing my eyes in the sharp area I can see the rest of the image with blur.

                                --- In PanoToolsNG@yahoogroups.com, ddraig <dwayne@...> wrote:
                                >
                                > On 25 November 2011 23:56, enridp <enridp@...> wrote:
                                >
                                > > **
                                > >
                                > >
                                > > I think I'm not catching the idea.
                                > > Even with instantaneous speed I can't see how this could make a good
                                > > ilusion (not without proper lenses covering all our FoV).
                                > > If you blur some area in my screen I will see it with blur.
                                > >
                                >
                                >
                                > The theory is that you will not notice the blur, as the bit you are
                                > actually looking at will be focussed, the rest blurred.
                                >
                                > Dwayne
                                > --
                                > ddraig@... irc.bluesphereweb.com #dna
                                > ...r.e.t.u.r.n....t.o....t.h.e....s.o.u.r.c.e...
                                > http://tinyurl.com/he-is-right-you-know-jpg
                                > our aim is wakefulness, our enemy is dreamless sleep
                                >
                              Your message has been successfully submitted and would be delivered to recipients shortly.