Loading ...
Sorry, an error occurred while loading the content.

Multi-Camera-One-Shot-solution: How do deal with parallax errors?

Expand Messages
  • Philipp B. Koch
    Hi everybody, I have a question regarding the physics or technique behind the stitching process in general. Normally, you would have a so-called nodal-adapter
    Message 1 of 16 , Apr 25, 2008
    • 0 Attachment
      Hi everybody,

      I have a question regarding the physics or technique behind the
      stitching process in general. Normally, you would have a so-called
      nodal-adapter to keep your lens at the no-parallax-point when shooting
      panoramas. But then -- and at this point maybe this topic could be
      somehow similar to the issues discussed in an earlier thread about
      off-centered shooting from high places like churches and so on -- there
      are products like the Ladybug2
      (http://www.ptgrey.com/products/ladybug2/index.asp) that have many
      lenses shooting at the same time in different directions, but also
      "off-centered". Plus there's a website about a similar DIY-project
      (http://www.vrhotwires.com/InexpensivePanoramicVideo.html), and again
      the lenses are far from being centered correctly. This page includes
      some proof-of-concept-videos showing that it works.

      So my question is this: Obviously it must me possible to deal with these
      parallax errors as long as you know the (never changing) exact lens
      distances and angles towards each other, the HFOV and whatever else
      might be important -- but how?! I would love to try something similar to
      what the second link shows, but of course I want to make sure that it is
      possible to stitch images captured this way beforehand. Can someone give
      me some (preferably practical) hints if / how this is possible?

      -- How can one make sure the images can be stitched in the end,
      independent of the captured scene?
      -- Differently asked: If I know the HFOV of the cameras, how many
      overlap would be needed to avoid parallax problems (if actually overlap
      is part of solution at all)?

      Any help would be much appreciated! It would be so fantastic to have
      such a thing like a one-shot-multi-cam-system :-)

      Regards, Philipp

      PS. In case it is important or of interest here: Beside my normal
      Fisheye-shooting, I would like to have a sort of one-shot-solution. I
      bought one, and the quality was so horrible that I returned it. So what
      I plan to do instead is to set up a system of several cameras that
      capture the full 360 degrees horizontally, and maybe one or two cams
      that point upwards so that I also get the zenith. I'm still unsure which
      specific camera model I want to try this with, so I don't know any
      technical data like HFOVs and so on yet. The only thing that's already
      clear is that I'll need to take IP-cams, since USB-connections are not
      fast enough and with firewire it seems to be a problem on Windows as
      well (at least a microsystem technician recently told me so) if I also
      want to capture video streams and not only still images.
    • Erik Krause
      ... You can t automatically fix parallax errors, no matter how much you know about the shooting details. If some background is hidden behind a foreground
      Message 2 of 16 , Apr 25, 2008
      • 0 Attachment
        On Friday, April 25, 2008 at 14:41, Philipp B. Koch wrote:

        > So my question is this: Obviously it must me possible to deal with these
        > parallax errors as long as you know the (never changing) exact lens distances
        > and angles towards each other, the HFOV and whatever else might be important
        > -- but how?!

        You can't automatically fix parallax errors, no matter how much you
        know about the shooting details. If some background is hidden behind
        a foreground detail there is no way to recover the background.

        However, the amount of parallax error is determined by relative
        subject distance, by NPP offset and by angle between shots. If you
        don't have near foreground together with far away background in the
        overlap, there probably won't be any problem. On
        http://wiki.panotools.org/Parallax#Calculation you find some formulas
        to calculate the expected error.

        best regards


        Erik Krause
        http://www.erik-krause.de
      • Philipp B. Koch
        Thank you for the information! ... Sounds convincing :-) ... So does the mean that for example the Ladbug2 cannot shoot scenes with close objects and far away
        Message 3 of 16 , Apr 25, 2008
        • 0 Attachment
          Thank you for the information!

          Erik Krause schrieb:
          > You can't automatically fix parallax errors, no matter how much you
          > know about the shooting details. If some background is hidden behind
          > a foreground detail there is no way to recover the background.
          >
          Sounds convincing :-)

          > However, the amount of parallax error is determined by relative
          > subject distance, by NPP offset and by angle between shots. If you
          > don't have near foreground together with far away background in the
          > overlap, there probably won't be any problem. On
          > http://wiki.panotools.org/Parallax#Calculation you find some formulas
          > to calculate the expected error.
          So does the mean that for example the Ladbug2 cannot shoot scenes with
          close objects and far away objects (so there are noticeably parallax
          errors)?

          If I'd try to build such a multi-cam-system: Am I right that minimizing
          the "inner space" between the cameras as much as possible would reduce
          parallax problems (since the cameras were closer to the NPP)?

          Regards, Philipp
        • Ian Wood
          ... Yes. To some extent, no shooting with nearby objects regardless of whether there s distant objects - get too close and the next-door cameras are
          Message 4 of 16 , Apr 25, 2008
          • 0 Attachment
            On 25 Apr 2008, at 14:15, Philipp B. Koch wrote:

            > So does the mean that for example the Ladbug2 cannot shoot scenes with
            > close objects and far away objects (so there are noticeably parallax
            > errors)?

            Yes. To some extent, no shooting with nearby objects regardless of
            whether there's distant objects - get too close and the next-door
            cameras are effectively seeing different sides of an object.

            > If I'd try to build such a multi-cam-system: Am I right that
            > minimizing
            > the "inner space" between the cameras as much as possible would reduce
            > parallax problems (since the cameras were closer to the NPP)?

            Yes, this is usually the biggest design problem with multi-camera
            systems. Have a look at the Agnos MrotatorRP for a good approach.

            http://tinyurl.com/5wjq3r

            Ian
          • Erik Krause
            ... Yes, of course. It might be worth trying to put them above each other, such that the lenses touch instead of putting the cameras back- to-back. You ll get
            Message 5 of 16 , Apr 25, 2008
            • 0 Attachment
              On Friday, April 25, 2008 at 15:15, Philipp B. Koch wrote:

              > If I'd try to build such a multi-cam-system: Am I right that minimizing
              > the "inner space" between the cameras as much as possible would reduce
              > parallax problems (since the cameras were closer to the NPP)?

              Yes, of course. It might be worth trying to put them above each
              other, such that the lenses touch instead of putting the cameras back-
              to-back. You'll get a NPP offset of about 6 cm instead of 20 cm or
              so. And dendent on your shooting subject a vertical parallax might be
              easier to tolerate than a horizontal one. This might be particularily
              true for architecture and indoor shots if no furniture top is only
              slightly below camera level...

              best regards
              Erik Krause
              http://www.erik-krause.de
            • Philipp B. Koch
              ... Thanks for the link. Hhm, if one has three (!!) fullframe DSLRs with corresponding fisheye lenses, this might be a good thing :-) Unfortunately, I don t
              Message 6 of 16 , Apr 25, 2008
              • 0 Attachment
                Ian Wood schrieb:
                >> If I'd try to build such a multi-cam-system: Am I right that
                >> minimizing
                >> the "inner space" between the cameras as much as possible would reduce
                >> parallax problems (since the cameras were closer to the NPP)?
                >>
                > Yes, this is usually the biggest design problem with multi-camera
                > systems. Have a look at the Agnos MrotatorRP for a good approach.
                >
                > http://tinyurl.com/5wjq3r
                >
                Thanks for the link. Hhm, if one has three (!!) fullframe DSLRs with
                corresponding fisheye lenses, this might be a good thing :-)
                Unfortunately, I don't even have one fullframe camera at least...
              • Philipp B. Koch
                ... This is an interesting idea. But still I m not sure how the vertical displacement could be dealt with. I mean, the lowermost camera s image and the one
                Message 7 of 16 , Apr 25, 2008
                • 0 Attachment
                  Erik Krause schrieb:
                  > On Friday, April 25, 2008 at 15:15, Philipp B. Koch wrote:
                  >
                  >
                  >> If I'd try to build such a multi-cam-system: Am I right that minimizing
                  >> the "inner space" between the cameras as much as possible would reduce
                  >> parallax problems (since the cameras were closer to the NPP)?
                  >>
                  >
                  > Yes, of course. It might be worth trying to put them above each
                  > other, such that the lenses touch instead of putting the cameras back-
                  > to-back. You'll get a NPP offset of about 6 cm instead of 20 cm or
                  > so. And dendent on your shooting subject a vertical parallax might be
                  > easier to tolerate than a horizontal one. This might be particularily
                  > true for architecture and indoor shots if no furniture top is only
                  > slightly below camera level...
                  This is an interesting idea. But still I'm not sure how the vertical
                  displacement could be dealt with.
                  I mean, the lowermost camera's image and the one from the uppermost cam
                  would have a very different (vertically shifted) view.
                  Could it be a solution to let the two cameras "on the ground" point
                  slighty upwards and the two cameras "on top" point slighty
                  downwards -- or, since it is a rectilinear picture, would this
                  perspective distorsion (vertical lines not parallel, but V- or A-shaped)
                  make the whole thing unstitchable?
                • Ian Wood
                  ... Fullframe fisheye images such as the Sigma 8mm on a 1.5 or 1.6 crop camera, not three fullframe cameras... ;-) Ian
                  Message 8 of 16 , Apr 25, 2008
                  • 0 Attachment
                    On 25 Apr 2008, at 19:07, Philipp B. Koch wrote:

                    >> http://tinyurl.com/5wjq3r
                    >>
                    > Thanks for the link. Hhm, if one has three (!!) fullframe DSLRs with
                    > corresponding fisheye lenses, this might be a good thing :-)
                    > Unfortunately, I don't even have one fullframe camera at least...

                    Fullframe fisheye images such as the Sigma 8mm on a 1.5 or 1.6 crop
                    camera, not three fullframe cameras... ;-)

                    Ian
                  • Erik Krause
                    ... Yes, about 12cm, still small compared to DSLR mounted back-to-back. ... Perspective correction is no problem for panotools, but there would be no benefit.
                    Message 9 of 16 , Apr 25, 2008
                    • 0 Attachment
                      On Friday, April 25, 2008 at 20:17, Philipp B. Koch wrote:

                      > I mean, the lowermost camera's image and the one from the uppermost cam
                      > would have a very different (vertically shifted) view.

                      Yes, about 12cm, still small compared to DSLR mounted back-to-back.

                      > Could it be a solution to let the two cameras "on the ground" point
                      > slighty upwards and the two cameras "on top" point slighty
                      > downwards -- or, since it is a rectilinear picture, would this
                      > perspective distorsion (vertical lines not parallel, but V- or A-shaped)
                      > make the whole thing unstitchable?

                      Perspective correction is no problem for panotools, but there would
                      be no benefit. Since the NPP is most probably inside the lens (at
                      least for wide angle and fisheye lenses[1]), there is no chance to
                      get nearer to each other than lens diameter.

                      [1] For some telephoto lenses the NPP can be even behind the camera.
                      For so called telecentric lenses it is at infinite distance behind
                      the camera, but you won't want to shoot a 360° panorama with those
                      ;-)

                      best regards
                      Erik Krause
                      http://www.erik-krause.de
                    • michel thoby
                      Unbelievable coincidence! I am currently experimenting with DuoPix and was shooting some street views this afternoon just to finally qualify the imaging
                      Message 10 of 16 , Apr 25, 2008
                      • 0 Attachment
                        Unbelievable coincidence!
                        I am currently experimenting with "DuoPix" and was shooting some
                        street views this afternoon just to finally qualify the imaging
                        system chain.
                        DuoPix is just a modernized revival of Dan Slater's Spherecam though.

                        Details are here:
                        http://michel.thoby.free.fr/DuoPix/DuoPix_solution.html

                        Comments welcome.

                        Best regards,

                        Michel

                        Le 25 avr. 08 à 15:39, Erik Krause a écrit :

                        > On Friday, April 25, 2008 at 15:15, Philipp B. Koch wrote:
                        >
                        > > If I'd try to build such a multi-cam-system: Am I right that
                        > minimizing
                        > > the "inner space" between the cameras as much as possible would
                        > reduce
                        > > parallax problems (since the cameras were closer to the NPP)?
                        >
                        > Yes, of course. It might be worth trying to put them above each
                        > other, such that the lenses touch instead of putting the cameras back-
                        > to-back. You'll get a NPP offset of about 6 cm instead of 20 cm or
                        > so. And dendent on your shooting subject a vertical parallax might be
                        > easier to tolerate than a horizontal one. This might be particularily
                        > true for architecture and indoor shots if no furniture top is only
                        > slightly below camera level...
                        >
                        > best regards
                        > Erik Krause
                        > http://www.erik-krause.de



                        [Non-text portions of this message have been removed]
                      • panovrx
                        ... cf: http://www.mediavr.com/twinrig1.jpg and I think you are right -- the additional vertical parallax is worth it for the reduction in horizontal -- I made
                        Message 11 of 16 , Apr 25, 2008
                        • 0 Attachment
                          --- In PanoToolsNG@yahoogroups.com, "Erik Krause" <erik.krause@...>
                          wrote:

                          > Yes, of course. It might be worth trying to put them above each
                          > other, such that the lenses touch instead of putting the cameras back-
                          > to-back. You'll get a NPP offset of about 6 cm instead of 20 cm or
                          > so. And dendent on your shooting subject a vertical parallax might be
                          > easier to tolerate than a horizontal one. This might be particularily
                          > true for architecture and indoor shots if no furniture top is only
                          > slightly below camera level...

                          cf:
                          http://www.mediavr.com/twinrig1.jpg

                          and I think you are right -- the additional vertical parallax is worth
                          it for the reduction in horizontal --
                          I made this rig (2 5D+trimmed 10.5mm) for 360 timelapse and you get 360
                          frames about 113 degrees high

                          Peter
                        • Philipp B. Koch
                          Hey Michel, very interesting indeed :-) Besides the fact of problems with synchronization, I find the mere possibility to get an amazing HFOV of 190 degrees
                          Message 12 of 16 , Apr 26, 2008
                          • 0 Attachment
                            Hey Michel,

                            very interesting indeed :-) Besides the fact of problems with
                            synchronization, I find the mere possibility to get an amazing HFOV of
                            190 degrees with such a rather cheap equipment really tempting. A quick
                            online check showed me I could get a Nikon CP5100 and the FC-E8 for
                            about 440 EUR altogether. Right now, I am working with an Olympus E-330
                            DSLR and the original Olympus Fisheye lens which gives me, due to the
                            FourThirds-crop, a HFOV of far less than 180 degrees (I need 6 images to
                            shoot 360° with sufficient overlap). So maybe this could be a good
                            "second system" for me, for fast pano shooting :-)

                            Best regards,
                            Philipp


                            michel thoby schrieb:
                            > Unbelievable coincidence!
                            > I am currently experimenting with "DuoPix" and was shooting some
                            > street views this afternoon just to finally qualify the imaging
                            > system chain.
                            > DuoPix is just a modernized revival of Dan Slater's Spherecam though.
                            >
                            > Details are here:
                            > http://michel.thoby.free.fr/DuoPix/DuoPix_solution.html
                            >
                            > Comments welcome.
                            >
                            > Best regards,
                            >
                            > Michel
                            >
                          • Philipp B. Koch
                            ... OK, so after all these solutions with only two or three cams, I wonder if one could get some camcorders to work with adapted fisheyes so that they had a
                            Message 13 of 16 , Apr 26, 2008
                            • 0 Attachment
                              panovrx schrieb:
                              > cf:
                              > http://www.mediavr.com/twinrig1.jpg
                              >
                              > and I think you are right -- the additional vertical parallax is worth
                              > it for the reduction in horizontal --
                              > I made this rig (2 5D+trimmed 10.5mm) for 360 timelapse and you get 360
                              > frames about 113 degrees high
                              >
                              > Peter

                              OK, so after all these solutions with only two or three cams, I wonder
                              if one could get some camcorders to work with adapted fisheyes so that
                              they had a huge HFOV as well? But, even with HD-camcorders, the quality
                              would still be an issue, I think. On the other hand, video panos (and
                              that's what I am interested here) can't have too big resolutions anyway,
                              since there must be a possibility to stream them.

                              Philipp
                            • Alan Ball
                              ... Nevertheless, it is possible to shift the no-parallax point optically. When I measured the no-parallax point on my Nikon Coolpix 995, it was just a couple
                              Message 14 of 16 , Apr 26, 2008
                              • 0 Attachment
                                --- In PanoToolsNG@yahoogroups.com, "Erik Krause" <erik.krause@...> wrote:
                                > You can't automatically fix parallax errors, no matter how much you
                                > know about the shooting details. If some background is hidden behind
                                > a foreground detail there is no way to recover the background.

                                Nevertheless, it is possible to shift the no-parallax point
                                optically. When I measured the no-parallax point on my Nikon
                                Coolpix 995, it was just a couple of cm behind the front of
                                the camera. However, with the TC-E2 telextender mounted, the
                                no-parallax point shifted to a point about 17 cm behind the
                                front of the lens, which was well behind the camera body.


                                For that particular camera-lens system, the no-parallax point
                                shift was would not be especially useful for your purpose because
                                the special optic produces a telephoto effect. Nevertheless,
                                it should be possible to design a lens more suitable for
                                a multi-camera panorama or, if you are lucky, to find a stock
                                lens combination that would suit your purpose.

                                scoundrel1728
                              • Philipp B. Koch
                                Dear Michel, we ve mailed some months ago about the Nikon P-5100 in combination with the Nikon fisheye adapter. Thanks again for all the information! I finally
                                Message 15 of 16 , Oct 28, 2008
                                • 0 Attachment
                                  Dear Michel,

                                  we've mailed some months ago about the Nikon P-5100 in combination with
                                  the Nikon fisheye adapter. Thanks again for all the information!
                                  I finally have all the stuff together and found enough time to
                                  experiment a little with the combination. So far, I've ended up using
                                  three shots since
                                  with two shots there's often a problem with color banding...

                                  But I would like to ask you one more question, if I may: How do you set
                                  the camera focus when taking pictures with the FE adapter? I tried both
                                  setting focus to infinity (the "mountain symbol") and setting it to
                                  autofocus. In each case, the pictures do not come out as sharp as I
                                  would have wished.
                                  Maybe I should tweak some other settings as well? (Besides, I found that
                                  turning off the cameras lens distorsion correction gives better results
                                  in combination with the fisheye adapter. Is that also your experience?)

                                  Thanks in advance, and thanks for sharing your finding about the cam and
                                  the fisheye adapter on your website!
                                  Best regards from Berlin,
                                  Philipp
                                • Philipp B. Koch
                                  Oops, sorry, this mail was not meant to be sent at PanotoolsNG ;-) Please ignore it. Best regards, Philipp Koch
                                  Message 16 of 16 , Oct 28, 2008
                                  • 0 Attachment
                                    Oops, sorry, this mail was not meant to be sent at PanotoolsNG ;-)
                                    Please ignore it.

                                    Best regards, Philipp Koch


                                    Philipp B. Koch schrieb:
                                    > Dear Michel,
                                    >
                                    > we've mailed some months ago about the Nikon P-5100 in combination with
                                    > the Nikon fisheye adapter. Thanks again for all the information!
                                    > I finally have all the stuff together and found enough time to
                                    > experiment a little with the combination. So far, I've ended up using
                                    > three shots since
                                    > with two shots there's often a problem with color banding...
                                    >
                                    > But I would like to ask you one more question, if I may: How do you set
                                    > the camera focus when taking pictures with the FE adapter? I tried both
                                    > setting focus to infinity (the "mountain symbol") and setting it to
                                    > autofocus. In each case, the pictures do not come out as sharp as I
                                    > would have wished.
                                    > Maybe I should tweak some other settings as well? (Besides, I found that
                                    > turning off the cameras lens distorsion correction gives better results
                                    > in combination with the fisheye adapter. Is that also your experience?)
                                    >
                                    > Thanks in advance, and thanks for sharing your finding about the cam and
                                    > the fisheye adapter on your website!
                                    > Best regards from Berlin,
                                    > Philipp
                                    >
                                    > ------------------------------------
                                    >
                                    >
                                  Your message has been successfully submitted and would be delivered to recipients shortly.