Loading ...
Sorry, an error occurred while loading the content.

Re: How does one get CMUcam into Line Mode?`

Expand Messages
  • peter_chatterton2001
    I ve written a program which is basically: while ((iData = com2InputStream.read()) != -1) { System.out.println( iData ); } but the output doesn t make any
    Message 1 of 17 , Oct 1, 2008
    • 0 Attachment
      I've written a program which is basically:
      while ((iData = com2InputStream.read()) != -1)
      {
      System.out.println( iData );
      }
      but the output doesn't make any sense:
      the Send Frame command generates 34 bytes

      Here is yesterday's output, which makes some sense but is still too
      short (for SF):

      CommsQuestion
      P Chatterton
      Send Frame
      1
      125
      123
      121
      120
      81
      131
      33
      43
      115
      93
      111
      105
      132
      120
      137
      101
      117
      112
      113
      121
      29
      28
      129
      129
      26
      127
      37
      105
      91
      144
      129
      105
      87
      Total of 34 numeric lines


      CommsQuestion
      P Chatterton
      Track Color
      255
      255
      0
      0
      0
      255
      0
      0
      0
      255
      0
      0
      0
      84
      255
      0
      0
      0
      and so on

      Here is today's Output.
      According to page 50 of the CMUcam2 manual, this is all wrong (for
      SF).

      CommsQuestion
      P Chatterton
      Send Frame
      Read InputStream, available =63
      117,
      121,
      49,
      103,
      106,
      105,
      126,
      112,
      116,
      53,
      19,
      25,
      19,
      44,
      119,
      124,
      18,
      21,
      125,
      129,
      112,
      47,
      126,
      90,
      112,
      117,
      121,
      131,
      116,
      129,
      -- it just stops, not even showing all 63 bytes.

      CommsQuestion
      P Chatterton
      Track Window
      Read InputStream, available =0
      255,
      255,
      255,
      254,
      255,
      255,
      ...and so on, ad infinitum.

      import com.ridgesoft.intellibrain.IntelliBrain;
      import javax.comm.SerialPort;
      import java.io.InputStream;
      import com.ridgesoft.robotics.sensors.CMUcam.CMUcam2;
      // *
      // * CommsQuestion
      // * Reading CMUcam data directly
      // *
      // *
      public class CommsQuestion
      {
      SerialPort comPort;
      InputStream com2InputStream;
      int iData;

      public CommsQuestion()
      {
      }
      public void test1()
      {
      try
      {
      // *
      // * Misc init.
      // *
      System.out.println( "CommsQuestion" );
      System.out.println( "P Chatterton" );
      // *
      // * Create & init the cam.
      // *
      CMUcam2 camera = new CMUcam2(IntelliBrain.getCom2(),
      115200);
      //
      comPort = IntelliBrain.getCom2();
      comPort.setSerialPortParams(115200, SerialPort.DATABITS_8,
      SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);
      com2InputStream = comPort.getInputStream();
      //
      camera.open();
      camera.setRGBMode(false); // switch to CrYCb
      mode
      camera.setWhiteBalance(true);
      camera.setAutoExposure(true);
      Thread.sleep(5000); // let the sensor
      adjust
      camera.setWhiteBalance(false); // disable further
      adjustment
      camera.setAutoExposure(false);
      // *
      // * Read a screen using either trackWindow or trackColor.
      // *
      //System.out.println( "Track Color" );
      //camera.trackColor();

      //System.out.println( "Track Window" );
      //camera.trackWindow();

      System.out.println( "Send Frame" );
      camera.sendCommand( "SF", false );
      // *
      // * Display data from cam's output stream
      // *
      System.out.println( "Read InputStream, available =" +
      com2InputStream.available() );
      while ((iData = com2InputStream.read()) != -1)
      {
      System.out.println( (int) iData + "," );
      }
      System.out.println( "Finished with InputStream, wait a
      sec..." );
      } // end try
      catch (Throwable t)
      {
      t.printStackTrace();
      }
      } // end main
      public static void main(String args[])
      {
      CommsQuestion p = new CommsQuestion();
      p.test1();
      }
      } // end CommsQuestion
      // C:\Robotics\FireFighting\CommsQuestion\CommsQuestion.java
    • RidgeSoft
      Hi Peter, The CMUcam class implements a thread that reads the input stream and processes the messages coming from the camera. Your program is attempting to
      Message 2 of 17 , Oct 2, 2008
      • 0 Attachment
        Hi Peter,

        The CMUcam class implements a thread that reads the input stream and
        processes the messages coming from the camera. Your program is
        attempting to read the same input stream. With two threads reading
        the same input stream, it isn't likely that either one will get what
        it expects.

        If you want your program to communicate directly with the CMUcam, use
        the serial input and output streams directly and don't use the CMUcam
        class. The purpose of the CMUcam class is to manage communications
        with the camera and present a simplified programatic interface to the
        camera. If the class meets your needs, you'll be better off using it
        and letting it manage communications with the camera. If the class
        doesn't meet your needs, you'll need to program using the IO streams
        directly. The CMUcam class doesn't provide for shared use of the IO
        streams.

        Regards,

        RidgeSoft Support

        --- In intellibrain@yahoogroups.com, "peter_chatterton2001"
        <peter@...> wrote:
        >
        > I've written a program which is basically:
        > while ((iData = com2InputStream.read()) != -1)
        > {
        > System.out.println( iData );
        > }
        > but the output doesn't make any sense:
        > the Send Frame command generates 34 bytes
        >
        > Here is yesterday's output, which makes some sense but is still too
        > short (for SF):
        >
        > CommsQuestion
        > P Chatterton
        > Send Frame
        > 1
        > 125
        > 123
        > 121
        > 120
        > 81
        > 131
        > 33
        > 43
        > 115
        > 93
        > 111
        > 105
        > 132
        > 120
        > 137
        > 101
        > 117
        > 112
        > 113
        > 121
        > 29
        > 28
        > 129
        > 129
        > 26
        > 127
        > 37
        > 105
        > 91
        > 144
        > 129
        > 105
        > 87
        > Total of 34 numeric lines
        >
        >
        > CommsQuestion
        > P Chatterton
        > Track Color
        > 255
        > 255
        > 0
        > 0
        > 0
        > 255
        > 0
        > 0
        > 0
        > 255
        > 0
        > 0
        > 0
        > 84
        > 255
        > 0
        > 0
        > 0
        > and so on
        >
        > Here is today's Output.
        > According to page 50 of the CMUcam2 manual, this is all wrong (for
        > SF).
        >
        > CommsQuestion
        > P Chatterton
        > Send Frame
        > Read InputStream, available =63
        > 117,
        > 121,
        > 49,
        > 103,
        > 106,
        > 105,
        > 126,
        > 112,
        > 116,
        > 53,
        > 19,
        > 25,
        > 19,
        > 44,
        > 119,
        > 124,
        > 18,
        > 21,
        > 125,
        > 129,
        > 112,
        > 47,
        > 126,
        > 90,
        > 112,
        > 117,
        > 121,
        > 131,
        > 116,
        > 129,
        > -- it just stops, not even showing all 63 bytes.
        >
        > CommsQuestion
        > P Chatterton
        > Track Window
        > Read InputStream, available =0
        > 255,
        > 255,
        > 255,
        > 254,
        > 255,
        > 255,
        > ...and so on, ad infinitum.
        >
        > import com.ridgesoft.intellibrain.IntelliBrain;
        > import javax.comm.SerialPort;
        > import java.io.InputStream;
        > import com.ridgesoft.robotics.sensors.CMUcam.CMUcam2;
        > // *
        > // * CommsQuestion
        > // * Reading CMUcam data directly
        > // *
        > // *
        > public class CommsQuestion
        > {
        > SerialPort comPort;
        > InputStream com2InputStream;
        > int iData;
        >
        > public CommsQuestion()
        > {
        > }
        > public void test1()
        > {
        > try
        > {
        > // *
        > // * Misc init.
        > // *
        > System.out.println( "CommsQuestion" );
        > System.out.println( "P Chatterton" );
        > // *
        > // * Create & init the cam.
        > // *
        > CMUcam2 camera = new CMUcam2(IntelliBrain.getCom2(),
        > 115200);
        > //
        > comPort = IntelliBrain.getCom2();
        > comPort.setSerialPortParams(115200,
        SerialPort.DATABITS_8,
        > SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);
        > com2InputStream = comPort.getInputStream();
        > //
        > camera.open();
        > camera.setRGBMode(false); // switch to CrYCb
        > mode
        > camera.setWhiteBalance(true);
        > camera.setAutoExposure(true);
        > Thread.sleep(5000); // let the sensor
        > adjust
        > camera.setWhiteBalance(false); // disable further
        > adjustment
        > camera.setAutoExposure(false);
        > // *
        > // * Read a screen using either trackWindow or
        trackColor.
        > // *
        > //System.out.println( "Track Color" );
        > //camera.trackColor();
        >
        > //System.out.println( "Track Window" );
        > //camera.trackWindow();
        >
        > System.out.println( "Send Frame" );
        > camera.sendCommand( "SF", false );
        > // *
        > // * Display data from cam's output stream
        > // *
        > System.out.println( "Read InputStream, available =" +
        > com2InputStream.available() );
        > while ((iData = com2InputStream.read()) != -1)
        > {
        > System.out.println( (int) iData + "," );
        > }
        > System.out.println( "Finished with InputStream, wait a
        > sec..." );
        > } // end try
        > catch (Throwable t)
        > {
        > t.printStackTrace();
        > }
        > } // end main
        > public static void main(String args[])
        > {
        > CommsQuestion p = new CommsQuestion();
        > p.test1();
        > }
        > } // end CommsQuestion
        > // C:\Robotics\FireFighting\CommsQuestion\CommsQuestion.java
        >
      • peter_chatterton2001
        On Sept 28 I was told that the example programs IntelliBrainComHello and IntelliBrainComEcho interfaced directly to the serial ports and that s what I based my
        Message 3 of 17 , Oct 2, 2008
        • 0 Attachment
          On Sept 28 I was told that the example programs IntelliBrainComHello
          and IntelliBrainComEcho interfaced directly to the serial ports and
          that's what I based my code on. This was confirmed by the Intellibrain
          User Guide on pages 18 and 19.

          Now you're saying not to do this.

          Could you please tell me what to use?

          Peter
        • RidgeSoft
          Hi Peter, It sounds like you plan to use features of the camera that are not supported by the CMUcam class. Therefore, you need to interface directly to the
          Message 4 of 17 , Oct 2, 2008
          • 0 Attachment
            Hi Peter,

            It sounds like you plan to use features of the camera that are not
            supported by the CMUcam class. Therefore, you need to interface
            directly to the camera via the serial ports, instead of using the
            CMUcam class. The code you included in your post appears to still be
            using the CMUcam class. You need to remove the code that instantiates
            the CMUcam2 "camera" object:

            CMUcam2 camera = new CMUcam2(IntelliBrain.getCom2(),
            115200);

            and all references to the "camera" object, such as camera.open(). You
            will need to accomplish these functions by sending appropriate commands
            directly to the CMUcam via the serial input and output streams.

            Regards,

            RidgeSoft Support

            --- In intellibrain@yahoogroups.com, "peter_chatterton2001" <peter@...>
            wrote:
            >
            > On Sept 28 I was told that the example programs IntelliBrainComHello
            > and IntelliBrainComEcho interfaced directly to the serial ports and
            > that's what I based my code on. This was confirmed by the
            Intellibrain
            > User Guide on pages 18 and 19.
            >
            > Now you're saying not to do this.
            >
            > Could you please tell me what to use?
            >
            > Peter
            >
          • peter_chatterton2001
            Okay, I now understand what you re saying about communicating directly, but aren t you implying that the IntelliBrainComEcho example is incorrect? Thanks,
            Message 5 of 17 , Oct 2, 2008
            • 0 Attachment
              Okay, I now understand what you're saying about communicating
              directly, but aren't you implying that the IntelliBrainComEcho
              example is incorrect?

              Thanks,
              Peter

              --- In intellibrain@yahoogroups.com, "RidgeSoft" <rs1@...> wrote:
              >
              > Hi Peter,
              >
              > It sounds like you plan to use features of the camera that are not
              > supported by the CMUcam class. Therefore, you need to interface
              > directly to the camera via the serial ports, instead of using the
              > CMUcam class. The code you included in your post appears to still
              be
              > using the CMUcam class. You need to remove the code that
              instantiates
              > the CMUcam2 "camera" object:
              >
              > CMUcam2 camera = new CMUcam2(IntelliBrain.getCom2(),
              > 115200);
              >
              > and all references to the "camera" object, such as camera.open().
              You
              > will need to accomplish these functions by sending appropriate
              commands
              > directly to the CMUcam via the serial input and output streams.
              >
              > Regards,
              >
              > RidgeSoft Support
              >
              > --- In intellibrain@yahoogroups.com, "peter_chatterton2001"
              <peter@>
              > wrote:
              > >
              > > On Sept 28 I was told that the example programs
              IntelliBrainComHello
              > > and IntelliBrainComEcho interfaced directly to the serial ports
              and
              > > that's what I based my code on. This was confirmed by the
              > Intellibrain
              > > User Guide on pages 18 and 19.
              > >
              > > Now you're saying not to do this.
              > >
              > > Could you please tell me what to use?
              > >
              > > Peter
              > >
              >
            • RidgeSoft
              Yes, the example is a good starting point. It illustrates how to read and write a serial port. Of course you will replace the echoing code with code that
              Message 6 of 17 , Oct 2, 2008
              • 0 Attachment
                Yes, the example is a good starting point. It illustrates how to read
                and write a serial port. Of course you will replace the echoing code
                with code that sends commands to the camera and code that processes the
                data the cammera sends back.

                Regards,

                RidgeSoft Support

                --- In intellibrain@yahoogroups.com, "peter_chatterton2001" <peter@...>
                wrote:
                >
                > Okay, I now understand what you're saying about communicating
                > directly, but aren't you implying that the IntelliBrainComEcho
                > example is incorrect?
                >
                > Thanks,
                > Peter
              • Paul King
                ... Hi Peter, I m another end-user like you, however I ve spent some time working on serial port communications on the IntelliBrain card, so I can comment here
                Message 7 of 17 , Oct 3, 2008
                • 0 Attachment
                  > aren't you implying that the IntelliBrainComEcho
                  > example
                  is incorrect
                   
                  Hi Peter,
                   
                  I'm another end-user like you, however I've spent some time working on serial port communications on the IntelliBrain card, so I can comment here from experience.
                   
                  A full serial port driver for a device can be a moderately complex piece of software to write.  You're looking at 50 - 400 lines of code, depending on how many cases you want to cover.
                   
                  If you take out the step of opening the serial port, the IntelliBrainComEcho example is only 2 lines long:
                      while ((data = inputStream.read()) != -1)
                          outputStream.write((byte) data);
                  And all it does is read a byte of input and write it to the output.  This illustrates the serial API but doesn't do anything at all to drive the device.  It is also illustrates a device-side implementation, whereas you want a controller-side implementation.
                   
                  The CMUCam2 downloads page has an example of a Java application that controls the CMUCam2 over a serial port, so this might be a good place to start for examples of Java code that can drive the CMUCam2 serial protocol:
                   
                  The Java class that handles reading the output of the CMUCam2 is called CameraSerial.java and is 360 lines of source code.  This code would need to be adapted to the subset of the Java library supported on the IntelliBrain, but the code to parse the CMUCam's output could probably be taken verbatim.  Unfortunately a lot of the CMUCam2 driver commands are scattered throughout this demo application, which is 3400 lines of Java source code altogether.  (The CMUCam2 has 47 commands and about 20 of them seem to be implemented in the demo.)  Fortunately 80% of it is the demo GUI application and only 20% seems to be specific to the CMUCam2. 
                   
                  I was looking at the earlier message of your original project goal -- "It's a robot for a fire-fighting competition (in two months) and I want to use vision to follow the white wall/black floor boundary and to spot the doorway indicator (a white line)."
                   
                  I work in the field of computer vision, and I'm a little skeptical that you could accomplish this while having IntelliBrain to the image processing, unless you happen to know a particular trick to solve this problem.
                   
                  The CMU "Line Mode" provides a lot of raw information in bit-packed format.  A CMUCam has 40,000 pixels, and the IntelliBrain can only do 10,000 loop iterations per second when there is no computation being performed per iteration.  Unless you know an especially clever algorithm for finding the visual structural elements you mention within an image, doing so could take multiple minutes per frame on the IntelliBrain, provided it didn't exceed it's 132KB memory capacity.
                   
                  If floating point math is involved (which is used in a lot of computer vision algorithms), you have to deal with the fact that the IntelliBrain is 40,000 times slower than a desktop PC at floating point (5,000 flops for IntelliBrain vs. 20 gigaflops for an Intel Core 2 Duo).
                   
                  Good luck!
                   
                  Paul
                   
                • peter_chatterton2001
                  Hi Paul, I really appreciate your comments; if I don t give up because of them, they will be basis of my ongoing work! One thing I might not have explained
                  Message 8 of 17 , Oct 4, 2008
                  • 0 Attachment
                    Hi Paul,

                    I really appreciate your comments; if I don't give up because of them,
                    they will be basis of my ongoing work!

                    One thing I might not have explained properly is that I want to use
                    Line Mode in conjuction with TC or TW. My understanding was that if I
                    only had one horizontal line then the output of the TC would be the
                    data for that line. I didn't want to get into image analysis.

                    Thanks,
                    Peter
                  • Paul King
                    Hi Peter, I see. I m not sure what you mean by if I only had one horizontal line then the output of the TC would be the data for that line The lines that
                    Message 9 of 17 , Oct 4, 2008
                    • 0 Attachment
                      Hi Peter,
                       
                      I see.
                       
                      I'm not sure what you mean by "if I only had one horizontal line then the output of the TC would be the data for that line"
                       
                      The "lines" that are returned in "line mode" on the CMU cam are scan lines of the captured image frame, and not visual lines, edges, or object contours present in the visual scene.
                       
                      So when you say "only one horizontal line" I'm not sure what this is referring to.  A long straight object in the visual scene, such as the door marker you refer to, when viewed in 3D perspective, projected onto the camera's CCD, and captured as a frame of image data, will most likely look like a slightly diagonal noisy thin rectangular region that spans many scan lines and many pixel columns.  If you tell the CMUCam to track all "white pixels" (to get your door tape line and the wall), both the tape region and the wall will be selected, plus any other white pixels caused by white lighting, other objects, or reflections.  Furthermore, all those white pixels in aggregate might span 100 camera scan lines, all of which would be returned by line mode.  How to figure out which 200 of the 3000 white pixels you get back are coming from the white tape line at the door is not an easy image analysis problem to solve.  It's especially hard if the tape line is not perfectly horizontal, which would only happen if the camera is facing it exactly straight on.  In a situation in which the room is being explored, you would rarely encounter this situation.
                       
                      There is an algorithm for finding straight lines from scanned image data called the Hough Transform.  However this algorithm is mathematically complex, involving matrix math and linear algebra, and is beyond the floating point capacity of the IntelliBrain.
                       
                      The only possibility I can see is that if you could come up with a trick for finding the door tape line with the CMUCam's built-in capabilities, you could use that.  For example, if the tape line is flourescent yellow whereas the wall is white, then you could select flourescent yellow pixels with the CMU cam, and use the location of the resulting pixel region as a navigation target without ever determining its shape (e.g. that it was a contiguous linear region).
                       
                      Which line mode were you specifically considering (there are two that affect TC and TC)?
                       
                      btw, my previous email had a math error.  The IntelliBrain is 4 million times slower at floating point than a Core2 Duo PC, not 40,000 times slower.  It's a useful perspective for what a microcontroller is really capable of and suitable for.  At the other end of the spectrum, a 3D graphics card is up to 60x faster yet again than a PC at floating point math.
                          IntelliBrain = 5,000 flops
                          Cray Y-MP in 1988 = 1 gigaflop
                          Intel Core2 CPU = 10 gigaflops per core
                          NVIDIA 9800 GTX 3D graphics card = 600 gigaflops (for $230!)
                       
                      Paul
                       
                       
                      ----- Original Message -----
                      Sent: Saturday, October 04, 2008 3:31 PM
                      Subject: [intellibrain] Re: How does one get CMUcam into Line Mode?`

                      Hi Paul,

                      I really appreciate your comments; if I don't give up because of them,
                      they will be basis of my ongoing work!

                      One thing I might not have explained properly is that I want to use
                      Line Mode in conjuction with TC or TW. My understanding was that if I
                      only had one horizontal line then the output of the TC would be the
                      data for that line. I didn't want to get into image analysis.

                      Thanks,
                      Peter

                    • peter_chatterton2001
                      Hi Paul, You guessed right, I was completely misled by the docs that couldn t be bothered to specify that they were talking about scan lines; see page 40, GM
                      Message 10 of 17 , Oct 7, 2008
                      • 0 Attachment
                        Hi Paul,

                        You guessed right, I was completely misled by the docs that
                        couldn't be bothered to specify that they were talking about scan
                        lines; see page 40, GM 1,2 of the CMUcam2 User Guide: "Sends the
                        [data] for every line being tracked in the image." How can you track
                        a scan line?

                        I'm going to dump the camera for this project but use it for a
                        mini-Magellan contest where I'll be looking for traffic cones
                        outdoors.

                        As a diagnostic tool I might try and write a program that would
                        pass back the cam's current image. This would be indep of the CMUcam
                        classes and only run when the RoboJDE camera stuff is not running. I
                        would probably hook up the USB to serial cable for it. Do you see
                        any obvious problems with this? It seems to me it would be a useful
                        tool.

                        Thanks,
                        Peter
                      • peter_chatterton2001
                        Hi Paul, You guessed right, I was completely misled by the docs that couldn t be bothered to specify that they were talking about scan lines; see page 40, GM
                        Message 11 of 17 , Oct 7, 2008
                        • 0 Attachment
                          Hi Paul,

                          You guessed right, I was completely misled by the docs that
                          couldn't be bothered to specify that they were talking about scan
                          lines; see page 40, GM 1,2 of the CMUcam2 User Guide: "Sends the
                          [data] for every line being tracked in the image." How can you track
                          a scan line?

                          I'm going to dump the camera for this project but use it for a
                          mini-Magellan contest where I'll be looking for traffic cones
                          outdoors.

                          As a diagnostic tool I might try and write a program that would
                          pass back the cam's current image. This would be indep of the CMUcam
                          classes and only run when the RoboJDE camera stuff is not running. I
                          would probably hook up the USB to serial cable for it. Do you see
                          any obvious problems with this? It seems to me it would be a useful
                          tool.

                          Thanks,
                          Peter
                        Your message has been successfully submitted and would be delivered to recipients shortly.