Loading ...
Sorry, an error occurred while loading the content.

BE slug-os images are porkier WRT LE images.

Expand Messages
  • Mark Gross
    I just noticed that my customized slug-os image when built with BE results in only 4 block free on the flash, but when I build using LE I have ~400 blocks
    Message 1 of 13 , Jul 21, 2007
    • 0 Attachment
      I just noticed that my "customized" slug-os image when built with BE
      results in only 4 block free on the flash, but when I build using LE I
      have ~400 blocks free on the flash.

      Also, given that USB camera drivers (and I'm sure other peripherals)
      tend to work better running LE, why then does anyone use the BE build?

      Is there a good justification for the big endean build at all?

      --mgross
    • Mike (mwester)
      (Mark - there seems to be a problem with your mailer; it s sending your message text as an attachment, with an empty body). Yes, there are many valid reasons
      Message 2 of 13 , Jul 21, 2007
      • 0 Attachment
        (Mark - there seems to be a problem with your mailer; it's sending your
        message text as an attachment, with an empty body).

        Yes, there are many valid reasons for big endian builds. Both endians are
        legitimate, and the fact that certain drivers and software "tend to work
        better" is an incorrect characterization. Rather, those drivers and
        software are *flawed* in that they assume things about the architecture that
        they should not, endian-ness being only one of the said assumptions. Things
        like word length, data alignment, presence of floating point, and others are
        other examples of assumptions various software packages make, assumptions
        that are often wrong on anything other than the architecture those packages
        were developed upon.

        Mike (mwester)

        ----- Original Message -----
        From: "Mark Gross" <markgross@...>
        To: <nslu2-linux@yahoogroups.com>
        Sent: Saturday, July 21, 2007 11:04 AM
        Subject: [nslu2-linux] BE slug-os images are porkier WRT LE images.
      • Mark Gross
        ... That sucks. I thought by using mutt I wouldn t have such problems :( ... such as? ... The software is fine. The camera driver works, but the camera
        Message 3 of 13 , Jul 22, 2007
        • 0 Attachment
          On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
          > (Mark - there seems to be a problem with your mailer; it's sending your
          > message text as an attachment, with an empty body).

          That sucks. I thought by using mutt I wouldn't have such problems :(


          >
          > Yes, there are many valid reasons for big endian builds. Both endians are

          such as?

          > legitimate, and the fact that certain drivers and software "tend to work
          > better" is an incorrect characterization. Rather, those drivers and
          > software are *flawed* in that they assume things about the architecture that
          > they should not, endian-ness being only one of the said assumptions. Things

          The software is fine. The camera driver works, but the camera doesn't
          understand BE.
          (It assumes LE) I think this is a valid and correct characterization of
          the way thing are. To fix this issue you'll need to put #ifdef BE's in
          the spc5xxx code, or use a drive / api that only take 8 bit values.
          (V4L API's take multi-bytes)

          There is not value in doing this so I have to ask, why bother with BE?
          It results in larger code and has compatibility issues with peripherals.
          (not the drivers the devices themselves)

          --mgross

          > like word length, data alignment, presence of floating point, and others are
          > other examples of assumptions various software packages make, assumptions
          > that are often wrong on anything other than the architecture those packages
          > were developed upon.
          >
          > Mike (mwester)
          >
          > ----- Original Message -----
          > From: "Mark Gross" <markgross@...>
          > To: <nslu2-linux@yahoogroups.com>
          > Sent: Saturday, July 21, 2007 11:04 AM
          > Subject: [nslu2-linux] BE slug-os images are porkier WRT LE images.
          >
          >
          >
          >
          >
          > Yahoo! Groups Links
          >
          >
          >
        • Bernhard Walle
          ... Not Mutt sucks but “Outlook Express”, the mail client the reader uses. Btw: Microsoft Outlook doesn’t have any problems. :-) If you want to avoid the
          Message 4 of 13 , Jul 22, 2007
          • 0 Attachment
            * Mark Gross <markgross@...> [2007-07-22 17:57]:
            > On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
            > > (Mark - there seems to be a problem with your mailer; it's sending your
            > > message text as an attachment, with an empty body).
            >
            > That sucks. I thought by using mutt I wouldn't have such problems :(

            Not Mutt sucks but “Outlook Express”, the mail client the reader uses.
            Btw: Microsoft Outlook doesn’t have any problems. :-)

            If you want to avoid the problem, you cannot use PGP/MIME but you have
            to use Inline-PGP.


            Thanks,
            Bernhard
          • Yann E. MORIN
            Mark and All! ... Network-native endianness is big. Running BE avoids swapping bytes to get higher throughput on constrained processors (IXP4xx are slow,
            Message 5 of 13 , Jul 22, 2007
            • 0 Attachment
              Mark and All!

              On Sunday 22 July 2007 17:57, Mark Gross wrote:
              > On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
              > > Yes, there are many valid reasons for big endian builds. Both endians are
              > such as?

              Network-native endianness is big. Running BE avoids swapping bytes to get
              higher throughput on constrained processors (IXP4xx are slow, remember).

              > The software is fine. The camera driver works, but the camera doesn't
              > understand BE.
              > (It assumes LE)

              Thus, the driver _is_ broken, because it makes hard assumptions upon the
              machine it is running on. Carefully written driver should cope with both
              endianness, and other architecture-dependent issues (alignment, word size
              and so on...).

              It is the responsibility of the driver to correctly _interpret_ the data
              coming from, and going to, the device:

              - if the device sends/wants LE data, then the driver should use the
              cpu_to_leXX and leXX_to_cpu familly of functions before using the data
              coming from the device (where XX is one of 16, 32, 64, maybe even 128...).

              - the same prevails with devices sending BE data: in this case, use the
              cpu_to_beXX and beXX_to_cpu functions.

              - if the device data is endian-neutral (that is, only byte-worth of data),
              then the driver has no need of these functions, of course.

              > I think this is a valid and correct characterization of
              > the way thing are. To fix this issue you'll need to put #ifdef BE's in
              > the spc5xxx code, or use a drive / api that only take 8 bit values.
              > (V4L API's take multi-bytes)

              No #ifdef, please. Use functions provided by the kernel includes! These are
              mostly macros that expand appropriately.

              > There is not value in doing this so I have to ask, why bother with BE?
              > It results in larger code and has compatibility issues with peripherals.
              > (not the drivers the devices themselves)

              Again, if the driver doesn't work with one endianness, then it is plainly
              _broken_. Not taking endianness (and word size, etc...) into account when
              writing a driver is a *major design flaw*.

              Regards,
              Yann E. MORIN.

              --
              .-----------------.--------------------.------------------.--------------------.
              | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
              | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^ |
              | --==< °_° >==-- °------------.-------: X AGAINST | /e\ There is no |
              | http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL | """ conspiracy. |
              °------------------------------°-------°------------------°--------------------°
            • David Given
              Yann E. MORIN wrote: [...] ... It s not entirely obvious why this is so. Here s an example: { char buffer[128]; read(fd, buffer, PACKET_SIZE); return *(long*)
              Message 6 of 13 , Jul 22, 2007
              • 0 Attachment
                Yann E. MORIN wrote:
                [...]
                > Again, if the driver doesn't work with one endianness, then it is plainly
                > _broken_. Not taking endianness (and word size, etc...) into account when
                > writing a driver is a *major design flaw*.

                It's not entirely obvious why this is so. Here's an example:

                {
                char buffer[128];
                read(fd, buffer, PACKET_SIZE);

                return *(long*) (buffer+n);
                }

                This returns the long in the data packet at offset n... right? In fact, this
                code is almost certain to break on some platforms. Not only does it rely on
                the fact that the data packet is in native endianness, which isn't always the
                case, but it also relies on the size of long, which may not be 32, and the
                fact that n is properly aligned, which it might not be. (Some platforms won't
                even throw an exception if you try to read unaligned values. They'll just get
                it *wrong*.)

                But it'll run fine on an i386.

                Basically, this style of code is symptomatic of non-portable design. Just
                putting #ifdefs around the return statement above doesn't fix the underlying
                problem, which is that the code is badly designed. If it was well designed, it
                wouldn't need to care about the native endianness.

                --
                ┌── dg@cowlark.com ─── http://www.cowlark.com ───────────────────

                │ "There does not now, nor will there ever, exist a programming language in
                │ which it is the least bit hard to write bad programs." --- Flon's Axiom
              • Yann E. MORIN
                David and All, ... I didn t say that was easy. :-/ ... Yeup. Bad programming... :-( Even in user space. [--SNIP--] ... That s why I try to run my programs on a
                Message 7 of 13 , Jul 22, 2007
                • 0 Attachment
                  David and All,

                  On Sunday 22 July 2007 21:08, David Given wrote:
                  > Yann E. MORIN wrote:
                  > > Again, if the driver doesn't work with one endianness, then it is plainly
                  > > _broken_. Not taking endianness (and word size, etc...) into account when
                  > > writing a driver is a *major design flaw*.
                  > It's not entirely obvious why this is so. Here's an example:

                  I didn't say that was easy. :-/

                  > {
                  > char buffer[128];
                  > read(fd, buffer, PACKET_SIZE);
                  >
                  > return *(long*) (buffer+n);
                  > }

                  Yeup. Bad programming... :-( Even in user space.

                  [--SNIP--]
                  > But it'll run fine on an i386.

                  That's why I try to run my programs on a second machine that is entirely
                  different from my desktop computer: x86_64 and ARM (nslu2!).

                  Those two machines are really different:
                  - x86_64: 64-bit
                  LE
                  unaligned load/store OK
                  fast

                  - ARM: 32-bit
                  BE and LE
                  unaligned load/store KO
                  slow as hell! :-)

                  Once those work OK, then almost all porting issues are dealt with. I then have
                  to try a MIPS build (WRT54GL) and I'm then almost confident there are no longer
                  any porting issue ( but no one's perfect! ;-) ).

                  > Basically, this style of code is symptomatic of non-portable design. Just
                  > putting #ifdefs around the return statement above doesn't fix the underlying
                  > problem, which is that the code is badly designed. If it was well designed, it
                  > wouldn't need to care about the native endianness.

                  I have to disagree with you. Well designed code _does_ care about endianness,
                  especially in the kernel. If the _hardware_ awaits special endianness, then
                  the _driver_ has to cope with it.

                  Say, a camera sends 16-bit encoded values: RRrr GGcc BBbb (capitals for high-
                  order bytes, R's for red etc...), transmitted in LE over a serial bus (rs232,
                  USB, firewire, whatever), thus the data would come as: rrRRggGGbbBB and so on.

                  The code handling this data would have to byte swap it on BE machines, but not
                  on LE machines, such as:

                  /* sample code not even close to compile */
                  {
                  u16 red, green, blue;
                  read2bytes(fd, &red); red = le16_to_cpu(red);
                  read2bytes(fd, &green); green = le16_to_cpu(green);
                  read2bytes(fd, &blue); blue = le16_to_cpu(blue);
                  send_to_userspace( red, green, blue );
                  }

                  Where le16_to_cpu(X) will expand to simply (X) on LE machines, but to something
                  like ((((X)&0xff00)>>8)|(((X)&0x00ff)<<8)) on BE machines. Look at the kernel
                  source code, there are plenty of examples there...

                  So, yes, the code *has* to care about endianness in this case. And not caring
                  is a design flaw.

                  But... Only drivers should have to handle endianness. Userspace code shouldn't,
                  except when it deals directly with IO's (such as reading the raw rs232 serial
                  port).

                  This is slidding a bit off-topic for the nslu2-linux ML, but I thought some
                  clarification was needed.

                  Regards,
                  Yann E. MORIN.

                  --
                  .-----------------.--------------------.------------------.--------------------.
                  | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
                  | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^ |
                  | --==< °_° >==-- °------------.-------: X AGAINST | /e\ There is no |
                  | http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL | """ conspiracy. |
                  °------------------------------°-------°------------------°--------------------°
                • David Given
                  ... [...] ... Actually, I d be inclined to write this as: char buffer[2]; read2bytes(fd, buffer); red = (buffer[1]
                  Message 8 of 13 , Jul 22, 2007
                  • 0 Attachment
                    Yann E. MORIN wrote:
                    > read2bytes(fd, &red); red = le16_to_cpu(red);
                    [...]
                    > Where le16_to_cpu(X) will expand to simply (X) on LE machines, but to something
                    > like ((((X)&0xff00)>>8)|(((X)&0x00ff)<<8)) on BE machines. Look at the kernel
                    > source code, there are plenty of examples there...

                    Actually, I'd be inclined to write this as:

                    char buffer[2];
                    read2bytes(fd, buffer); red = (buffer[1]<<8) | (buffer[0]);

                    This makes the code clearer, more portable, simpler, more reliable, reduces
                    the maintenance load considerably, and the performance hit is so trivial
                    (three extra instructions on x86, and *one* extra instruction on the ARM!)
                    that it'll be entirely absorbed by the time it takes to call the function...

                    --
                    ┌── dg@cowlark.com ─── http://www.cowlark.com ───────────────────

                    │ "There does not now, nor will there ever, exist a programming language in
                    │ which it is the least bit hard to write bad programs." --- Flon's Axiom
                  • Yann E. MORIN
                    David and All, ... In this case, your code will only work on machines that have the opposite endianness of your hardware. You _do_ make _strong_ assumptions on
                    Message 9 of 13 , Jul 23, 2007
                    • 0 Attachment
                      David and All,

                      On Monday 23 July 2007 00:52, David Given wrote:
                      > Yann E. MORIN wrote:
                      > > read2bytes(fd, &red); red = le16_to_cpu(red);
                      > Actually, I'd be inclined to write this as:
                      > char buffer[2];
                      > read2bytes(fd, buffer); red = (buffer[1]<<8) | (buffer[0]);

                      In this case, your code will only work on machines that have the opposite
                      endianness of your hardware. You _do_ make _strong_ assumptions on the machine
                      you run on. This _is_ a clear design flaw.

                      Plus, you made a mistake (I think): buffer[1] is a char, so it's possible that
                      buffer[1] << 8 overflows. See, that's not that simple!

                      Macros such as leXX_to_cpu et al. have been exercised and are reliable, when
                      local hacks are not.

                      > This makes the code clearer, more portable, simpler, more reliable, reduces
                      > the maintenance load considerably, and the performance hit is so trivial

                      Why don't you like with le16_to_cpu? It does exactly what it's named: convert
                      a 16-bit little endian value in a 16-bit values of the same endianness the CPU
                      is configured in. That _is_ good design, as the code will behave correctly on
                      both endianness. This code _is_ portable, yours _is not_.

                      Plus, it's not obvious why you are swapping bytes there. Using le16_to_cpu is
                      explict, and _that_ is a maintenance gain: no question what it is for, the
                      name says it all.

                      > (three extra instructions on x86, and *one* extra instruction on the ARM!)
                      > that it'll be entirely absorbed by the time it takes to call the function...

                      So what? On some archs, le16_to_cpu might even expand to an assembly
                      instruction that does the swapping (some archs have that).

                      Plus, the compiler can arrange for out-of-order execution for those CPU that
                      are (deeply) pipelined.

                      Plus, some arch can have DMA-mapped coherent memory, when the DMA controller does
                      the swapping by itself. Plus many more issues we can't even imagine...

                      Again, writing endian-neutral, alignement-neutral, word-size-neutral, portable
                      code is no easy task. This is really difficult. But when you have something to
                      do, do it simply, use explicit names and simple constructs. In short, don't try
                      to be smart. In these areas, the compiler is smarter than we could ever be.

                      Regards,
                      Yann E. MORIN.

                      --
                      .-----------------.--------------------.------------------.--------------------.
                      | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
                      | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^ |
                      | --==< °_° >==-- °------------.-------: X AGAINST | /e\ There is no |
                      | http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL | """ conspiracy. |
                      °------------------------------°-------°------------------°--------------------°
                    • David Given
                      ... Hash: SHA1 Yann E. MORIN wrote: [...] ... Um, how so? I m making an assumption of the endianness of the data that s being read in the I/O packet, not about
                      Message 10 of 13 , Jul 27, 2007
                      • 0 Attachment
                        -----BEGIN PGP SIGNED MESSAGE-----
                        Hash: SHA1

                        Yann E. MORIN wrote:
                        [...]
                        > In this case, your code will only work on machines that have the opposite
                        > endianness of your hardware. You _do_ make _strong_ assumptions on the machine
                        > you run on. This _is_ a clear design flaw.

                        Um, how so? I'm making an assumption of the endianness of the data that's
                        being read in the I/O packet, not about the machine; and I'm allowed to make
                        that assumption, because in our hypothetical example I'm a device driver.

                        (I did however misread your original message and produced something that
                        thinks the packet is *big* endian. D'oh.)

                        > Plus, you made a mistake (I think): buffer[1] is a char, so it's possible that
                        > buffer[1] << 8 overflows. See, that's not that simple!

                        buffer[1] ought to get promoted to an int before the shift happens, so it
                        won't overflow --- but the code will still go wrong because chars are signed
                        (I should have used uint8_t). I am assuming that ints are at least 16 bits but
                        I reckon that's justifiable these days.

                        [...]
                        > Why don't you like with le16_to_cpu? It does exactly what it's named:
                        > convert a 16-bit little endian value in a 16-bit values of the same
                        > endianness the CPU is configured in
                        [...]

                        I don't like it because I think it's the wrong abstraction for the job.

                        - From memory, the kernel's preferred idiom is:

                        uint16_t b;
                        read(fd, &b, sizeof(b));
                        b = le16_to_cpu(b);

                        Three immediate maintenance issues I can see with this approach are:

                        - - b is overloaded to behave as both a 16-bit integer and a two-byte buffer.
                        There's an implicit cast to void* in read() that makes this happen. This makes
                        type analysis hard; the value of b, interpreted as an integer, is meaningless
                        until le16_to_cpu writes to it. If the three statements above are not near
                        each other, this can make debugging hard. It also means you don't get to use
                        const correctness.

                        - - the behaviour of the code depends intrinsically on uint16_t being the same
                        size as the encoded value in the buffer. Change it to a uint32_t, and you've
                        introduced a platform-specific bug: the code will still work fine on LE
                        systems, but not BE. This is far too brittle for comfort, particularly when
                        the value you're trying to read out of the buffer is of some other type.

                        - - le16_to_cpu() is *defined* to have platform specific behaviour, which means
                        you've got multiple implementations of it, which means that you've doubled the
                        amount of testing you have to do.

                        The thing is, le16_to_cpu() is fundamentally a low-level operation, a simple
                        wrapper around a byte-swap instruction. That's fine if you're writing machine
                        code, but we're not doing that. What the programmer is actually trying to do
                        is to express the concept 'parse an encoded integer out of a buffer' (or a
                        file descriptor). By using a low-level operation like that, the programmer is
                        forced to second-guess the compiler by manually reading the value into an
                        integer variable, where it doesn't belong. Apart from wasting valuable
                        programmer thought time, this hides information from the compiler --- some
                        platforms don't have a byte-swap instruction, but *do* have the ability to
                        read byte-swapped data out of memory (such as the PowerPC).

                        Compare with the following equivalent code:

                        extern unsigned int read_uint16(uint8_t* buffer);

                        uint8_t buffer[2];
                        int b;
                        read(fd, buffer, sizeof(buffer));
                        b = read_uint16(buffer);

                        The buffer is now declared as a buffer; the type of b is now irrelevant to the
                        encoding in the buffer (and can even be a double or a float, heaven forbid,
                        and the code will still work); type integrity is preserved throughout; it'll
                        probably compile into exactly the same code as before, and may even be faster;
                        read_uint16() can be implemented the same way on all platforms, regardless of
                        endianness, which makes reliability testing easier; and if you're using C99,
                        you can even write it like this for added reliability

                        char buffer[2];
                        read(fd, buffer, sizeof(buffer));
                        const int b = read_uint16(buffer);

                        But most importantly of all, it more accurately represents the programmer's
                        original intention, and so life considerably easier for everybody involved:
                        the original author, any maintenance programmers, even the compiler writers.
                        Just because it's possible to write machine code in C doesn't mean we *have*
                        to --- it's not 1980 any more!

                        This is probably more analysis than three lines of code really deserve, but
                        I've spent a lot of time in the embedded software industry, and I've seen this
                        sort of thing happen a lot (and had to clean up after them, too). I will admit
                        to having perpetrated it a few times, but I really try not to. People tend to
                        miss that abstraction in the wrong place can subtly screw you over later,
                        because people tend to base their thought patterns around the abstractions. A
                        misplaced helper function early in the project can distort the overall design
                        of the application. I'd much rather not abstract over trivial one-liners such
                        as the original byte-swap code until I get a chance to see how it fits in to
                        the larger project.

                        - --
                        ┌── dg@cowlark.com ─── http://www.cowlark.com ───────────────────

                        │ "There does not now, nor will there ever, exist a programming language in
                        │ which it is the least bit hard to write bad programs." --- Flon's Axiom


                        -----BEGIN PGP SIGNATURE-----
                        Version: GnuPG v1.4.6 (GNU/Linux)
                        Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

                        iD8DBQFGqhmYf9E0noFvlzgRAmEVAJ9MojpTDfZ9fpsyDdFVZwYZqyqi5gCguPwi
                        6WbJ23B3DAeWTWYGR9k2zfw=
                        =ogHq
                        -----END PGP SIGNATURE-----
                      • Mark Gross
                        ... I remember they are not too bad. (pretty big cache and what not) How about some data showing BE kicks LE s ass. Because, thats what it will take me to
                        Message 11 of 13 , Jul 30, 2007
                        • 0 Attachment
                          On Sun, Jul 22, 2007 at 08:17:44PM +0200, Yann E. MORIN wrote:
                          > Mark and All!
                          >
                          > On Sunday 22 July 2007 17:57, Mark Gross wrote:
                          > > On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
                          > > > Yes, there are many valid reasons for big endian builds. Both endians are
                          > > such as?
                          >
                          > Network-native endianness is big. Running BE avoids swapping bytes to get
                          > higher throughput on constrained processors (IXP4xx are slow, remember).
                          >

                          I remember they are not too bad. (pretty big cache and what not) How
                          about some data showing BE kicks LE's ass. Because, thats what it will
                          take me to accept the argument that the hassle of getting the BE drivers
                          working is worth the bother.

                          Also, lets not forget that the same slugos-image under BE had zero
                          blocks free, where the LE version of the same image has over 400 blocks
                          free. That's something worth using even if the BE spcxxx driver is not
                          BE safe.

                          Why is the LE image smaller than the BE?

                          > > The software is fine. The camera driver works, but the camera doesn't
                          > > understand BE.
                          > > (It assumes LE)
                          >
                          > Thus, the driver _is_ broken, because it makes hard assumptions upon the
                          > machine it is running on. Carefully written driver should cope with both
                          > endianness, and other architecture-dependent issues (alignment, word size
                          > and so on...).
                          >

                          yeah, but if it works for LE then what is my motivation to fix the
                          driver for BE operation?

                          > It is the responsibility of the driver to correctly _interpret_ the data
                          > coming from, and going to, the device:
                          >
                          > - if the device sends/wants LE data, then the driver should use the
                          > cpu_to_leXX and leXX_to_cpu familly of functions before using the data
                          > coming from the device (where XX is one of 16, 32, 64, maybe even 128...).
                          >
                          > - the same prevails with devices sending BE data: in this case, use the
                          > cpu_to_beXX and beXX_to_cpu functions.
                          >
                          > - if the device data is endian-neutral (that is, only byte-worth of data),
                          > then the driver has no need of these functions, of course.
                          >

                          yes this is all true but somewhat orthogonal to my query.

                          > > I think this is a valid and correct characterization of
                          > > the way thing are. To fix this issue you'll need to put #ifdef BE's in
                          > > the spc5xxx code, or use a drive / api that only take 8 bit values.
                          > > (V4L API's take multi-bytes)
                          >
                          > No #ifdef, please. Use functions provided by the kernel includes! These are
                          > mostly macros that expand appropriately.
                          >
                          > > There is not value in doing this so I have to ask, why bother with BE?
                          > > It results in larger code and has compatibility issues with peripherals.
                          > > (not the drivers the devices themselves)
                          >
                          > Again, if the driver doesn't work with one endianness, then it is plainly
                          > _broken_. Not taking endianness (and word size, etc...) into account when
                          > writing a driver is a *major design flaw*.

                          yes but as I'm not eager to fix the spaxx driver to work under BE
                          systems I don't know what I will do about it.


                          --mgross

                          >
                          > Regards,
                          > Yann E. MORIN.
                          >
                          > --
                          > .-----------------.--------------------.------------------.--------------------.
                          > | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
                          > | +0/33 662376056 | Software Designer | \ / CAMPAIGN | ^ |
                          > | --==< °_° >==-- °------------.-------: X AGAINST | /e\ There is no |
                          > | http://ymorin.is-a-geek.org/ | (*_*) | / \ HTML MAIL | """ conspiracy. |
                          > °------------------------------°-------°------------------°--------------------°
                          >
                          >
                          >
                          >
                          > Yahoo! Groups Links
                          >
                          >
                          >
                        • Mike (mwester)
                          ... endians are ... get ... I m confused now. Didn t you start this discussion asking why the BE images exist? I don t recall that anyone was trying to force
                          Message 12 of 13 , Jul 30, 2007
                          • 0 Attachment
                            On Monday, July 30, 2007 10:43 PM, Mark Gross wrote:

                            > > On Sunday 22 July 2007 17:57, Mark Gross wrote:
                            > > > On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
                            > > > > Yes, there are many valid reasons for big endian builds. Both
                            endians are
                            > > > such as?
                            > >
                            > > Network-native endianness is big. Running BE avoids swapping bytes to
                            get
                            > > higher throughput on constrained processors (IXP4xx are slow, remember).
                            > >
                            >
                            > I remember they are not too bad. (pretty big cache and what not) How
                            > about some data showing BE kicks LE's ass. Because, thats what it will
                            > take me to accept the argument that the hassle of getting the BE drivers
                            > working is worth the bother.

                            I'm confused now. Didn't you start this discussion asking why the BE images
                            exist? I don't recall that anyone was trying to force you to do anything at
                            all. It certainly sounds like you have your mind made up. That's fine.
                            But why are you trying to convince this community that we should drop BE
                            support? Don't answer that -- we'll continue to be endian-neutral, as
                            picking one over the other has been a battle fought over the same
                            blood-drenched turf for decades now. The ARM cpu runs in either, and as
                            long as people are willing to make Linux run on it in either mode, then
                            there's really no issue.

                            > Also, lets not forget that the same slugos-image under BE had zero
                            > blocks free, where the LE version of the same image has over 400 blocks
                            > free. That's something worth using even if the BE spcxxx driver is not
                            > BE safe.
                            >
                            > Why is the LE image smaller than the BE?

                            You're right, but your assumption is flawed.

                            The BE image is larger. The bulk of the difference seems to be due not to
                            endianness, but rather to an error in the build system somewhere.
                            /sbin/ldconfig is dynamically linked in the LE image, but *statically*
                            linked in the BE image. Of course that makes it much larger. There are
                            still some other executables where one is different in size from the other,
                            and perhaps if we ever run so close to the edge in terms of flash space I'll
                            investigate the 2KB size difference here and there that remains after the
                            ldconfig problem gets sorted out.

                            From an instruction set point-of-view, there's no difference. So any
                            difference in data or executable sizes is based on application coding, which
                            can be fixed. But why bother? Just like the speed advantage for BE being
                            of little concern for most applications, a few KB here or there is also of
                            little concern for most applications. For every extreme case supporting
                            this endianness, someone else has another extreme case supporting the other
                            endianness. And so the battle continues.

                            Frankly, I think we should find and fix the ldconfig problem, but beyond
                            that I have no interest in fighting battles over a few bits/second here and
                            a few Kbytes there. I just want it to WORK, no matter what endianness is
                            selected.

                            > > > The software is fine. The camera driver works, but the camera doesn't
                            > > > understand BE.
                            > > > (It assumes LE)
                            > >
                            > > Thus, the driver _is_ broken, because it makes hard assumptions upon the
                            > > machine it is running on. Carefully written driver should cope with both
                            > > endianness, and other architecture-dependent issues (alignment, word
                            size
                            > > and so on...).
                            > >
                            >
                            > yeah, but if it works for LE then what is my motivation to fix the
                            > driver for BE operation?

                            Well, I think that it's clear that YOUR motivation is ZERO. That's fine,
                            that's why this is a hobby for most of us. Nobody has to do what they don't
                            want to do.

                            MY motivation for making something work in both endianness is the same
                            motivation I have to make it work on both x86 and ARM. I write better
                            software that way. It really is amazing how many errors and assumptions
                            creep into code when one writes (even subconciously) for a particular
                            architecture. For example -- the ppp driver in the kernel executes floating
                            point instructions. That's verboten in the kernel. But apparently this
                            slipped through, because the x86 hardware does it regardless of the kernel's
                            rules. Building the driver for ARM made that problem very apparent. There
                            are countless other examples where architectural items such as alignment,
                            atomicity, cache effects, and of course endianness have had latent problems
                            that could have been readily uncovered had the code been tested on other
                            architectures. Having ported the Via Velocity NIC driver from LE-only to
                            support both LE and BE, I am well aware of just how awful code can be that
                            is written with an assumption of a single architecture.

                            I am happy that I have a device -- the NSLU2 -- that can run the same OS and
                            software in both endianness to test my code upon -- it's a huge savings in
                            hardware, as well as a productivity enhancement. And like I said, it also
                            means that I don't have to be quite so ashamed of my code in peer reviews.

                            [snip]
                            > > Again, if the driver doesn't work with one endianness, then it is
                            plainly
                            > > _broken_. Not taking endianness (and word size, etc...) into account
                            when
                            > > writing a driver is a *major design flaw*.
                            >
                            > yes but as I'm not eager to fix the spaxx driver to work under BE
                            > systems I don't know what I will do about it.
                            >

                            Don't do anything about it. Nobody is pushing you to do anything. Again,
                            as I recall the thread, you demanded to know why people bothered with making
                            a BE system, since it didn't seem to meet your needs. You got an answer,
                            which you rejected. No problem, it's a hobby. Nobody is ordering you to
                            port anything to BE, or even use it. Likewise, we'll continue to support
                            both - despite some folks who don't agree with that.

                            Install SlugOS/LE, or Debian -- and enjoy it!

                            >
                            > --mgross
                            >

                            Mike (mwester)
                          • Mark Gross
                            To be clear I never demanded anything. I just posed the question of why bother with the BE. You make some good academic justifications below. If we ve
                            Message 13 of 13 , Jul 31, 2007
                            • 0 Attachment
                              To be clear I never demanded anything. I just posed the question of
                              why bother with the BE. You make some good academic justifications
                              below.

                              If we've exposed a linker bug in the BE build then this thread is a good
                              thing, but I don't think its fair to paint me as demanding anything.

                              --mgross

                              On Tue, Jul 31, 2007 at 12:43:03AM -0500, Mike (mwester) wrote:
                              > On Monday, July 30, 2007 10:43 PM, Mark Gross wrote:
                              >
                              > > > On Sunday 22 July 2007 17:57, Mark Gross wrote:
                              > > > > On Sat, Jul 21, 2007 at 12:49:57PM -0500, Mike (mwester) wrote:
                              > > > > > Yes, there are many valid reasons for big endian builds. Both
                              > endians are
                              > > > > such as?
                              > > >
                              > > > Network-native endianness is big. Running BE avoids swapping bytes to
                              > get
                              > > > higher throughput on constrained processors (IXP4xx are slow, remember).
                              > > >
                              > >
                              > > I remember they are not too bad. (pretty big cache and what not) How
                              > > about some data showing BE kicks LE's ass. Because, thats what it will
                              > > take me to accept the argument that the hassle of getting the BE drivers
                              > > working is worth the bother.
                              >
                              > I'm confused now. Didn't you start this discussion asking why the BE images
                              > exist? I don't recall that anyone was trying to force you to do anything at
                              > all. It certainly sounds like you have your mind made up. That's fine.
                              > But why are you trying to convince this community that we should drop BE
                              > support? Don't answer that -- we'll continue to be endian-neutral, as

                              I'm not trying to convince anyone, but I did want to know if I was
                              missing something.

                              > picking one over the other has been a battle fought over the same
                              > blood-drenched turf for decades now. The ARM cpu runs in either, and as
                              > long as people are willing to make Linux run on it in either mode, then
                              > there's really no issue.
                              >
                              > > Also, lets not forget that the same slugos-image under BE had zero
                              > > blocks free, where the LE version of the same image has over 400 blocks
                              > > free. That's something worth using even if the BE spcxxx driver is not
                              > > BE safe.
                              > >
                              > > Why is the LE image smaller than the BE?
                              >
                              > You're right, but your assumption is flawed.
                              >
                              > The BE image is larger. The bulk of the difference seems to be due not to
                              > endianness, but rather to an error in the build system somewhere.
                              > /sbin/ldconfig is dynamically linked in the LE image, but *statically*
                              > linked in the BE image. Of course that makes it much larger. There are
                              > still some other executables where one is different in size from the other,
                              > and perhaps if we ever run so close to the edge in terms of flash space I'll
                              > investigate the 2KB size difference here and there that remains after the
                              > ldconfig problem gets sorted out.
                              >

                              Cool.

                              > >From an instruction set point-of-view, there's no difference. So any
                              > difference in data or executable sizes is based on application coding, which
                              > can be fixed. But why bother? Just like the speed advantage for BE being
                              > of little concern for most applications, a few KB here or there is also of
                              > little concern for most applications. For every extreme case supporting
                              > this endianness, someone else has another extreme case supporting the other
                              > endianness. And so the battle continues.
                              >
                              > Frankly, I think we should find and fix the ldconfig problem, but beyond
                              > that I have no interest in fighting battles over a few bits/second here and
                              > a few Kbytes there. I just want it to WORK, no matter what endianness is
                              > selected.
                              >
                              > > > > The software is fine. The camera driver works, but the camera doesn't
                              > > > > understand BE.
                              > > > > (It assumes LE)
                              > > >
                              > > > Thus, the driver _is_ broken, because it makes hard assumptions upon the
                              > > > machine it is running on. Carefully written driver should cope with both
                              > > > endianness, and other architecture-dependent issues (alignment, word
                              > size
                              > > > and so on...).
                              > > >
                              > >
                              > > yeah, but if it works for LE then what is my motivation to fix the
                              > > driver for BE operation?
                              >
                              > Well, I think that it's clear that YOUR motivation is ZERO. That's fine,
                              > that's why this is a hobby for most of us. Nobody has to do what they don't
                              > want to do.
                              >

                              Its a hobby for me too.

                              > MY motivation for making something work in both endianness is the same
                              > motivation I have to make it work on both x86 and ARM. I write better
                              > software that way. It really is amazing how many errors and assumptions
                              > creep into code when one writes (even subconciously) for a particular
                              > architecture. For example -- the ppp driver in the kernel executes floating
                              > point instructions. That's verboten in the kernel. But apparently this
                              > slipped through, because the x86 hardware does it regardless of the kernel's
                              > rules. Building the driver for ARM made that problem very apparent. There
                              > are countless other examples where architectural items such as alignment,
                              > atomicity, cache effects, and of course endianness have had latent problems
                              > that could have been readily uncovered had the code been tested on other
                              > architectures. Having ported the Via Velocity NIC driver from LE-only to
                              > support both LE and BE, I am well aware of just how awful code can be that
                              > is written with an assumption of a single architecture.
                              >
                              > I am happy that I have a device -- the NSLU2 -- that can run the same OS and
                              > software in both endianness to test my code upon -- it's a huge savings in
                              > hardware, as well as a productivity enhancement. And like I said, it also
                              > means that I don't have to be quite so ashamed of my code in peer reviews.
                              >

                              cool.

                              > [snip]
                              > > > Again, if the driver doesn't work with one endianness, then it is
                              > plainly
                              > > > _broken_. Not taking endianness (and word size, etc...) into account
                              > when
                              > > > writing a driver is a *major design flaw*.
                              > >
                              > > yes but as I'm not eager to fix the spaxx driver to work under BE
                              > > systems I don't know what I will do about it.
                              > >
                              >
                              > Don't do anything about it. Nobody is pushing you to do anything. Again,
                              > as I recall the thread, you demanded to know why people bothered with making

                              I didn't demand anything. I just asked "why bother with BE".

                              > a BE system, since it didn't seem to meet your needs. You got an answer,
                              > which you rejected. No problem, it's a hobby. Nobody is ordering you to

                              I was trying to see if there was something more than an academic
                              argument. I didn't mean to "reject" them.

                              > port anything to BE, or even use it. Likewise, we'll continue to support
                              > both - despite some folks who don't agree with that.
                              >
                              > Install SlugOS/LE, or Debian -- and enjoy it!

                              thanks,

                              --mgross

                              >
                              > >
                              > > --mgross
                              > >
                              >
                              > Mike (mwester)
                              >
                            Your message has been successfully submitted and would be delivered to recipients shortly.