Loading ...
Sorry, an error occurred while loading the content.

Hutter's prize

Expand Messages
  • Eray Ozkural
    http://www.hutter1.net/prize.htm Interesting, but don t hold your breath it seems he is just trying to make a point. :) -- Eray Ozkural (exa), PhD candidate.
    Message 1 of 29 , Aug 6, 2006
    • 0 Attachment
      http://www.hutter1.net/prize.htm

      Interesting, but don't hold your breath it seems he is
      just trying to make a point. :)

      --
      Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
      http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
      ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
      Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
    • Eray Ozkural
      hold your breath :) i saw that this is the continuation of the compression challenges by Matt Mahoney, so if you can beat PAQ8H on the wikipedia corpus you are
      Message 2 of 29 , Aug 6, 2006
      • 0 Attachment
        hold your breath :) i saw that this is the continuation of the
        compression challenges by Matt Mahoney, so if you can beat
        PAQ8H on the wikipedia corpus you are in business.

        interesting.

        best,

        --- In ai-philosophy@yahoogroups.com, "Eray Ozkural" <erayo@...> wrote:
        >
        > http://www.hutter1.net/prize.htm
        >
        > Interesting, but don't hold your breath it seems he is
        > just trying to make a point. :)
        >
        > --
        > Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent
        University, Ankara
        > http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
        > ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
        > Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
        >
      • jrstern
        I hereby submit Cygnus X-1 as the most intelligent thing in the known universe, more intelligent than Winzip, more intelligent than my trash compactor, more
        Message 3 of 29 , Aug 6, 2006
        • 0 Attachment
          I hereby submit Cygnus X-1 as the most intelligent thing in the known
          universe, more intelligent than Winzip, more intelligent than my trash
          compactor, more intelligent than a can of condensed soup, capable of
          reducing any file to one large bit, which apparently Mr. Hutter
          considers to be a demonstration of intelligence.
        • John J. Gagne
          ... LOL ;o) I second it Josh. Now if only we could have a decompressor of Cyg X-1. Now that would be better then Chicken and stars ! JJG
          Message 4 of 29 , Aug 6, 2006
          • 0 Attachment
            --- In ai-philosophy@yahoogroups.com, "jrstern" <jrstern@...> wrote:
            >
            > I hereby submit Cygnus X-1 as the most intelligent thing in the
            > known universe, more intelligent than Winzip, more intelligent than
            > my trash compactor, more intelligent than a can of condensed soup,
            > capable of reducing any file to one large bit, which apparently Mr.
            > Hutter considers to be a demonstration of intelligence.
            >

            LOL ;o) I second it Josh. Now if only we could have a decompressor of
            Cyg X-1. Now that would be better then "Chicken and stars"!

            JJG
          • rooftop8000
            ... University, Ankara ... If i understand it correctly you have to make a program P so: size(P)
            Message 5 of 29 , Aug 6, 2006
            • 0 Attachment
              --- In ai-philosophy@yahoogroups.com, "Eray Ozkural" <erayo@...> wrote:
              >
              > http://www.hutter1.net/prize.htm
              >
              > Interesting, but don't hold your breath it seems he is
              > just trying to make a point. :)
              >
              > --
              > Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent
              University, Ankara
              > http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
              > ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
              > Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
              >


              If i understand it correctly you have to make a program P so:
              size(P) < size(wikipedia-knowledge-file) = 100MB, and P has to
              generate the wiki-file.

              This doesn't proove anything except how random the wiki-file is (thus
              how much bits of information it contains).

              If you had a program equivalent to the human brain, it wouldnt win
              because size(P) would be bigger than 100 MB and no human brain can
              (after studying) generate the entire wiki-file from memory.


              from the site:
              "This compression contest is motivated by the fact that being able to
              compress well is closely related to acting intelligently. In order to
              compress data, one has to find regularities in them, which is
              intrinsically difficult (many researchers live from analyzing data and
              finding compact models)."

              You could generate all the turing machines with a size smaller than
              17'674'700 bytes and see which one generates the wiki-file. Which part
              of this is intelligent?

              Or am i missing anything? :)
            • Eray Ozkural
              ... You are not missing anything. The above would be akin to using Levin Search to find the smallest computer that does this. Basically, a universal
              Message 6 of 29 , Aug 6, 2006
              • 0 Attachment
                On 8/6/06, rooftop8000 <rooftop8000@...> wrote:
                > You could generate all the turing machines with a size smaller than
                > 17'674'700 bytes and see which one generates the wiki-file. Which part
                > of this is intelligent?
                >
                > Or am i missing anything? :)

                You are not missing anything. The above would be akin to
                using Levin Search to find the smallest computer that does
                this. Basically, a "universal compressor". Of course, you cannot
                enumerate them all, so it is practically *impossible*. Works only
                in theory.

                You are also right about the need for a specific
                compressor with a built-in knowledge base like
                the human brain, the knowledge base itself may be
                way bigger than 100MB but it can be used to compress
                the wiki file to a tiny sum perhaps.

                Best,

                --
                Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
              • KAZ
                ... From: rooftop8000 To: ai-philosophy@yahoogroups.com Sent: Sunday, August 6, 2006 11:45:34 AM Subject: [ai-philosophy] Re: Hutter s prize ... I suppose I
                Message 7 of 29 , Aug 6, 2006
                • 0 Attachment
                  ----- Original Message ----
                  From: rooftop8000
                  To: ai-philosophy@yahoogroups.com
                  Sent: Sunday, August 6, 2006 11:45:34 AM
                  Subject: [ai-philosophy] Re: Hutter's prize

                  > --- In ai-philosophy@ yahoogroups. com, "Eray Ozkural" wrote:
                  >
                  > http://www.hutter1 net/prize. htm
                  >
                  > > Interesting, but don't hold your breath it seems he is
                  > > just trying to make a point. :)

                  > If i understand it correctly you have to make a program P so:
                  > size(P) < size(wikipedia- knowledge- file) = 100MB, and P has to
                  > generate the wiki-file.


                  I suppose I should just do it, in case I could actually win the money this way, but it seems to me that the obvious solution for his prize is to write a program which takes a compressed data file equal to the URL of the wiki file, accesses the wiki file itself across the net, and makes a local copy.

                  Duh.

                  Technically, as is implied, compression is just the replacement of some information with other information which represents it. If I say "Lord of the Rings" to a fantasy fan, I have delivered the compressed equivilent of the three-book-long story and all of its history to him.

                  The phrase "a one mile by one mile grid of squares one inch on the side, alternating black and white" is a very fine, but perfect, compression of the actual thing.

                  So, really, is a URL. I don't see, at a glance, anything on the site which requires that the application at hand not access the 'net.

                  I could have his ferarri.

                  --
                  Words of the Sentient:

                  A people living under the perpetual menace of war and invasion is very easy to
                  govern. It demands no social reforms. It does not haggle over expenditures
                  on armaments and military equipment. It pays without discussion, it ruins
                  itself, and that is an excellent thing for the syndicates of financiers and
                  manufacturers for whom patriotic terrors are an abundant source of gain.
                  -- Anatole France

                  E-Mail: KazVorpal@...
                  Yahoo Messenger/AIM/AOL: KazVorpal
                  MSN Messenger: KazVorpal@...
                  ICQ: 1912557
                  http://360.yahoo.com/kazvorpal
                • William Modlin
                  ... Seems like some of you missed the requirement that the program run and reproduce the original wiki file WITHOUT USE OF ANY EXTERNAL DATA. No corpus of
                  Message 8 of 29 , Aug 6, 2006
                  • 0 Attachment
                    --- KAZ <kazvorpal@...> wrote:

                    > ----- Original Message ----
                    > From: rooftop8000
                    > To: ai-philosophy@yahoogroups.com
                    > Sent: Sunday, August 6, 2006 11:45:34 AM
                    > Subject: [ai-philosophy] Re: Hutter's prize
                    >
                    > > --- In ai-philosophy@ yahoogroups. com, "Eray
                    > Ozkural" wrote:
                    > >
                    > > http://www.hutter1 net/prize. htm
                    > >
                    > > > Interesting, but don't hold your breath it
                    > seems he is
                    > > > just trying to make a point. :)
                    >
                    > > If i understand it correctly you have to make a
                    > program P so:
                    > > size(P) < size(wikipedia- knowledge- file) =
                    > 100MB, and P has to
                    > > generate the wiki-file.
                    >
                    >
                    > I suppose I should just do it, in case I could
                    > actually win the money this way, but it seems to me
                    > that the obvious solution for his prize is to write
                    > a program which takes a compressed data file equal
                    > to the URL of the wiki file, accesses the wiki file
                    > itself across the net, and makes a local copy.
                    >
                    > Duh.
                    >
                    > Technically, as is implied, compression is just the
                    > replacement of some information with other
                    > information which represents it. If I say "Lord of
                    > the Rings" to a fantasy fan, I have delivered the
                    > compressed equivilent of the three-book-long story
                    > and all of its history to him.
                    >
                    > The phrase "a one mile by one mile grid of squares
                    > one inch on the side, alternating black and white"
                    > is a very fine, but perfect, compression of the
                    > actual thing.
                    >
                    > So, really, is a URL. I don't see, at a glance,
                    > anything on the site which requires that the
                    > application at hand not access the 'net.
                    >
                    > I could have his ferarri.
                    >
                    > --
                    > Words of the Sentient:
                    >
                    > A people living under the perpetual menace of war
                    > and invasion is very easy to
                    > govern. It demands no social reforms. It does not
                    > haggle over expenditures
                    > on armaments and military equipment. It pays
                    > without discussion, it ruins
                    > itself, and that is an excellent thing for the
                    > syndicates of financiers and
                    > manufacturers for whom patriotic terrors are an
                    > abundant source of gain.
                    > -- Anatole France
                    >
                    > E-Mail: KazVorpal@...
                    > Yahoo Messenger/AIM/AOL: KazVorpal
                    > MSN Messenger: KazVorpal@...
                    > ICQ: 1912557
                    > http://360.yahoo.com/kazvorpal
                    >
                    >
                    >
                    Seems like some of you missed the requirement that the
                    program run and reproduce the original wiki file
                    WITHOUT USE OF ANY EXTERNAL DATA. No corpus of other
                    knowledge. No web links. Just a self-contained
                    program of size less than the stated size, that
                    produces the original data.
                  • Eray Ozkural
                    ... Sure, and that is a good challenge in itself. However, I don t think that the remarks about general intelligence apply? -- Eray Ozkural (exa), PhD
                    Message 9 of 29 , Aug 6, 2006
                    • 0 Attachment
                      On 8/7/06, William Modlin <wdmodlin@...> wrote:
                      > Seems like some of you missed the requirement that the
                      > program run and reproduce the original wiki file
                      > WITHOUT USE OF ANY EXTERNAL DATA. No corpus of other
                      > knowledge. No web links. Just a self-contained
                      > program of size less than the stated size, that
                      > produces the original data.

                      Sure, and that is a good challenge in itself. However, I don't
                      think that the remarks about general intelligence apply?

                      --
                      Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                      http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                      ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                      Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                    • KAZ
                      ... From: William Modlin To: ai-philosophy@yahoogroups.com Sent: Sunday, August 6, 2006 4:59:03 PM Subject: Re: [ai-philosophy] Re: Hutter s prize ... Yeah, I
                      Message 10 of 29 , Aug 6, 2006
                      • 0 Attachment
                        ----- Original Message ----
                        From: William Modlin
                        To: ai-philosophy@yahoogroups.com
                        Sent: Sunday, August 6, 2006 4:59:03 PM
                        Subject: Re: [ai-philosophy] Re: Hutter's prize

                        > Seems like some of you missed the requirement that the
                        > program run and reproduce the original wiki file
                        > WITHOUT USE OF ANY EXTERNAL DATA. No corpus of other
                        > knowledge. No web links. Just a self-contained
                        > program of size less than the stated size, that
                        > produces the original data.

                        Yeah, I saw the lack of network connection caveat, after I wrote that post.

                        Ah well.
                      • William Modlin
                        ... Well. He does seem to go a bit overboard. I don t think that non-lossy compression per se is a step toward AI. I consider it likely that a winning entry
                        Message 11 of 29 , Aug 6, 2006
                        • 0 Attachment
                          --- Eray Ozkural <erayo@...> wrote:

                          > On 8/7/06, William Modlin <wdmodlin@...>
                          > wrote:
                          > > Seems like some of you missed the requirement that
                          > the
                          > > program run and reproduce the original wiki file
                          > > WITHOUT USE OF ANY EXTERNAL DATA. No corpus of
                          > other
                          > > knowledge. No web links. Just a self-contained
                          > > program of size less than the stated size, that
                          > > produces the original data.
                          >
                          > Sure, and that is a good challenge in itself.
                          > However, I don't
                          > think that the remarks about general intelligence
                          > apply?
                          >
                          > --
                          > Eray Ozkural (exa), PhD candidate. Comp. Sci.

                          Well. He does seem to go a bit overboard. I don't
                          think that non-lossy compression per se is a step
                          toward AI. I consider it likely that a winning entry
                          in this contest might have no relevance to
                          intelligence at all, except perhaps as evidence of the
                          cleverness of the programmer.

                          However, I sympathise with his feeling that there is
                          indeed a deep connection between compression
                          algorithms and intelligence, in that both depend
                          centrally on discovery of useful redundancy or
                          patterning in the data available for examination.

                          To me, the important difference is that for non-lossy
                          compression one must find patterns that can be used to
                          regenerate the surface data exactly, while
                          intelligence is more generally concerned with finding
                          underlying patterns that may be modified by
                          interaction with less-patterned factors to produce the
                          surface data.

                          The deeper patterns that intelligence must discover
                          often account for very little of the variance of the
                          data: most of the information in a Shannon sense
                          results from superficial influences that aren't
                          important or useful to understanding, yet must be
                          dealt with for literal reproduction. If ones goal is
                          to reproduce the data exactly, then most of the work
                          is directed toward capturing and modelling those
                          superfical processes, not the processes that really
                          matter.

                          I have no confidence that those words will properly
                          convey the notion I'm trying to get across, but at the
                          moment I am too tired to try for a better exposition.
                          I will try to clarify later if there is any interest
                          expressed.

                          Bill Modlin
                        • KAZ
                          ... From: William Modlin To: ai-philosophy@yahoogroups.com Sent: Sunday, August 6, 2006 11:53:46 PM Subject: Re: [ai-philosophy] Re: Hutter s prize ... Well,
                          Message 12 of 29 , Aug 6, 2006
                          • 0 Attachment
                            ----- Original Message ----
                            From: William Modlin
                            To: ai-philosophy@yahoogroups.com
                            Sent: Sunday, August 6, 2006 11:53:46 PM
                            Subject: Re: [ai-philosophy] Re: Hutter's prize

                            > Well. He does seem to go a bit overboard. I don't
                            > think that non-lossy compression per se is a step
                            > toward AI. I consider it likely that a winning entry
                            > in this contest might have no relevance to
                            > intelligence at all, except perhaps as evidence of the
                            > cleverness of the programmer.

                            > However, I sympathise with his feeling that there is
                            > indeed a deep connection between compression
                            > algorithms and intelligence, in that both depend
                            > centrally on discovery of useful redundancy or
                            > patterning in the data available for examination.

                            Well, you could certainly develop superior compression algorithms with something akin to artificial intelligence, though.

                            An AI, with a lossless /intellect/, instead of a sloppy organic one, and with the speed of a machine, could examine data, especially something full of unusual patterns like wikipedia, and generate a custom compression scheme for each part, maximizing compression far more than any generic algorithm could do.
                          • Eray Ozkural
                            ... Precisely. Though, don t overestimate the performance of current machines. They are pretty slow for the kind of compression you talk above. I am working
                            Message 13 of 29 , Aug 7, 2006
                            • 0 Attachment
                              On 8/7/06, KAZ <kazvorpal@...> wrote:
                              > An AI, with a lossless /intellect/, instead of a sloppy organic one, and with the speed of a machine, could examine data, especially something full of unusual patterns like wikipedia, and generate a custom compression scheme for each part, maximizing compression far more than any generic algorithm could do.

                              Precisely.

                              Though, don't overestimate the performance of current machines.
                              They are pretty slow for the kind of compression you talk above. I
                              am working with PAQ8F and it is one of the slowest programs that
                              you can run on small bitstrings. Ouch.

                              --
                              Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                              http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                              ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                              Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                            • Eray Ozkural
                              ... Well, yes, but Shannon information also matters for lossy compression of the data in the way of deep understanding of the data. It seems rate-distortion
                              Message 14 of 29 , Aug 7, 2006
                              • 0 Attachment
                                On 8/7/06, William Modlin <wdmodlin@...> wrote:
                                > The deeper patterns that intelligence must discover
                                > often account for very little of the variance of the
                                > data: most of the information in a Shannon sense
                                > results from superficial influences that aren't
                                > important or useful to understanding, yet must be
                                > dealt with for literal reproduction. If ones goal is
                                > to reproduce the data exactly, then most of the work
                                > is directed toward capturing and modelling those
                                > superfical processes, not the processes that really
                                > matter.

                                Well, yes, but Shannon information also matters for
                                lossy compression of the data in the way of "deep
                                understanding" of the data. It seems rate-distortion
                                theory has something to do with this.


                                Best,

                                --
                                Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                                http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                                ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                                Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                              • William Modlin
                                ... It s all Shannon information , at all levels. The point I was trying to make was that a large part of the information content of most data we have to
                                Message 15 of 29 , Aug 7, 2006
                                • 0 Attachment
                                  --- Eray Ozkural <erayo@...> wrote:

                                  > On 8/7/06, William Modlin <wdmodlin@...>
                                  > wrote:
                                  > > The deeper patterns that intelligence must
                                  > discover
                                  > > often account for very little of the variance of
                                  > the
                                  > > data: most of the information in a Shannon sense
                                  > > results from superficial influences that aren't
                                  > > important or useful to understanding, yet must be
                                  > > dealt with for literal reproduction. If ones goal
                                  > is
                                  > > to reproduce the data exactly, then most of the
                                  > work
                                  > > is directed toward capturing and modelling those
                                  > > superfical processes, not the processes that
                                  > really
                                  > > matter.
                                  >
                                  > Well, yes, but Shannon information also matters for
                                  > lossy compression of the data in the way of "deep
                                  > understanding" of the data. It seems rate-distortion
                                  > theory has something to do with this.
                                  >

                                  It's all "Shannon information", at all levels. The
                                  point I was trying to make was that a large part of
                                  the information content of most data we have to deal
                                  with, and of raw sensory data in particular, is
                                  generated by transient and incidental processes with
                                  no enduring predictive value. Intelligence is about
                                  cutting through the surface noise to find the
                                  persistent regularities. These generally are not the
                                  regularities that contribute the most to compression
                                  of any particular bounded sample of the data. Focus
                                  on literal compression forces one to concentrate on
                                  finding and taking advantage of all those transient
                                  regularities, which can distract from the search for
                                  stuff that is useful in the longer term.

                                  Both intelligence and compression have to detect
                                  recurrences. For best compression, one must take
                                  advantage of all recurrences to improve the encoding:
                                  a bit saved is a bit saved, even if the "regularity"
                                  which allows the saving is accidental, local, and
                                  essentially meaningless. Intelligence isn't concerned
                                  with actual compression ratios per se, and
                                  concentrates its resources on longer term regularities
                                  that have a higher probability of being useful in
                                  future situations, even if this emphasis does not
                                  produce the highest compression in the short term.

                                  Of course, regularities don't come to us labelled as
                                  long-term or short-term. The first time we encounter
                                  any particular regularity it appears as merely a
                                  second ocurrence of some combination encountered
                                  before, which could be a meaningless coincidence. All
                                  important discoveries begin this way, as observations
                                  indistinguishable from all the other coincidences that
                                  will eventually turn out to be of no consequence at
                                  all. So regardless of whether our goal is compression
                                  or understanding, we begin with mechanisms or
                                  algorithms for noting all coincidences. The subtle
                                  difference is that if our goal is best compression we
                                  must devote resources to taking advantage of every
                                  coincidence that allows a saving, while if our goal is
                                  undertanding we can afford to forget about things that
                                  don't pan out, freeing resources for working on more
                                  promising leads.

                                  It is easy to be seduced by the elegance of a global
                                  view... certainly a perfect compressor would
                                  understand everything, so in the limit compression and
                                  understanding converge. But a practical intelligence
                                  works with constrained resources and makes tradeoffs
                                  which will be different from those of a practical
                                  compressor.

                                  Bill Modlin
                                • Eray Ozkural
                                  You have the right ideas but all of that stuff is covered in information theory: http://arxiv.org/abs/cond-mat/9902341
                                  Message 16 of 29 , Aug 7, 2006
                                  • 0 Attachment
                                    You have the right ideas but all of that stuff
                                    is covered in information theory:

                                    http://arxiv.org/abs/cond-mat/9902341
                                    http://www.princeton.edu/~wbialek/wbialek.html

                                    --
                                    Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                                    http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                                    ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                                    Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                                  • William Modlin
                                    ... Of course it is. I was familiar with both those links. But there still seems to be something missing in the translation of the formal descriptions to
                                    Message 17 of 29 , Aug 7, 2006
                                    • 0 Attachment
                                      --- Eray Ozkural <erayo@...> wrote:

                                      > You have the right ideas but all of that stuff
                                      > is covered in information theory:
                                      >
                                      > http://arxiv.org/abs/cond-mat/9902341
                                      > http://www.princeton.edu/~wbialek/wbialek.html

                                      Of course it is. I was familiar with both those
                                      links. But there still seems to be something missing
                                      in the translation of the formal descriptions to
                                      practice, and it shows up in casual errors in the
                                      discussions here, at least to my mind.

                                      As a most recent case in point, Kaz said:

                                      "An AI, with a lossless /intellect/, instead of a
                                      sloppy organic one, and with the speed of a machine,
                                      could examine data, especially something full of
                                      unusual patterns like wikipedia, and generate a custom
                                      compression scheme for each part, maximizing
                                      compression far more than any generic algorithm could
                                      do."

                                      And you responded:

                                      "Precisely."

                                      ... followed by some cautions about the practical
                                      slowness of current machines for doing that sort of
                                      detailed analysis.

                                      Which suggests that you see nothing wrong with Kaz'
                                      premise that a lossless intellect is desirable, and
                                      that generating custom compression schemes for each
                                      part is somehow relevant to intelligence.
                                    • KAZ
                                      ... From: Eray Ozkural To: ai-philosophy@yahoogroups.com Sent: Monday, August 7, 2006 5:58:50 AM Subject: Re: [ai-philosophy] Re: Hutter s prize ... They also
                                      Message 18 of 29 , Aug 7, 2006
                                      • 0 Attachment
                                        ----- Original Message ----
                                        From: Eray Ozkural
                                        To: ai-philosophy@yahoogroups.com
                                        Sent: Monday, August 7, 2006 5:58:50 AM
                                        Subject: Re: [ai-philosophy] Re: Hutter's prize

                                        On 8/7/06, KAZ wrote:
                                        > > An AI, with a lossless /intellect/, instead of a sloppy organic one,
                                        > > and with the speed of a machine, could examine data, especially

                                        > Though, don't overestimate the performance of current machines.
                                        > They are pretty slow for the kind of compression you talk above. I

                                        They also aren't AI. Well, in a sense they are, but not in the conventional way we use when discussing AI on here.


                                        If someone was actually able to program a human-quality AI, /that/ could probably handle complex custom compression.

                                        --

                                        Words of the Sentient:

                                        Advertising is 85 percent confusion and 15 percent commission. --Fred Allen

                                        E-Mail: KazVorpal@...
                                        Yahoo Messenger/AIM/AOL: KazVorpal
                                        MSN Messenger: KazVorpal@...
                                        ICQ: 1912557
                                        http://360.yahoo.com/kazvorpal
                                      • Eray Ozkural
                                        ... What I thought was... the lossless compression per se may not be the ultimate test of intelligence. Yet, being able to solve any optimization problem
                                        Message 19 of 29 , Aug 7, 2006
                                        • 0 Attachment
                                          On 8/7/06, William Modlin <wdmodlin@...> wrote:
                                          > Which suggests that you see nothing wrong with Kaz'
                                          > premise that a lossless intellect is desirable, and
                                          > that generating custom compression schemes for each
                                          > part is somehow relevant to intelligence.

                                          What I thought was... the lossless compression per se may
                                          not be the ultimate test of intelligence. Yet, being able to
                                          solve any optimization problem intelligently, including program
                                          size optimization, certainly is.

                                          It might also be the case that an effective universal compressor
                                          entails strong AI, but I wouldn't try to show that by compressing
                                          the wikipedia corpus. It seems to be the wrong problem definition
                                          right now.

                                          Best,

                                          --
                                          Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                                          http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                                          ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                                          Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                                        • KAZ
                                          ... From: William Modlin To: ai-philosophy@yahoogroups.com Sent: Monday, August 7, 2006 11:22:11 AM Subject: Re: [ai-philosophy] Re: Hutter s prize ... Are you
                                          Message 20 of 29 , Aug 7, 2006
                                          • 0 Attachment
                                            ----- Original Message ----
                                            From: William Modlin
                                            To: ai-philosophy@yahoogroups.com
                                            Sent: Monday, August 7, 2006 11:22:11 AM
                                            Subject: Re: [ai-philosophy] Re: Hutter's prize

                                            > Which suggests that you see nothing wrong with Kaz'
                                            > premise that a lossless intellect is desirable, and
                                            > that generating custom compression schemes for each
                                            > part is somehow relevant to intelligence.


                                            Are you disagreeing with the premise of lossless intellect as being desirable?

                                            And you have the relationship backward; intelligence is a useful tool for creating custom compression schemes. This doesn't necessarily require custom compression schemes to be relevant to intelligence.

                                            It certainly could be a useful goal for producing AI, sort of like the premise that the moon shot produced computers, teflon, microwave ovens, et cetera, as by-products.

                                            (setting aside the fallacy of that myth)

                                            --
                                            Words of the Sentient:

                                            Now and then an innocent man is sent to the legislature. -- Kim Hubbard

                                            E-Mail: KazVorpal@...
                                            Yahoo Messenger/AIM/AOL: KazVorpal
                                            MSN Messenger: KazVorpal@...
                                            ICQ: 1912557
                                            http://360.yahoo.com/kazvorpal
                                          • William Modlin
                                            ... Agreed. :)
                                            Message 21 of 29 , Aug 7, 2006
                                            • 0 Attachment
                                              --- Eray Ozkural <erayo@...> wrote:

                                              > It might also be the case that an effective
                                              > universal compressor
                                              > entails strong AI, but I wouldn't try to show that
                                              > by compressing
                                              > the wikipedia corpus. It seems to be the wrong
                                              > problem definition
                                              > right now.

                                              Agreed. :)
                                            • KAZ
                                              ... From: William Modlin To: ai-philosophy@yahoogroups.com Sent: Monday, August 7, 2006 12:43:48 PM Subject: Re: [ai-philosophy] Re: Hutter s prize ... I m
                                              Message 22 of 29 , Aug 7, 2006
                                              • 0 Attachment
                                                ----- Original Message ----
                                                From: William Modlin
                                                To: ai-philosophy@yahoogroups.com
                                                Sent: Monday, August 7, 2006 12:43:48 PM
                                                Subject: Re: [ai-philosophy] Re: Hutter's prize

                                                --- Eray Ozkural wrote:

                                                > It might also be the case that an effective
                                                > universal compressor
                                                > entails strong AI, but I wouldn't try to show that
                                                > by compressing
                                                > the wikipedia corpus. It seems to be the wrong
                                                > problem definition
                                                > right now.

                                                > Agreed.

                                                I'm guessing that he views this as an arbitrary goal, the solution of which would require a lot of advances which will be useful to AI development in general.

                                                Like the moonshot, except without coercive government turning the project into an actual undermining of technology and the more abstract goals at hand the way NASA does.


                                                --
                                                Words of the Sentient:

                                                Free trade is not a principle, it is an expedient. -- Benjamin Disraeli

                                                E-Mail: KazVorpal@...
                                                Yahoo Messenger/AIM/AOL: KazVorpal
                                                MSN Messenger: KazVorpal@...
                                                ICQ: 1912557
                                                http://360.yahoo.com/kazvorpal
                                              • Eray Ozkural
                                                ... PAQ8 is actually an AI, because it can predict time series data. However, it does not do this in the way that a strong AI would do. Has some fixed models
                                                Message 23 of 29 , Aug 7, 2006
                                                • 0 Attachment
                                                  On 8/7/06, KAZ <kazvorpal@...> wrote:
                                                  > ----- Original Message ----
                                                  > From: Eray Ozkural
                                                  > To: ai-philosophy@yahoogroups.com
                                                  > Sent: Monday, August 7, 2006 5:58:50 AM
                                                  > Subject: Re: [ai-philosophy] Re: Hutter's prize
                                                  >
                                                  > On 8/7/06, KAZ wrote:
                                                  > > > An AI, with a lossless /intellect/, instead of a sloppy organic one,
                                                  > > > and with the speed of a machine, could examine data, especially
                                                  >
                                                  > > Though, don't overestimate the performance of current machines.
                                                  > > They are pretty slow for the kind of compression you talk above. I
                                                  >
                                                  > They also aren't AI. Well, in a sense they are, but not in the conventional way we use when discussing AI on here.
                                                  >
                                                  >
                                                  > If someone was actually able to program a human-quality AI, /that/ could probably handle complex custom compression.

                                                  PAQ8 is actually an AI, because it can predict time series data.

                                                  However, it does not do this in the way that a strong AI would do.

                                                  Has some fixed models etc. and estimators, and these are mixed
                                                  using neural nets. It's fairly standard classified ensemble.

                                                  Best,

                                                  --
                                                  Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                                                  http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                                                  ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                                                  Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                                                • John J. Gagne
                                                  ... What are you using PAQ8 compression for? IOW, what kind of work are you doing with it? JJG
                                                  Message 24 of 29 , Aug 7, 2006
                                                  • 0 Attachment
                                                    --- In ai-philosophy@yahoogroups.com, "Eray Ozkural" <erayo@...> wrote:
                                                    >
                                                    > PAQ8 is actually an AI, because it can predict time series data.
                                                    >
                                                    > However, it does not do this in the way that a strong AI would do.
                                                    >
                                                    > Has some fixed models etc. and estimators, and these are mixed
                                                    > using neural nets. It's fairly standard classified ensemble.
                                                    >

                                                    What are you using PAQ8 compression for? IOW, what kind of work are you
                                                    doing with it?

                                                    JJG
                                                  • Eray Ozkural
                                                    ... I meant it is a classifier ensemble of some sort, not classified. I have been trying out my micro-synthetic algorithm but I ran into some troubles with the
                                                    Message 25 of 29 , Aug 7, 2006
                                                    • 0 Attachment
                                                      On 8/8/06, John J. Gagne <fitness4eb@...> wrote:
                                                      > --- In ai-philosophy@yahoogroups.com, "Eray Ozkural" <erayo@...> wrote:
                                                      > >
                                                      > > PAQ8 is actually an AI, because it can predict time series data.
                                                      > >
                                                      > > However, it does not do this in the way that a strong AI would do.
                                                      > >
                                                      > > Has some fixed models etc. and estimators, and these are mixed
                                                      > > using neural nets. It's fairly standard classified ensemble.
                                                      > >
                                                      >
                                                      > What are you using PAQ8 compression for? IOW, what kind of work are you
                                                      > doing with it?

                                                      I meant it is a classifier ensemble of some sort, not classified.

                                                      I have been trying out my micro-synthetic algorithm but I ran into
                                                      some troubles with the efficiency of PAQ8, I might actually have to
                                                      go back to using bzip2 or maybe I should port a PPM implementation
                                                      from Java to C++ :/

                                                      The algorithm finds commonality among given bitstrings. I had posted
                                                      a successful pattern that the program had found. I was hoping to find
                                                      better patterns with paq but it didn't happen for some reason and I
                                                      missed the submission deadline :(

                                                      Best,

                                                      --
                                                      Eray Ozkural (exa), PhD candidate. Comp. Sci. Dept., Bilkent University, Ankara
                                                      http://www.cs.bilkent.edu.tr/~erayo Malfunct: http://www.malfunct.com
                                                      ai-philosophy: http://groups.yahoo.com/group/ai-philosophy
                                                      Pardus: www.uludag.org.tr KDE Project: http://www.kde.org
                                                    • William Modlin
                                                      ... Yes. In the first place, it is impossible. An instantiated intellect is a bounded system with finite resources processing ongoing input from a much larger
                                                      Message 26 of 29 , Aug 8, 2006
                                                      • 0 Attachment
                                                        --- KAZ <kazvorpal@...> wrote:

                                                        > Are you disagreeing with the premise of lossless
                                                        > intellect as being desirable?

                                                        Yes.

                                                        In the first place, it is impossible. An instantiated
                                                        intellect is a bounded system with finite resources
                                                        processing ongoing input from a much larger external
                                                        universe... sooner or later it has to run out of room
                                                        and start losing something.

                                                        In the second place, it is undesirable. Most of the
                                                        information in available data is itself unpredictable
                                                        and has no value for prediction of the future. It is
                                                        a waste of finite resources to try to avoid losing
                                                        some of it.

                                                        One can quibble over the nature and purpose of
                                                        intellect, but clearly predicting the future is a
                                                        central part of it. So it is desirable for an
                                                        intellect to concentrate its efforts on information
                                                        which is useful for that purpose, and for it to
                                                        discard information which is not useful for that
                                                        purpose.

                                                        Of course, information doesn't come with labels
                                                        telling us what is useful and what is not, so making
                                                        the distinction is a heuristic matter and we have to
                                                        retain for a while a lot of information that may turn
                                                        out to be useless eventually and we may discard some
                                                        things that would have been important if we only
                                                        knew... but a good intellect is one that has good
                                                        heuristics for deciding what to throw away. Not one
                                                        that tries to retain everything without loss.

                                                        Processing a tiny toy chunk of data like the wiki file
                                                        in exquisite detail to wring the last possible bit out
                                                        of a compressed representation is a play-puzzle, not a
                                                        step toward a useful intelligence. A good AI might be
                                                        good at solving play-puzzles like this, but then again
                                                        it might not be, and in any case that's not what we
                                                        should be working toward.

                                                        Intelligence is about summarizing, seeing the big
                                                        picture, focusing on what matters and avoiding
                                                        distraction by irrelevant detail. It is about
                                                        avoiding combinatorial explosions of possibilities,
                                                        about rapidly and ruthlessly collapsing huge volumes
                                                        of detail into manageable models. It isn't about
                                                        literal non-lossy compression.

                                                        Bill Modlin
                                                      • KAZ
                                                        ... From: William Modlin To: ai-philosophy@yahoogroups.com Sent: Tuesday, August 8, 2006 2:40:43 PM Subject: Re: [ai-philosophy] Re: Hutter s prize ... Not
                                                        Message 27 of 29 , Aug 8, 2006
                                                        • 0 Attachment
                                                          ----- Original Message ----
                                                          From: William Modlin
                                                          To: ai-philosophy@yahoogroups.com
                                                          Sent: Tuesday, August 8, 2006 2:40:43 PM
                                                          Subject: Re: [ai-philosophy] Re: Hutter's prize

                                                          > In the first place, it is impossible. An instantiated
                                                          > intellect is a bounded system with finite resources
                                                          > processing ongoing input from a much larger external
                                                          > universe... sooner or later it has to run out of room
                                                          > and start losing something.

                                                          Not necessarily. Well, if "sooner or later" means infinity, perhaps.

                                                          What is required is more creativity than to just store everything as raw sequential data.

                                                          Imagine how much bigger that wikifile would be if it stored data in such a format. All changes saved as snapshots of the pages affected.

                                                          The data can be stored, instead, as relationships and differences, while still being lossless.

                                                          > In the second place, it is undesirable. Most of the
                                                          > information in available data is itself unpredictable
                                                          > and has no value for prediction of the future. It is
                                                          > a waste of finite resources to try to avoid losing
                                                          > some of it.

                                                          This really sounds like the workup for sour grapes about how lame-brained humans are.

                                                          As I said, no redundancy need be stored for losslessness. When something's almost the same as something else, you store the differences, with indexing to allow a on-the-fly reconstruction of the redundant parts.

                                                          > One can quibble over the nature and purpose of
                                                          > intellect, but clearly predicting the future is a
                                                          > central part of it.

                                                          Not really. What is clear is that predicting the future is a standard symptom, and that it seems useful to us, in this temporal environment.

                                                          Not that this is the "purpose" of intellect, nor that it's central to its nature.

                                                          > So it is desirable for an
                                                          > intellect to concentrate its efforts on information
                                                          > which is useful for that purpose, and for it to
                                                          > discard information which is not useful for that
                                                          > purpose.

                                                          This does not follow from the above, even if I stipulate the above as true.

                                                          All you would have established is that prediction is useful and /part/ of the definition of intellect, not that it's such an exclusive aspect that it therefore gains exclusive claim over the design and function of the thinker.

                                                          > Of course, information doesn't come with labels
                                                          > telling us what is useful and what is not, so making
                                                          > the distinction is a heuristic matter and we have to
                                                          > retain for a while a lot of information that may turn
                                                          > out to be useless eventually and we may discard some
                                                          > things that would have been important if we only
                                                          > knew... but a good intellect is one that has good
                                                          > heuristics for deciding what to throw away. Not one
                                                          > that tries to retain everything without loss.

                                                          Or else you're mistaken about the need to discard, which you haven't actually proven or justified.

                                                          > Processing a tiny toy chunk of data like the wiki file
                                                          > in exquisite detail to wring the last possible bit out
                                                          > of a compressed representation is a play-puzzle, not a

                                                          Yes, but it represents thousands of times more data than you may realize, because IT is a lossless repository of every change that ever was made to it. Every time someone added something which was removed later, or which was then displaced by something else, the previous states were all saved. But it's not saved as thousands of copies of each wiki page, one for every single change. It's saved, losslessly, as differentials.

                                                          > Intelligence is about summarizing, seeing the big
                                                          > picture, focusing on what matters and avoiding
                                                          > distraction by irrelevant detail.

                                                          The only way in which you can definitively say this is to mean it as a useless truism.

                                                          In reality, intelligence may be able to function just fine while being conscious of all the allegedly irrelevent detail.

                                                          Your analysis of this reminds me of socialist governments talking about how economics must center around conservation and rationing...it's true that /socialism/ can only produce a system of rationing and conservation, and yet with a higher technology called a free market one can produce a system centered on using whatever is desired, while still ending up more efficient in the long run.

                                                          We're, regarding computation, still low-tech, having to worry about conservation and resources. But this doesn't mean that more advanced intellects could never be more abundant in their self-management.

                                                          --
                                                          Words of the Sentient:

                                                          My great objection to this government is, that it does not leave us the means
                                                          of defending our rights, or waging war against tyrants. -- Patrick Henry

                                                          E-Mail: KazVorpal@...
                                                          Yahoo Messenger/AIM/AOL: KazVorpal
                                                          MSN Messenger: KazVorpal@...
                                                          ICQ: 1912557
                                                          http://360.yahoo.com/kazvorpal
                                                        • boris
                                                          Only lossless compression is objectively measurable, lossy compression depends on subjective tollerable loss limit. In fact, there s no objective distinction
                                                          Message 28 of 29 , Aug 8, 2006
                                                          • 0 Attachment
                                                            Only lossless compression is objectively measurable, lossy compression depends on subjective "tollerable" loss limit. In fact, there's no objective distinction between them: lossy compression is a partial or selective lossless compression, the trick is to quantify the loss vs preserved info. This differentiation must be done on the finest economical level of resolution: by evaluating individual patterns & their variables, formed by lossless compression, & "losing" those with below-average compression ratio.
                                                            This is why existing data compression algorithms seem to be irrelevant for AI: they're not pattern-selective. 
                                                          • William Modlin
                                                            ... Of course one stores relationships and differences . That s what it MEANS to compress data: discover redundancies (repetitious or predictably changing
                                                            Message 29 of 29 , Aug 9, 2006
                                                            • 0 Attachment
                                                              --- KAZ <kazvorpal@...> wrote:

                                                              > ----- Original Message ----
                                                              > From: William Modlin
                                                              > To: ai-philosophy@yahoogroups.com
                                                              > Sent: Tuesday, August 8, 2006 2:40:43 PM
                                                              > Subject: Re: [ai-philosophy] Re: Hutter's prize
                                                              >
                                                              > > In the first place, it is impossible.
                                                              > > An instantiated intellect is a bounded
                                                              > > system with finite resources processing
                                                              > > ongoing input from a much larger external
                                                              > > universe... sooner or later it has to run out
                                                              > > of room and start losing something.
                                                              >
                                                              > Not necessarily. Well, if "sooner or later" means
                                                              > infinity, perhaps.
                                                              >
                                                              > What is required is more creativity than to just
                                                              > store everything as raw sequential data.
                                                              >
                                                              > Imagine how much bigger that wikifile would be if it
                                                              > stored data in such a format. All changes saved as
                                                              > snapshots of the pages affected.
                                                              >
                                                              > The data can be stored, instead, as relationships
                                                              > and differences, while still being lossless.

                                                              Of course one "stores relationships and differences".
                                                              That's what it MEANS to compress data: discover
                                                              redundancies (repetitious or predictably changing
                                                              data) and encode the data in such a way that the
                                                              redundant aspects are noted economically, taking less
                                                              space than the unpredictable changes or differences.

                                                              But you seem to miss the point that there is a limit
                                                              to how much this can save you, an irreducable minimum
                                                              amount of space that any given string of bits takes to
                                                              store, no matter how cleverly one encodes it, no
                                                              matter how "creative" one may be.

                                                              Incoming real-world sensory data has an irreducible
                                                              minimum compressed bandwidth. So if we have finite
                                                              storage and continuous input, we will run out of space
                                                              in finite time. While we don't have good numbers for
                                                              the minimal compressed bandwidth for raw human sensory
                                                              data, it is probably large enough that any currently
                                                              practical physical storage mechanism would run out of
                                                              space in weeks or months. Hardly an infinite time.

                                                              You seem to imagine some magical compression approach
                                                              that can keep getting better, so that you can cram
                                                              arbitrary amounts of input into a finite space. The
                                                              only way that could be possible is if the input were
                                                              discoverably entirely algorithmic and thus completely
                                                              predictable. We do not appear to live in such a
                                                              universe.

                                                              > > In the second place, it is undesirable. Most of
                                                              > > the information in available data is itself
                                                              > > unpredictable and has no value for prediction of
                                                              > > the future. It is a waste of finite resources to
                                                              > > try to avoid losing some of it.
                                                              >
                                                              > This really sounds like the workup for sour grapes
                                                              > about how lame-brained humans are.

                                                              Hmm. Does arguing about things one does not
                                                              understand qualify as 'lame-brained'? :)

                                                              > As I said, no redundancy need be stored for
                                                              > losslessness. When something's almost the same as
                                                              > something else, you store the differences, with
                                                              > indexing to allow a on-the-fly reconstruction of the
                                                              > redundant parts.

                                                              Right. And as I said, this just reduces the number of
                                                              bits per second that you have to store by some ratio
                                                              characteristic of the sort of data you are trying to
                                                              compress. It can't magically reduce the flow to zero.
                                                              Continuous input overflows finite storage. Period.

                                                              > > One can quibble over the nature and purpose of
                                                              > > intellect, but clearly predicting the future is a
                                                              > > central part of it.
                                                              >
                                                              > Not really. What is clear is that predicting the
                                                              > future is a standard symptom, and that it seems
                                                              > useful to us, in this temporal environment.
                                                              >
                                                              > Not that this is the "purpose" of intellect, nor
                                                              > that it's central to its nature.

                                                              Well. Hmm. I'm a bit taken aback by the gulf between
                                                              us that seems to be revealed by those words. I can't
                                                              imagine what sort of purpose you have in mind, if it
                                                              is not to be tied to prediction. And the remark about
                                                              "this temporal environment"? What other sort of
                                                              environment is there? Even if we could switch to an
                                                              extra-dimensional view in which world lines are
                                                              static, we'd still be left with a search for
                                                              relationships among the distributed elements of
                                                              information. We would still be trying to predict some
                                                              elements as functions of others, even if you drop the
                                                              temporal designation of the separation between them.

                                                              In our usual mundane view, intelligence evolved
                                                              because its predictions allowed organisms more time to
                                                              react appropriately to a changing environment, thus
                                                              enhancing survivability. That's at a resonable basis
                                                              for saying that the "purpose' of intelligence is
                                                              prediction. What sort of purpose do you propose?

                                                              As to the "centrality' issue, what else is there?

                                                              Given some data, some collection of bits or events or
                                                              whatever... some aspects of that collection may be
                                                              discovered to be expressable as functionally related
                                                              to other aspects. That allows for a more economical
                                                              encoding of the combination and equivalently for
                                                              prediction of some aspects given information about the
                                                              other related aspects. That's what intelligence is
                                                              about: discovering usable relationships (similarities,
                                                              differences, patterns, etc) among elements of
                                                              experience.

                                                              The elements or aspects which are not useful for
                                                              prediction (or compression, same thing) are those for
                                                              which we can discover no relationship whatever to
                                                              anything else... they just are what they are, and
                                                              knowing about them gives us no information about
                                                              anything else. They have no implications, no
                                                              consequences, they just sit there taking up space. So
                                                              eventually, when we start running out of room and
                                                              still have not discovered any use for these things, we
                                                              throw them away and re-use the space for something new
                                                              that might turn out to be useful. Voila. Lossy
                                                              compression. end of story. :)
                                                            Your message has been successfully submitted and would be delivered to recipients shortly.