Loading ...
Sorry, an error occurred while loading the content.
 

Photos

Expand Messages
  • Kenneth Stanley
    Happy New Year everyone! I can t remember what the photos of layers and spirals are from in the photos section. Can someone remind me? thanks, ken
    Message 1 of 21 , Jan 3, 2004
      Happy New Year everyone! I can't remember what the photos
      of "layers" and "spirals" are from in the photos section. Can
      someone remind me?

      thanks,
      ken
    • Derek James
      ... Happy New Year to you too. :) ... Ah, those are mine and Philip s. They re samples of the two different graphical layouts of networks that the
      Message 2 of 21 , Jan 4, 2004
        --- Kenneth Stanley <kstanley@...> wrote:
        > Happy New Year everyone!

        Happy New Year to you too. :)

        > I can't remember what the
        > photos
        > of "layers" and "spirals" are from in the photos
        > section. Can
        > someone remind me?

        Ah, those are mine and Philip's. They're samples of
        the two different graphical layouts of networks that
        the implementation we're working on produces.

        They're screen shots of sample output. The actual
        graphical output for everything in our implementation
        (networks, speciation statistics, fitness graphs) are
        all done in SVG, which is scalable. I've uploaded a
        couple more examples in the "Files" section, in SVG
        format. SVG is XML-based, so we felt it would be a
        more efficient way of storing graphical experimental
        data. You can view the .svg files with a browser,
        with the SVG plug-in:

        http://www.adobe.com/svg/viewer/install/main.html

        Once the plug-in is installed in your browser, just
        double-click on the .svg file you want to view. You
        can then right click on the image and select to zoom
        in or out.

        We don't know if it's practical, or even really useful
        yet, but we wanted to be able to store an image of
        each network which clearly shows all of the
        connectivity and the value of the weights on every
        connection.

        If you have any questions, let me know.

        Thanks,
        Derek

        __________________________________
        Do you Yahoo!?
        Protect your identity with Yahoo! Mail AddressGuard
        http://antispam.yahoo.com/whatsnewfree
      • Kenneth Stanley
        Hey Derek I think it s a pretty cool way to display networks. Have you thought of having the line thickness proportional to the weight magnitude of
        Message 3 of 21 , Jan 4, 2004
          Hey Derek I think it's a pretty cool way to display networks. Have
          you thought of having the line thickness proportional to the weight
          magnitude of connections (if svg allows that)?

          --- In neat@yahoogroups.com, Derek James <blue5432@y...> wrote:
          > --- Kenneth Stanley <kstanley@c...> wrote:
          > > Happy New Year everyone!
          >
          > Happy New Year to you too. :)
          >
          > > I can't remember what the
          > > photos
          > > of "layers" and "spirals" are from in the photos
          > > section. Can
          > > someone remind me?
          >
          > Ah, those are mine and Philip's. They're samples of
          > the two different graphical layouts of networks that
          > the implementation we're working on produces.
          >
          > They're screen shots of sample output. The actual
          > graphical output for everything in our implementation
          > (networks, speciation statistics, fitness graphs) are
          > all done in SVG, which is scalable. I've uploaded a
          > couple more examples in the "Files" section, in SVG
          > format. SVG is XML-based, so we felt it would be a
          > more efficient way of storing graphical experimental
          > data. You can view the .svg files with a browser,
          > with the SVG plug-in:
          >
          > http://www.adobe.com/svg/viewer/install/main.html
          >
          > Once the plug-in is installed in your browser, just
          > double-click on the .svg file you want to view. You
          > can then right click on the image and select to zoom
          > in or out.
          >
          > We don't know if it's practical, or even really useful
          > yet, but we wanted to be able to store an image of
          > each network which clearly shows all of the
          > connectivity and the value of the weights on every
          > connection.
          >
          > If you have any questions, let me know.
          >
          > Thanks,
          > Derek
          >
          > __________________________________
          > Do you Yahoo!?
          > Protect your identity with Yahoo! Mail AddressGuard
          > http://antispam.yahoo.com/whatsnewfree
        • Derek James
          ... Sure SVG would allow that. It s just another graphical format, a vector-based one. This is the way JNEAT displays networks, right? This leads me to
          Message 4 of 21 , Jan 5, 2004
            --- Kenneth Stanley <kstanley@...> wrote:
            > Hey Derek I think it's a pretty cool way to display
            > networks. Have
            > you thought of having the line thickness
            > proportional to the weight
            > magnitude of connections (if svg allows that)?

            Sure SVG would allow that. It's just another
            graphical format, a vector-based one.

            This is the way JNEAT displays networks, right?

            This leads me to another question: To what extent do
            you use graphical information to analyze results? The
            use of graphing fitness throughout a run is obvious.
            Graphing the quantities of given numbers of
            individuals in species can be useful as well.

            But how often, and how do you use the graphical
            representation of the networks? This is an open
            question to the group. Also, Ken, I've noticed in
            some of your papers you've analyzed the functioning of
            the resultant network. Was graphical output of the
            topologies useful for this? Otherwise, how did you go
            about it?

            Derek

            __________________________________
            Do you Yahoo!?
            Protect your identity with Yahoo! Mail AddressGuard
            http://antispam.yahoo.com/whatsnewfree
          • Kenneth Stanley
            ... Yes I believe so. ... In general you never need graphical information but it helps. For example, if you look at the graph you can see which nodes are most
            Message 5 of 21 , Jan 7, 2004
              --- In neat@yahoogroups.com, Derek James <blue5432@y...> wrote:
              > --- Kenneth Stanley <kstanley@c...> wrote:
              > > Hey Derek I think it's a pretty cool way to display
              > > networks. Have
              > > you thought of having the line thickness
              > > proportional to the weight
              > > magnitude of connections (if svg allows that)?
              >
              > Sure SVG would allow that. It's just another
              > graphical format, a vector-based one.
              >
              > This is the way JNEAT displays networks, right?
              >

              Yes I believe so.

              > This leads me to another question: To what extent do
              > you use graphical information to analyze results? The
              > use of graphing fitness throughout a run is obvious.
              > Graphing the quantities of given numbers of
              > individuals in species can be useful as well.
              >
              > But how often, and how do you use the graphical
              > representation of the networks? This is an open
              > question to the group. Also, Ken, I've noticed in
              > some of your papers you've analyzed the functioning of
              > the resultant network. Was graphical output of the
              > topologies useful for this? Otherwise, how did you go
              > about it?
              >

              In general you never need graphical information but it helps.
              For example, if you look at the graph you can see which nodes
              are most highly connected, how many connections are recurrent,
              which inputs get used the most, etc. You can use that to decide
              what parts to ablate (i.e. take out) or keep track of if you
              are trying to analyze functionality. Also, I think species
              visualizations (like in our EC paper) can be nice as well to
              understand the overall dynamics of the system.

              I'd also be interested if anyone else has some novel ideas for
              using graphics for analysis.

              ken
            • Derek James
              ... ... I agree. So far we re generating five types of graphical information: 1) spiral layout for neural network 2) layer layout for neural network 3)
              Message 6 of 21 , Jan 9, 2004
                --- Kenneth Stanley <kstanley@...> wrote:
                > In general you never need graphical information but
                > it helps.
                <snip>
                > Also, I think
                > species
                > visualizations (like in our EC paper) can be nice as
                > well to
                > understand the overall dynamics of the system.

                I agree. So far we're generating five types of
                graphical information:

                1) spiral layout for neural network
                2) layer layout for neural network
                3) graph of fitness over the run, with maximum,
                minimum, and average fitness for the population
                4) speciation graph very much like the one from your
                paper
                5) display of each game, and sequence of moves (for
                Tic-Tac-Toe, Gomoku, and Go)

                I'll post some examples of the fitness and species
                graphs this weekend.

                > I'd also be interested if anyone else has some novel
                > ideas for
                > using graphics for analysis.

                Right now we're strongly considering representing
                heredity and lineages graphically. A family tree,
                basically, showing parent-child relationships, when
                particular offspring arose, etc. I'm not exactly sure
                how I want to format this information yet, but I
                believe it would be useful information to represent
                graphically.

                Derek

                __________________________________
                Do you Yahoo!?
                Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                http://hotjobs.sweepstakes.yahoo.com/signingbonus
              • Derek James
                Hi all, Here s a sample of a fitness graph from one of our recent XOR runs:
                Message 7 of 21 , Jan 10, 2004
                  Hi all,

                  Here's a sample of a fitness graph from one of our
                  recent XOR runs:

                  http://photos.groups.yahoo.com/group/neat/vwp?.dir=/&.dnm=Sample+XOR+Fitness+Graph.gif&.src=gr&.view=t&.hires=t

                  It's also in the Files section in SVG format.

                  The x-axis is the generation number, the y-axis is the
                  fitness. Our nets range from around 1.4 to 9.0 in
                  fitness (out of a possible range of 0-16, with 16
                  being the max fitness). The blue bar for each
                  generation is the range of fitness for the population,
                  with a black dot at the average fitness.

                  We're getting incremental progress, rather than the
                  plateaus and sudden jumps you would expect from the
                  XOR domain. And our nets seem to hang at 3 correct
                  outputs, but not the fourth.

                  We're investigating our transcription of genomes into
                  networks and our the activation of our nets as
                  possible culprits, but we're still plugging away at
                  finding all the wrinkles.

                  Just thought I'd share some of our graphical
                  information in the meantime.

                  Derek

                  __________________________________
                  Do you Yahoo!?
                  Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                  http://hotjobs.sweepstakes.yahoo.com/signingbonus
                • Kenneth Stanley
                  ... One thing I ve wanted to do but haven t gotten around to is make the speciation graphs also show lineage. It is entirely possible to do. Notice in my
                  Message 8 of 21 , Jan 10, 2004
                    --- In neat@yahoogroups.com, Derek James <blue5432@y...> wrote:

                    >
                    > Right now we're strongly considering representing
                    > heredity and lineages graphically. A family tree,
                    > basically, showing parent-child relationships, when
                    > particular offspring arose, etc. I'm not exactly sure
                    > how I want to format this information yet, but I
                    > believe it would be useful information to represent
                    > graphically.
                    >

                    One thing I've wanted to do but haven't gotten around to is make the
                    speciation graphs also show lineage. It is entirely possible to do.
                    Notice in my speciation graphs I introduce new species on the
                    right-hand side. Instead, you could instroduce them as flowing out of
                    the species from which they originated (since we can know that). Then
                    you would see kind of "splits" in species as they flow downward where
                    new species broke off. There are some special cases, such as two
                    different species both producing the same new species at the same
                    time, but in general those don't have to be depicted since they are so
                    rare anyway.

                    ken
                  • Derek James
                    Well, we finally got our implementation firing on all cylinders. Here s the fitness graph of a successful XOR run:
                    Message 9 of 21 , Jan 11, 2004
                      Well, we finally got our implementation firing on all
                      cylinders. Here's the fitness graph of a successful
                      XOR run:

                      http://photos.groups.yahoo.com/group/neat/vwp?.dir=/&.dnm=XOR+Converged.gif&.src=gr&.view=t&.hires=t

                      Derek

                      __________________________________
                      Do you Yahoo!?
                      Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                      http://hotjobs.sweepstakes.yahoo.com/signingbonus
                    • Kenneth Stanley
                      That s great it looks like things are working! It s hard to tell from the graph, what is the average generation where it first gets it right? (Where right
                      Message 10 of 21 , Jan 11, 2004
                        That's great it looks like things are working! It's hard to tell from
                        the graph, what is the average generation where it first gets it
                        right? (Where right means that the outputs that should be 1 are above
                        .0.5 and the ones that should be 0 are below) And what size population
                        are you using?

                        Mattias had some interesting results from trying different sigmoid
                        functions with his system on XOR. He found it could make a big
                        impact. If you feel performance could improve, it may be worth
                        experimenting with activation functions a bit. Maybe Mattias can
                        comment on this since he went through a lot of interesting discoveries
                        on that front.

                        ken


                        --- In neat@yahoogroups.com, Derek James <blue5432@y...> wrote:
                        > Well, we finally got our implementation firing on all
                        > cylinders. Here's the fitness graph of a successful
                        > XOR run:
                        >
                        >
                        http://photos.groups.yahoo.com/group/neat/vwp?.dir
                        =/&.dnm=XOR+Converged.gif&.src=gr&.view=t&.hires=t
                        >
                        > Derek
                        >
                        > __________________________________
                        > Do you Yahoo!?
                        > Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                        > http://hotjobs.sweepstakes.yahoo.com/signingbonus
                      • Derek James
                        ... Yeah...it s a bit of a relief, actually. :) ... I ve done a couple dozen runs this evening, with different parameters, and it looks like it s solving XOR
                        Message 11 of 21 , Jan 11, 2004
                          --- Kenneth Stanley <kstanley@...> wrote:
                          > That's great it looks like things are working!

                          Yeah...it's a bit of a relief, actually. :)

                          >It's
                          > hard to tell from
                          > the graph, what is the average generation where it
                          > first gets it
                          > right? (Where right means that the outputs that
                          > should be 1 are above
                          > .0.5 and the ones that should be 0 are below)

                          I've done a couple dozen runs this evening, with
                          different parameters, and it looks like it's solving
                          XOR on average between 10 and 20 generations.

                          On the graph I posted in the Photos section, it seems
                          to have "solved" XOR around the 20th generation, but I
                          had the mutation rates pretty low on that run. With
                          the topology mutation rates at the standard 0.03
                          connection mutation and 0.01 neuron mutation, it seems
                          to solve for XOR pretty consistently within 10 or 12
                          generations.

                          > And
                          > what size population
                          > are you using?

                          That is listed at the top of the graph, along with
                          most of the major parameters: 100.

                          > Mattias had some interesting results from trying
                          > different sigmoid
                          > functions with his system on XOR. He found it could
                          > make a big
                          > impact. If you feel performance could improve, it
                          > may be worth
                          > experimenting with activation functions a bit.

                          I would be interested in hearing from Mattias...

                          So far we've tried both the standard sigmoid and
                          hyperbolic tangent. Tanh actually seems to perform
                          slightly better, but both of them are solving for XOR
                          early and consistently.

                          Derek

                          __________________________________
                          Do you Yahoo!?
                          Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                          http://hotjobs.sweepstakes.yahoo.com/signingbonus
                        • Derek James
                          Also, here s a speciation graph from one of our recent XOR runs:
                          Message 12 of 21 , Jan 11, 2004
                            Also, here's a speciation graph from one of our recent
                            XOR runs:

                            http://photos.groups.yahoo.com/group/neat/vwp?.dir=/&.dnm=Speciation+from+XOR+run.gif&.src=gr&.view=t&.hires=t

                            ...though I'm still tinkering with the format of this
                            graphic.

                            This is with the speciation threshold set at 0.9,
                            while normalizing for chromosome size.

                            Derek

                            __________________________________
                            Do you Yahoo!?
                            Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes
                            http://hotjobs.sweepstakes.yahoo.com/signingbonus
                          • Mattias Fagerlund
                            ... Sigmoid; When I use a sigmoid that centers around 0.5 (it returns values in the range 0 to 1) I get results that are on par with Kens. But when I use a
                            Message 13 of 21 , Jan 12, 2004
                              > Mattias had some interesting results from trying different sigmoid
                              > functions with his system on XOR. He found it could make a big
                              > impact. If you feel performance could improve, it may be worth
                              > experimenting with activation functions a bit. Maybe Mattias can
                              > comment on this since he went through a lot of interesting discoveries
                              > on that front.

                              Sigmoid;
                              When I use a sigmoid that centers around 0.5 (it returns values in the
                              range 0 to 1) I get results that are on par with Kens. But when I use a
                              sigmoid that's centered around 0 (range=[-1..1]) my results are far
                              better than those reported by Ken.

                              Also, the fitness method for XOR is debatable; once a particular result
                              is correct (on the right side of 0.5) the case could be given a fitness
                              of 1. In kens code, returning 0.51 for a true means a fitness of
                              only .49 fabs(0.51-1) (though it's considered a hit). I would argue that
                              it's of fitness 1. Finding correct answers is much faster with that
                              fitness method, and it's valid because
                              1) the particular case is allready solved, so further evolution should
                              be targeted at solving the other cases
                              2) the completed result is as correct as the original method
                              3) it's considerably faster to solve

                              cheers,
                              m
                            • Mattias Fagerlund
                              ... Beyond the simple sigmoid and variations on it, I ve also used a gaussian transfer function (think bell curve), which wildly improved my results on the
                              Message 14 of 21 , Jan 12, 2004
                                > I would be interested in hearing from Mattias...
                                >
                                > So far we've tried both the standard sigmoid and
                                > hyperbolic tangent. Tanh actually seems to perform
                                > slightly better, but both of them are solving for XOR
                                > early and consistently.

                                Beyond the simple sigmoid and variations on it, I've also used a
                                gaussian transfer function (think bell curve), which wildly improved my
                                results on the artificial ant. But it can't be used with XOR, because
                                gaussian can solve XOR without internal nodes... ;)

                                cheers,
                                m
                              • Darren Izzard
                                ... I d love to know how these would do on my shape recogniser. --Darren
                                Message 15 of 21 , Jan 12, 2004
                                  > > So far we've tried both the standard sigmoid and
                                  > > hyperbolic tangent. Tanh actually seems to perform
                                  > > slightly better, but both of them are solving for XOR
                                  > > early and consistently.
                                  > Beyond the simple sigmoid and variations on it, I've also used a
                                  > gaussian transfer function (think bell curve), which wildly improved my
                                  > results on the artificial ant.

                                  I'd love to know how these would do on my shape recogniser.

                                  --Darren
                                • Mattias Fagerlund
                                  ... Where can I read more about your shape recognizer? cheers, m
                                  Message 16 of 21 , Jan 12, 2004
                                    > I'd love to know how these would do on my shape recogniser.

                                    Where can I read more about your shape recognizer?

                                    cheers,
                                    m
                                  • Darren Izzard
                                    ... On this list, in the archives ;) --Darren
                                    Message 17 of 21 , Jan 12, 2004
                                      > > I'd love to know how these would do on my shape recogniser.
                                      > Where can I read more about your shape recognizer?

                                      On this list, in the archives ;)

                                      --Darren
                                    • Mattias Fagerlund
                                      ... Ah, found it! I may have to recreate your system one day. To see how DelphiNEAT compares, and to find out how my alternative node types behave. I d also
                                      Message 18 of 21 , Jan 12, 2004
                                        > > > I'd love to know how these would do on my shape recogniser.
                                        > > Where can I read more about your shape recognizer?
                                        >
                                        > On this list, in the archives ;)
                                        >
                                        > --Darren

                                        Ah, found it! I may have to recreate your system one day. To see how
                                        DelphiNEAT compares, and to find out how my alternative node types
                                        behave. I'd also like to do character recognition (OCR)...

                                        How are all you GO evolvers fairing?

                                        cheers,
                                        m
                                      • Kenneth Stanley
                                        ... how ... I ve had some interesting results. I m actually writing up the paper for it now. Unfortunately I can t really post it until (and unless) it s
                                        Message 19 of 21 , Jan 13, 2004
                                          --- In neat@yahoogroups.com, "Mattias Fagerlund" <mattias@c...> wrote:
                                          > > > > I'd love to know how these would do on my shape recogniser.
                                          > > > Where can I read more about your shape recognizer?
                                          > >
                                          > > On this list, in the archives ;)
                                          > >
                                          > > --Darren
                                          >
                                          > Ah, found it! I may have to recreate your system one day. To see
                                          how
                                          > DelphiNEAT compares, and to find out how my alternative node types
                                          > behave. I'd also like to do character recognition (OCR)...
                                          >
                                          > How are all you GO evolvers fairing?
                                          >

                                          I've had some interesting results. I'm actually writing up the paper
                                          for it now. Unfortunately I can't really post it until (and unless)
                                          it's accepted, so it might take a little time to come out, but suffice
                                          it to say it uses a novel representation for processing Go with an NN.

                                          ken
                                        • Kenneth Stanley
                                          ... http://photos.groups.yahoo.com/group/neat/vwp?.dir =/&.dnm=Speciation+from+XOR+run.gif&.src=gr&.view=t&.hires=t ... Hey, Derek, you used elitism in the
                                          Message 20 of 21 , Jan 13, 2004
                                            --- In neat@yahoogroups.com, Derek James <blue5432@y...> wrote:
                                            > Also, here's a speciation graph from one of our recent
                                            > XOR runs:
                                            >
                                            >
                                            http://photos.groups.yahoo.com/group/neat/vwp?.dir
                                            =/&.dnm=Speciation+from+XOR+run.gif&.src=gr&.view=t&.hires=t
                                            >
                                            > ...though I'm still tinkering with the format of this
                                            > graphic.
                                            >
                                            > This is with the speciation threshold set at 0.9,
                                            > while normalizing for chromosome size.
                                            >
                                            > Derek
                                            >

                                            Hey, Derek, you used elitism in the species right? In other words,
                                            the champion of any species alotted more than 3 (or 5 or whatever)
                                            offspring was copied into the next generation?

                                            The speciation looks good, but there is some fluctuation in the size
                                            of species. I am not sure if that means anything, but I don't always
                                            see that much fluctuation. Of course it depends on the fitness
                                            function, and you may have a slightly more sensitive one than I used
                                            in XOR.

                                            ken
                                          • Mattias Fagerlund
                                            ... You TEASE! ;) But I shall wait patiently to read the paper... cheers, m
                                            Message 21 of 21 , Jan 13, 2004
                                              > > How are all you GO evolvers fairing?
                                              > >
                                              >
                                              > I've had some interesting results. I'm actually writing up the paper
                                              > for it now. Unfortunately I can't really post it until (and unless)
                                              > it's accepted, so it might take a little time to come out, but suffice
                                              > it to say it uses a novel representation for processing Go with an NN.

                                              You TEASE! ;) But I shall wait patiently to read the paper...

                                              cheers,
                                              m
                                            Your message has been successfully submitted and would be delivered to recipients shortly.