Loading ...
Sorry, an error occurred while loading the content.

New paper on why modules evolve, and how to evolve modular artificial neural net

Expand Messages
  • Jeff
    Hello all, I m extremely pleased to announce a new paper on a subject that many--including myself--think is critical to making significant progress in our
    Message 1 of 2 , Feb 6, 2013
    • 0 Attachment

      Hello all,


      I'm extremely pleased to announce a new paper on a subject that many--including myself--think is critical to making significant progress in our field: the evolution of modularity. 


      Jean-Baptiste Mouret, Hod Lipson and I have a new paper that 


      1) sheds light on why modularity may evolve in biological networks (e.g. neural, genetic, metabolic, protein-protein, etc.)


      2) provides a simple technique for evolving neural networks that are modular and have increased evolvability, in that they adapt faster to new environments. The modules that formed solved subproblems in the domain. 


      Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. http://dx.doi.org/10.1098/rspb.2012.2863 (pdf)


      Abstract: A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments). A key driver of evolvability is the widespread modularity of biological networks—their organization as functional, sparsely connected subunits—but there is no consensus regarding why modularity itself evolved. Although most hypotheses assume indirect selection for evolvability, here we demonstrate that the ubiquitous, direct selection pressure to reduce the cost of connections between network nodes causes the emergence of modular networks. Computational evolution experiments with selection pressures to maximize network performance and minimize connection costs yield networks that are significantly more modular and more evolvable than control experiments that only select for performance. These results will catalyse research in numerous disciplines, such as neuroscience and genetics, and enhance our ability to harness evolution for engineering purposes.


      Video: http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng


      There has been some nice coverage of this work in the popular press, in case you are interested:


      National Geographic: http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/

      MIT's Technology Review: http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/ 

      Fast Company: http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning

      Cornell Chronicle: http://www.news.cornell.edu/stories/Jan13/modNetwork.html

      ScienceDaily: http://www.sciencedaily.com/releases/2013/01/130130082300.htm


      Please let me know what you think and if you have any questions. 


      Best regards,

      Jeff Clune


      Assistant Professor

      Computer Science

      University of Wyoming

      jclune@...

      jeffclune.com

    • Schmidhuber Juergen
      Sorry for the delayed reply! The paper mentions that Santiago Ramón y Cajal already pointed out that evolution has created mostly short connections in animal
      Message 2 of 2 , Feb 21, 2013
      • 0 Attachment
        Sorry for the delayed reply!

        The paper mentions that Santiago Ramón y Cajal already pointed out that evolution has created mostly short connections in animal brains.

        Minimization of connection costs should also encourage modularization, e.g., http://arxiv.org/abs/1210.0118 (2012).

        But who first had such a wire length term in an objective function to be minimized by evolutionary computation or other machine learning methods?
        I am aware of pioneering work by Legenstein and Maass:

        R. A. Legenstein and W. Maass. Neural circuits for pattern recognition with small total wire length. Theoretical Computer Science, 287:239-249, 2002.
        R. A. Legenstein and W. Maass. Wire length as a circuit complexity measure. Journal of Computer and System Sciences, 70:53-72, 2005.

        Is there any earlier relevant work? Pointers will be appreciated.

        Jürgen Schmidhuber
        http://www.idsia.ch/~juergen/whatsnew.html


        PS: We have new positions for postdocs and PhD students. Highly competitive salary at the award-winning Swiss AI Lab IDSIA http://www.idsia.ch/  in the world's leading science nation http://www.idsia.ch/~juergen/switzerland.html .  Please follow instructions under http://www.idsia.ch/~juergen/eu2013.html






        On Feb 6, 2013, at 7:48 PM, Jeff wrote:

         

        Hello all,


        I'm extremely pleased to announce a new paper on a subject that many--including myself--think is critical to making significant progress in our field: the evolution of modularity. 


        Jean-Baptiste Mouret, Hod Lipson and I have a new paper that 


        1) sheds light on why modularity may evolve in biological networks (e.g. neural, genetic, metabolic, protein-protein, etc.)


        2) provides a simple technique for evolving neural networks that are modular and have increased evolvability, in that they adapt faster to new environments. The modules that formed solved subproblems in the domain. 


        Cite: Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins of modularity. Proceedings of the Royal Society B. 280: 20122863. http://dx.doi.org/10.1098/rspb.2012.2863 (pdf)


        Abstract: A central biological question is how natural organisms are so evolvable (capable of quickly adapting to new environments). A key driver of evolvability is the widespread modularity of biological networks—their organization as functional, sparsely connected subunits—but there is no consensus regarding why modularity itself evolved. Although most hypotheses assume indirect selection for evolvability, here we demonstrate that the ubiquitous, direct selection pressure to reduce the cost of connections between network nodes causes the emergence of modular networks. Computational evolution experiments with selection pressures to maximize network performance and minimize connection costs yield networks that are significantly more modular and more evolvable than control experiments that only select for performance. These results will catalyse research in numerous disciplines, such as neuroscience and genetics, and enhance our ability to harness evolution for engineering purposes.


        Video: http://www.youtube.com/watch?feature=player_embedded&v=SG4_aW8LMng


        There has been some nice coverage of this work in the popular press, in case you are interested:


        National Geographic: http://phenomena.nationalgeographic.com/2013/01/30/the-parts-of-life/

        MIT's Technology Review: http://www.technologyreview.com/view/428504/computer-scientists-reproduce-the-evolution-of-evolvability/ 

        Fast Company: http://www.fastcompany.com/3005313/evolved-brains-robots-creep-closer-animal-learning

        Cornell Chronicle: http://www.news.cornell.edu/stories/Jan13/modNetwork.html

        ScienceDaily: http://www.sciencedaily.com/releases/2013/01/130130082300.htm


        Please let me know what you think and if you have any questions. 



        Best regards,

        Jeff Clune


        Assistant Professor

        Computer Science

        University of Wyoming

        jclune@...

        jeffclune.com




      Your message has been successfully submitted and would be delivered to recipients shortly.