Loading ...
Sorry, an error occurred while loading the content.

Opening very large networks

Expand Messages
  • Steve Wolstenholme
    A recent bug report suggested that there is a problem opening network files when the grid is above 10,000 rows and 100 columns. After a lot of work with the
    Message 1 of 24 , Jan 14, 2007
    • 0 Attachment
      A recent bug report suggested that there is a problem opening network
      files when the grid is above 10,000 rows and 100 columns.

      After a lot of work with the person who reported the bug it seems that
      the only problem is related to the network opening speed when tracing
      is enabled.

      This is because a lot scaling and averaging of column values is done
      while the network file is being read in. This saves time later when
      the network is learning.

      With tracing enabled the calculation of average values is repeated for
      every row. This makes tracing very simple but makes the file opening
      slow. On top of that the calculated values are not added to the column
      slaves until the file is fully loaded making the apparent opening time
      is even slower.

      The problem is fixed in the next update. For now please don't use
      tracing with large network files unless it is absolutely necessary.

      Steve

      --
      Steve Wolstenholme Neural Planner Software

      EasyNN-plus. The easy way to build neural networks.
      http://www.easynn.com
    • PDickens
      All of the e mails that have been coming through recently are always a mess. Previously they were never a problem, but now they all look like this, what are
      Message 2 of 24 , Jan 14, 2007
      • 0 Attachment
        All of the e'mails that have been coming through recently are always a
        mess.

        Previously they were never a problem, but now they all look like this,
        what are you doing to cause this?



        [Image]




        [Non-text portions of this message have been removed]
      • PDickens
        I have also had problems with large networks, but not because of using Tracing . The networks concerned have 128 columns and 320,129 rows The problems have
        Message 3 of 24 , Jan 14, 2007
        • 0 Attachment
          I have also had problems with large networks, but not because of using
          'Tracing'.

          The networks concerned have 128 columns and 320,129 rows
          The problems have been multivarious:

          a) When changing all zeros in the Network to a higher value [to create
          noise] (using edit replace), the software just hangs after about half an
          hour and does not complete the task..
          b) When changing all zeros in the Network to a higher value [to create
          noise] (using edit replace), the software falls over. i.e. ENN plus
          disappears completely.
          c) When changing all zeros in the Network to a higher value [to create
          noise] (using edit replace), the software (I assume, based on the
          previous two examples) has made the platform fall over (W2000Pro), i.e.
          the computer crashes. (This is not a memory problem there are Gigs. of
          memory

          Consequentially I did this process within "TextPad" (much more powerful
          than Notepad). which it completed in a few seconds.

          I then reloaded the amended file into ENNplus. It took a tremendous
          amount of time to load, nearly an hour (even though I was using Intel's
          fastest CPU)
          The result was after successfully loasding and trying to start the
          network.

          a) The software just hangs.
          b) the software falls over.

          I had intended to take the size up, by increments, to 3.5 million rows,
          but clearly that will not be possible.

          Consequent upon the above, I reduced the number of rows to 115,000 with
          these results:

          a) When changing all zeros in the Network to a higher value [to create
          noise] (using edit replace), the software just hangs after about half an
          hour and does not complete the task..
          b) When changing all zeros in the Network to a higher value [to create
          noise] (using edit replace), the software falls over. i.e. ENN plus
          disappears completely.

          Consequentially I did this process within "TextPad".
          I then reloaded the amended file into ENNplus. It still took a
          tremendous amount of time to load, but it did load and start.

          I then repeated the operation with 165,000 rows which I had edited in
          'Textpad' first. reloaded it and it started.

          So somewhere between 165,000 rows and and 320,129 rows, ENNPlus becomes
          unstable.
          And clearly there is a problem with Edit Replace for any network of this
          sort of size. i.e. 100,000 and above. How far down one has to go in
          size, I do not know.


          ==============================================

          Steve Wolstenholme wrote:

          > A recent bug report suggested that there is a problem opening network
          > files when the grid is above 10,000 rows and 100 columns.
          >
          > After a lot of work with the person who reported the bug it seems that
          >
          > the only problem is related to the network opening speed when tracing
          > is enabled.
          >
          > This is because a lot scaling and averaging of column values is done
          > while the network file is being read in. This saves time later when
          > the network is learning.
          >
          > With tracing enabled the calculation of average values is repeated for
          >
          > every row. This makes tracing very simple but makes the file opening
          > slow. On top of that the calculated values are not added to the column
          >
          > slaves until the file is fully loaded making the apparent opening time
          >
          > is even slower.
          >
          > The problem is fixed in the next update. For now please don't use
          > tracing with large network files unless it is absolutely necessary.
          >
          > Steve
          >
          > --
          > Steve Wolstenholme Neural Planner Software
          >
          > EasyNN-plus. The easy way to build neural networks.
          > http://www.easynn.com
          >


          [Non-text portions of this message have been removed]
        • Steve Wolstenholme
          ... Nothing has been changed. Some people send images but it s a text only group. -- Steve Wolstenholme Neural Planner Software EasyNN-plus. The easy way
          Message 4 of 24 , Jan 14, 2007
          • 0 Attachment
            On Sun, 14 Jan 2007 15:42:51 +0000, you wrote:

            >All of the e'mails that have been coming through recently are always a
            >mess.
            >
            >Previously they were never a problem, but now they all look like this,
            >what are you doing to cause this?
            >
            >


            Nothing has been changed. Some people send images but it's a text only
            group.

            --
            Steve Wolstenholme Neural Planner Software

            EasyNN-plus. The easy way to build neural networks.
            http://www.easynn.com
          • alain arsenault
            how can i apply a genetic algorythm to easynn...Thks ... Nothing has been changed. Some people send images but it s a text only group. -- Steve Wolstenholme
            Message 5 of 24 , Jan 14, 2007
            • 0 Attachment
              how can i apply a genetic algorythm to easynn...Thks

              Steve Wolstenholme <steve@...> wrote: On Sun, 14 Jan 2007 15:42:51 +0000, you wrote:

              >All of the e'mails that have been coming through recently are always a
              >mess.
              >
              >Previously they were never a problem, but now they all look like this,
              >what are you doing to cause this?
              >
              >

              Nothing has been changed. Some people send images but it's a text only
              group.

              --
              Steve Wolstenholme Neural Planner Software

              EasyNN-plus. The easy way to build neural networks.
              http://www.easynn.com





              ---------------------------------
              Everyone is raving about the all-new Yahoo! Mail beta.

              [Non-text portions of this message have been removed]
            • Steve Wolstenholme
              ... Edit - Replace is a bit better in the next update but still not satisfactory. The problem is that Edit - Replace is too general and takes too much time
              Message 6 of 24 , Jan 14, 2007
              • 0 Attachment
                On Sun, 14 Jan 2007 16:24:38 +0000, you wrote:

                >And clearly there is a problem with Edit Replace for any network of this
                >sort of size. i.e. 100,000 and above. How far down one has to go in
                >size, I do not know.

                Edit - Replace is a bit better in the next update but still not
                satisfactory. The problem is that Edit - Replace is too general and
                takes too much time loading the whole grid cell object, changing one
                item and then saving the whole object. It is a lot of work to split
                the object into component parts but that is the way it needs to be
                done.

                It is the number of rows that slows the Edit but it is the high number
                columns that causes the other delays. Grids with a million rows and
                five columns performs OK but 100,000 rows with 200 columns is
                painfully slow.

                --
                Steve Wolstenholme Neural Planner Software

                EasyNN-plus. The easy way to build neural networks.
                http://www.easynn.com
              • Steve Wolstenholme
                ... I ve moved this into another thread because it is nothing to do with large networks. EasyNN-plus does not use genetic training at all. It only uses
                Message 7 of 24 , Jan 14, 2007
                • 0 Attachment
                  On Sun, 14 Jan 2007 08:53:39 -0800 (PST), alain arsenault wrote:

                  >how can i apply a genetic algorythm to easynn...Thks

                  I've moved this into another thread because it is nothing to do with
                  large networks.

                  EasyNN-plus does not use genetic training at all. It only uses
                  backpropagation for training.

                  --
                  Steve Wolstenholme Neural Planner Software

                  EasyNN-plus. The easy way to build neural networks.
                  http://www.easynn.com
                • PDickens
                  Is there ever any likelihood of Genetic algorithms and / or Genetic training tools being included in a future edition of ENN Plus ? ... [Non-text portions of
                  Message 8 of 24 , Jan 14, 2007
                  • 0 Attachment
                    Is there ever any likelihood of Genetic algorithms and / or Genetic
                    training tools being included in a future edition of 'ENN Plus' ?



                    Steve Wolstenholme wrote:

                    > On Sun, 14 Jan 2007 08:53:39 -0800 (PST), alain arsenault wrote:
                    >
                    > >how can i apply a genetic algorythm to easynn...Thks
                    >
                    > I've moved this into another thread because it is nothing to do with
                    > large networks.
                    >
                    > EasyNN-plus does not use genetic training at all. It only uses
                    > backpropagation for training.
                    >
                    > --
                    > Steve Wolstenholme Neural Planner Software
                    >
                    > EasyNN-plus. The easy way to build neural networks.
                    > http://www.easynn.com
                    >


                    [Non-text portions of this message have been removed]
                  • Steve Wolstenholme
                    ... Genetic training is not suitable for EasyNN-plus. Genetic optimization of training parameters and subsequent network growth has already been tried but the
                    Message 9 of 24 , Jan 14, 2007
                    • 0 Attachment
                      On Sun, 14 Jan 2007 19:24:02 +0000, PDickens wrote:

                      >Is there ever any likelihood of Genetic algorithms and / or Genetic
                      >training tools being included in a future edition of 'ENN Plus' ?

                      Genetic training is not suitable for EasyNN-plus. Genetic optimization
                      of training parameters and subsequent network growth has already been
                      tried but the results were disappointing and much slower than the
                      simple methods currently used.

                      --
                      Steve Wolstenholme Neural Planner Software

                      EasyNN-plus. The easy way to build neural networks.
                      http://www.easynn.com
                    • Steve Wolstenholme
                      ... I have solved a major part of the problem with Edit Replace. It is now much faster. The changes will be in the next update. I m still looking at other
                      Message 10 of 24 , Jan 16, 2007
                      • 0 Attachment
                        On Sun, 14 Jan 2007 17:11:08 +0000, I wrote:

                        >The problem is that Edit - Replace is too general and
                        >takes too much time loading the whole grid cell object, changing one
                        >item and then saving the whole object. It is a lot of work to split
                        >the object into component parts but that is the way it needs to be
                        >done.

                        I have solved a major part of the problem with Edit > Replace. It is
                        now much faster. The changes will be in the next update.

                        I'm still looking at other editing and file opening performance
                        problems.

                        Steve

                        --
                        Steve Wolstenholme Neural Planner Software

                        EasyNN-plus. The easy way to build neural networks.
                        http://www.easynn.com
                      • P. Dickens
                        Good to hear, there is almost always another way out and a good programer will find it, however when there isn t, an exceptional programer will create it,
                        Message 11 of 24 , Jan 16, 2007
                        • 0 Attachment
                          Good to hear, there is almost always another way out and a good
                          programer will find it, however when there isn't, an exceptional
                          programer will create it, methinks your the latter.

                          Well done and thanks,

                          P. Dickens

                          ------------------------------------
                          Steve Wolstenholme wrote:

                          > On Sun, 14 Jan 2007 17:11:08 +0000, I wrote:
                          >
                          > >The problem is that Edit - Replace is too general and
                          > >takes too much time loading the whole grid cell object, changing one
                          > >item and then saving the whole object. It is a lot of work to split
                          > >the object into component parts but that is the way it needs to be
                          > >done.
                          >
                          > I have solved a major part of the problem with Edit > Replace. It is
                          > now much faster. The changes will be in the next update.
                          >
                          > I'm still looking at other editing and file opening performance
                          > problems.
                          >
                          > Steve
                          >
                          > --
                          > Steve Wolstenholme Neural Planner Software
                          >
                          > EasyNN-plus. The easy way to build neural networks.
                          > http://www.easynn.com
                          >


                          [Non-text portions of this message have been removed]
                        • Steve Wolstenholme
                          ... Thanks for the compliment. In this case I should have known better. Manipulating rows of objects using indexes is too slow when there are many thousands of
                          Message 12 of 24 , Jan 16, 2007
                          • 0 Attachment
                            On Tue, 16 Jan 2007 13:49:20 +0000, you wrote:

                            >Good to hear, there is almost always another way out and a good
                            >programer will find it, however when there isn't, an exceptional
                            >programer will create it, methinks your the latter.
                            >
                            >Well done and thanks,
                            >
                            >P. Dickens

                            Thanks for the compliment. In this case I should have known better.
                            Manipulating rows of objects using indexes is too slow when there are
                            many thousands of them. I should have used links but for some reason I
                            didn't. I do now! It's a lot faster.

                            I've just read your message about using TextPad. How do you get the
                            data back into EasyNN-plus? Import?

                            Steve

                            --
                            Steve Wolstenholme Neural Planner Software

                            EasyNN-plus. The easy way to build neural networks.
                            http://www.easynn.com
                          • P. Dickens
                            Yes. It creates a simple text file txt with none of the rubbish that accrues in Wordpad or Word Pro or Word. it treats text just as Notepad does and is
                            Message 13 of 24 , Jan 16, 2007
                            • 0 Attachment
                              Yes.

                              It creates a simple text file txt with none of the rubbish that accrues
                              in Wordpad or Word Pro or Word.
                              it treats text just as Notepad does and is excellent for cleaning out
                              the rubbish when copying and pasting from spreadsheets, Html or
                              Wordprocessor programs.
                              Besides which, it can handle huge files, have gone to 3.5 million lines
                              so far. It will import in Blocks any position, so I can double and
                              treble up (not done 4 yet) the width of a full spreadsheet when I want
                              to add spreadsheet data side by side, without any hassle.
                              It is brilliant at manipulating text and will manipulate tags
                              (wordprocessor and HTML), as well as sorting text as per a database.
                              It is wonderfully fast with huge files when changing data or text
                              (search and change) and leaves notepad standing.
                              You can also pragram it. Very useful with multiple repeat file changes.



                              Steve Wolstenholme wrote:

                              > On Tue, 16 Jan 2007 13:49:20 +0000, you wrote:
                              >
                              > >Good to hear, there is almost always another way out and a good
                              > >programer will find it, however when there isn't, an exceptional
                              > >programer will create it, methinks your the latter.
                              > >
                              > >Well done and thanks,
                              > >
                              > >P. Dickens
                              >
                              > Thanks for the compliment. In this case I should have known better.
                              > Manipulating rows of objects using indexes is too slow when there are
                              > many thousands of them. I should have used links but for some reason I
                              >
                              > didn't. I do now! It's a lot faster.
                              >
                              > I've just read your message about using TextPad. How do you get the
                              > data back into EasyNN-plus? Import?
                              >
                              > Steve
                              >
                              > --
                              > Steve Wolstenholme Neural Planner Software
                              >
                              > EasyNN-plus. The easy way to build neural networks.
                              > http://www.easynn.com
                              >


                              [Non-text portions of this message have been removed]
                            • Steve Wolstenholme
                              ... How long did it take to import the file back into EasyNN-plus once you had done the editing with TextPad? ... I ll get a copy. Steve -- Steve Wolstenholme
                              Message 14 of 24 , Jan 16, 2007
                              • 0 Attachment
                                On Tue, 16 Jan 2007 18:03:59 +0000, you wrote:

                                >Yes.

                                How long did it take to import the file back into EasyNN-plus once you
                                had done the editing with TextPad?

                                >
                                >It creates a simple text file txt with none of the rubbish that accrues
                                >in Wordpad or Word Pro or Word.

                                I'll get a copy.

                                Steve

                                --
                                Steve Wolstenholme Neural Planner Software

                                EasyNN-plus. The easy way to build neural networks.
                                http://www.easynn.com
                              • P. Dickens
                                The short answer is, not long enough for me to take much notice, longer than a few minutes though. I will test it again in a second instance and time it. Here
                                Message 15 of 24 , Jan 16, 2007
                                • 0 Attachment
                                  The short answer is, not long enough for me to take much notice, longer
                                  than a few minutes though.
                                  I will test it again in a second instance and time it.

                                  Here is the URL for TextPad
                                  http://www.textpad.com/

                                  Here's the spec.
                                  * Conforms to the Microsoft Windows Guidelines for Accessible
                                  Software Design recommendations.
                                  * The 32-bit edition can handle file sizes up to the limits of
                                  virtual memory. The 16-bit edition is limited to files of up to 32,500
                                  lines, with a maximum line length of 4096 bytes.
                                  * The number of concurrent edits is limited by available resources.
                                  This limit is much greater on Windows NT/2000 than on Windows 9x.
                                  * Supports Universal Naming Convention (UNC) style, and long file
                                  names with spaces.
                                  * 16-bit Unicode, UTF-8 and 8-bit text files with single and double
                                  byte characters can be edited.
                                  * DOS, UNIX, Macintosh, Netscape and mixed end of line characters
                                  are supported.
                                  * Text can be in either the ANSI (Windows) or OEM (DOS) code sets.
                                  * Each document has a single MDI window with up to 4 views.
                                  * The full range of fonts installed in MS Windows are supported.
                                  * Up to 32 tab stops can be defined, along with a default tab
                                  spacing, and an indent size.
                                  * Up to 16 (user configurable) most recently used files are listed
                                  on the File menu.
                                  * Up to 64 user-defined commands can be added to the Tools menu.
                                  * Up to 64 keystroke macros can be added to the Macros menu.
                                  * Year 2000 compliant.

                                  I've been using it for years and the chap who created it is very
                                  helpful.
                                  Here is his address: I've messed up the e'mail address for obvious
                                  reasons, you can soon clean it.

                                  Helios Software Solutions
                                  PO Box 619
                                  LONGRIDGE
                                  PR3 2GW
                                  England
                                  FAX: +44 (0)1772 786375
                                  E-Mail: XXXXXsalesXXXX@...

                                  For £9.50 + VAT it is a real bargain and he deserves it.
                                  .
                                  I'll let you know re the timings, unfortunately I have another ENN
                                  instance that has been doing a "check grid" for the last two hours and
                                  hasn't finished yet, (the last large network I did took most of the
                                  night), so I will have to wait until it does finish, because that is
                                  the only machine I have available at the moment with the same power.. So
                                  I will be in touch ASAP.

                                  -------------------------------------
                                  Steve Wolstenholme wrote:

                                  > On Tue, 16 Jan 2007 18:03:59 +0000, you wrote:
                                  >
                                  > >Yes.
                                  >
                                  > How long did it take to import the file back into EasyNN-plus once you
                                  >
                                  > had done the editing with TextPad?
                                  >
                                  > >
                                  > >It creates a simple text file txt with none of the rubbish that
                                  > accrues
                                  > >in Wordpad or Word Pro or Word.
                                  >
                                  > I'll get a copy.
                                  >
                                  > Steve
                                  >
                                  > --
                                  > Steve Wolstenholme Neural Planner Software
                                  >
                                  > EasyNN-plus. The easy way to build neural networks.
                                  > http://www.easynn.com
                                  >


                                  [Non-text portions of this message have been removed]
                                • P. Dickens
                                  320,129 rows with 10 columns. Around 45 , but bear in mind I had another instance running and yet another checking. ... [Non-text portions of this message
                                  Message 16 of 24 , Jan 16, 2007
                                  • 0 Attachment
                                    320,129 rows with 10 columns.
                                    Around 45' , but bear in mind I had another instance running and yet
                                    another checking.


                                    "P. Dickens" wrote:

                                    > The short answer is, not long enough for me to take much notice,
                                    > longer
                                    > than a few minutes though.
                                    > I will test it again in a second instance and time it.
                                    >
                                    > Here is the URL for TextPad
                                    > http://www.textpad.com/
                                    >
                                    > Here's the spec.
                                    > * Conforms to the Microsoft Windows Guidelines for Accessible
                                    > Software Design recommendations.
                                    > * The 32-bit edition can handle file sizes up to the limits of
                                    > virtual memory. The 16-bit edition is limited to files of up to 32,500
                                    >
                                    > lines, with a maximum line length of 4096 bytes.
                                    > * The number of concurrent edits is limited by available resources.
                                    > This limit is much greater on Windows NT/2000 than on Windows 9x.
                                    > * Supports Universal Naming Convention (UNC) style, and long file
                                    > names with spaces.
                                    > * 16-bit Unicode, UTF-8 and 8-bit text files with single and double
                                    > byte characters can be edited.
                                    > * DOS, UNIX, Macintosh, Netscape and mixed end of line characters
                                    > are supported.
                                    > * Text can be in either the ANSI (Windows) or OEM (DOS) code sets.
                                    > * Each document has a single MDI window with up to 4 views.
                                    > * The full range of fonts installed in MS Windows are supported.
                                    > * Up to 32 tab stops can be defined, along with a default tab
                                    > spacing, and an indent size.
                                    > * Up to 16 (user configurable) most recently used files are listed
                                    > on the File menu.
                                    > * Up to 64 user-defined commands can be added to the Tools menu.
                                    > * Up to 64 keystroke macros can be added to the Macros menu.
                                    > * Year 2000 compliant.
                                    >
                                    > I've been using it for years and the chap who created it is very
                                    > helpful.
                                    > Here is his address: I've messed up the e'mail address for obvious
                                    > reasons, you can soon clean it.
                                    >
                                    > Helios Software Solutions
                                    > PO Box 619
                                    > LONGRIDGE
                                    > PR3 2GW
                                    > England
                                    > FAX: +44 (0)1772 786375
                                    > E-Mail: XXXXXsalesXXXX@...
                                    >
                                    > For £9.50 + VAT it is a real bargain and he deserves it.
                                    > .
                                    > I'll let you know re the timings, unfortunately I have another ENN
                                    > instance that has been doing a "check grid" for the last two hours and
                                    >
                                    > hasn't finished yet, (the last large network I did took most of the
                                    > night), so I will have to wait until it does finish, because that is
                                    > the only machine I have available at the moment with the same power..
                                    > So
                                    > I will be in touch ASAP.
                                    >
                                    > -------------------------------------
                                    > Steve Wolstenholme wrote:
                                    >
                                    > > On Tue, 16 Jan 2007 18:03:59 +0000, you wrote:
                                    > >
                                    > > >Yes.
                                    > >
                                    > > How long did it take to import the file back into EasyNN-plus once
                                    > you
                                    > >
                                    > > had done the editing with TextPad?
                                    > >
                                    > > >
                                    > > >It creates a simple text file txt with none of the rubbish that
                                    > > accrues
                                    > > >in Wordpad or Word Pro or Word.
                                    > >
                                    > > I'll get a copy.
                                    > >
                                    > > Steve
                                    > >
                                    > > --
                                    > > Steve Wolstenholme Neural Planner Software
                                    > >
                                    > > EasyNN-plus. The easy way to build neural networks.
                                    > > http://www.easynn.com
                                    > >
                                    >
                                    > [Non-text portions of this message have been removed]
                                    >
                                    >


                                    [Non-text portions of this message have been removed]
                                  • John Williams
                                    ... There are plenty text editors out there that are free. Look here. Note Tab Light http://www.notetab.com/download.php
                                    Message 17 of 24 , Jan 16, 2007
                                    • 0 Attachment
                                      P. Dickens wrote:
                                      > The short answer is, not long enough for me to take much notice, longer
                                      > than a few minutes though.
                                      > I will test it again in a second instance and time it.
                                      >
                                      > Here is the URL for TextPad
                                      > http://www.textpad.com/

                                      There are plenty text editors out there that are free.

                                      Look here. Note Tab Light

                                      http://www.notetab.com/download.php
                                    • Steve Wolstenholme
                                      ... Are any of the columns text or image mode? Steve -- Steve Wolstenholme Neural Planner Software EasyNN-plus. The easy way to build neural networks.
                                      Message 18 of 24 , Jan 17, 2007
                                      • 0 Attachment
                                        On Tue, 16 Jan 2007 21:16:44 +0000, you wrote:

                                        >320,129 rows with 10 columns.
                                        >Around 45' , but bear in mind I had another instance running and yet
                                        >another checking.

                                        Are any of the columns text or image mode?

                                        Steve

                                        --
                                        Steve Wolstenholme Neural Planner Software

                                        EasyNN-plus. The easy way to build neural networks.
                                        http://www.easynn.com
                                      • P. Dickens
                                        No, they are all numeric data and treated as real . Except the first naming column for each row and the first name row for each column imported with the data.
                                        Message 19 of 24 , Jan 17, 2007
                                        • 0 Attachment
                                          No, they are all numeric data and treated as 'real'.
                                          Except the first naming column for each row and the first name row for
                                          each column imported with the data.

                                          Steve Wolstenholme wrote:

                                          > On Tue, 16 Jan 2007 21:16:44 +0000, you wrote:
                                          >
                                          > >320,129 rows with 10 columns.
                                          > >Around 45' , but bear in mind I had another instance running and yet
                                          > >another checking.
                                          >
                                          > Are any of the columns text or image mode?
                                          >
                                          > Steve
                                          >
                                          > --
                                          > Steve Wolstenholme Neural Planner Software
                                          >
                                          > EasyNN-plus. The easy way to build neural networks.
                                          > http://www.easynn.com
                                          >


                                          [Non-text portions of this message have been removed]
                                        • Steve Wolstenholme
                                          Any missing values? If so, what is the Missing data action? I still guessing! Steve -- Steve Wolstenholme Neural Planner Software EasyNN-plus. The easy
                                          Message 20 of 24 , Jan 18, 2007
                                          • 0 Attachment
                                            Any "missing" values? If so, what is the Missing data action?

                                            I still guessing!

                                            Steve

                                            --
                                            Steve Wolstenholme Neural Planner Software

                                            EasyNN-plus. The easy way to build neural networks.
                                            http://www.easynn.com
                                          • P. Dickens
                                            It recommends to do a grid check. When it finds problem lines e.g. those out of the range they are converted. Problem lines such as conflicts I give a
                                            Message 21 of 24 , Jan 18, 2007
                                            • 0 Attachment
                                              It recommends to do a grid check. When it finds problem lines e.g.
                                              those out of the range they are converted. Problem lines such as
                                              conflicts I give a blanket delete instruction. There were about 2.25k
                                              lines deleted which relative to the quantity the network, I wasn't
                                              bothered about.

                                              There was one line with a problem title (typing error) which I dealt
                                              with in TextPad.

                                              Not to sure as to whether I have actually answered your question.

                                              P. Dickens

                                              Steve Wolstenholme wrote:

                                              > Any "missing" values? If so, what is the Missing data action?
                                              >
                                              > I still guessing!
                                              >
                                              > Steve
                                              >
                                              > --
                                              > Steve Wolstenholme Neural Planner Software
                                              >
                                              > EasyNN-plus. The easy way to build neural networks.
                                              > http://www.easynn.com
                                              >


                                              [Non-text portions of this message have been removed]
                                            • P. Dickens
                                              It recommends to do a grid check. When it finds problem lines e.g. those out of the range they are converted. Problem lines such as conflicts I give a
                                              Message 22 of 24 , Jan 18, 2007
                                              • 0 Attachment
                                                It recommends to do a grid check. When it finds problem lines e.g.
                                                those out of the range they are converted. Problem lines such as
                                                conflicts I give a
                                                blanket delete instruction. There were about 2.25k lines deleted which
                                                relative to the quantity the network, I wasn't bothered about.

                                                There was one line with a problem title (typing error) which I dealt
                                                with in TextPad.

                                                Not too sure as to whether I have actually answered your question.

                                                P. Dickens


                                                Steve Wolstenholme wrote:

                                                > Any "missing" values? If so, what is the Missing data action?
                                                >
                                                > I still guessing!
                                                >
                                                > Steve
                                                >
                                                > --
                                                > Steve Wolstenholme Neural Planner Software
                                                >
                                                > EasyNN-plus. The easy way to build neural networks.
                                                > http://www.easynn.com
                                                >


                                                [Non-text portions of this message have been removed]
                                              • Steve Wolstenholme
                                                ... I was wondering if any of the Input values are set to ? meaning missing data. The substitutes for missing data are in Tools Options ... especially
                                                Message 23 of 24 , Jan 18, 2007
                                                • 0 Attachment
                                                  On Thu, 18 Jan 2007 22:48:49 +0000, you wrote:

                                                  >It recommends to do a grid check. When it finds problem lines e.g.
                                                  >those out of the range they are converted. Problem lines such as
                                                  >conflicts I give a blanket delete instruction. There were about 2.25k
                                                  >lines deleted which relative to the quantity the network, I wasn't
                                                  >bothered about.
                                                  >
                                                  >There was one line with a problem title (typing error) which I dealt
                                                  >with in TextPad.
                                                  >
                                                  >Not to sure as to whether I have actually answered your question.
                                                  >
                                                  >P. Dickens

                                                  I was wondering if any of the Input values are set to '?' meaning
                                                  missing data. The substitutes for missing data are in Tools > Options
                                                  > Missing data. Median is the default and that's fast but the others,
                                                  especially average, takes a bit of time to calculate. Every row has to
                                                  be searched to find highest, lowest and calculate the average. With
                                                  all the rows you are using that could be a lot of time.

                                                  As a matter of interest why are you using so much data? Have you tried
                                                  a much lower number of examples, selected at random?

                                                  Steve

                                                  --
                                                  Steve Wolstenholme Neural Planner Software

                                                  EasyNN-plus. The easy way to build neural networks.
                                                  http://www.easynn.com
                                                • P. Dickens
                                                  No, there were no ? nor missing data. The data rows are carefully vetted using a set of tools to ensure each line is complete. As with all calculation tools
                                                  Message 24 of 24 , Jan 19, 2007
                                                  • 0 Attachment
                                                    No, there were no "?" nor missing data. The data rows are carefully
                                                    vetted using a set of tools to ensure each line is complete. As with all
                                                    calculation tools I always insist on the use of triangulation to verify
                                                    accuracy and completeness. I have seen too many spreadsheets with
                                                    appalling errors, that would in the event have disastrous results, if
                                                    they had not been thoroughly checked first. And I know that a piece of
                                                    research done by Deloitte, I believe, (one of the big 7 as they were at
                                                    the time, a few years ago) did an audit of spreadsheets within a sample
                                                    of companies they were auditing and found fundamental errors in
                                                    something like 70%+ of the sheets they checked. And these were sheets
                                                    upon which manufacturing, stock control and costing processes were
                                                    dependent. So my rules are always 1. Triangulation, 2. Triangulation
                                                    and 3. Triangulation.

                                                    No other error correction other than that, that I mentioned was done.
                                                    So I cannot help you on that score.

                                                    Data sets:: We have been using various data sets going from a few
                                                    hundred through a few thousand up to the levels currently being used.
                                                    In some cases to test comparative results from different networks. In
                                                    others to improve results and yet in others to improve the overall
                                                    spread of data being used in the network. The results are always
                                                    compared to smaller samples previously used, and the proof of the
                                                    pudding of course is in the Generalisation.

                                                    Some of the data solutions have proved quite intractable, and I can tell
                                                    you for a fact that Kolmogorov was wrong with regard to the number of
                                                    hidden layers.

                                                    P. Dickens.

                                                    Steve Wolstenholme wrote:

                                                    > On Thu, 18 Jan 2007 22:48:49 +0000, you wrote:
                                                    >
                                                    > >It recommends to do a grid check. When it finds problem lines e.g.
                                                    > >those out of the range they are converted. Problem lines such as
                                                    > >conflicts I give a blanket delete instruction. There were about 2.25k
                                                    >
                                                    > >lines deleted which relative to the quantity the network, I wasn't
                                                    > >bothered about.
                                                    > >
                                                    > >There was one line with a problem title (typing error) which I dealt
                                                    > >with in TextPad.
                                                    > >
                                                    > >Not to sure as to whether I have actually answered your question.
                                                    > >
                                                    > >P. Dickens
                                                    >
                                                    > I was wondering if any of the Input values are set to '?' meaning
                                                    > missing data. The substitutes for missing data are in Tools > Options
                                                    > > Missing data. Median is the default and that's fast but the others,
                                                    > especially average, takes a bit of time to calculate. Every row has to
                                                    >
                                                    > be searched to find highest, lowest and calculate the average. With
                                                    > all the rows you are using that could be a lot of time.
                                                    >
                                                    > As a matter of interest why are you using so much data? Have you tried
                                                    >
                                                    > a much lower number of examples, selected at random?
                                                    >
                                                    > Steve
                                                    >
                                                    > --
                                                    > Steve Wolstenholme Neural Planner Software
                                                    >
                                                    > EasyNN-plus. The easy way to build neural networks.
                                                    > http://www.easynn.com
                                                    >


                                                    [Non-text portions of this message have been removed]
                                                  Your message has been successfully submitted and would be delivered to recipients shortly.