Loading ...
Sorry, an error occurred while loading the content.

How to get a variable to work, and not a file

Expand Messages
  • C Brito
    Friends, I use several clips to pickup a line from a document, and work in that line. Sometimes I use GetFileFirst, and other no, for exemple to pickup a line
    Message 1 of 6 , Oct 14, 2013
    • 0 Attachment
      Friends,

      I use several clips to pickup a line from a document, and work in that line. Sometimes I use GetFileFirst, and other no, for exemple to pickup a line that are not a file, as a list of restaurants NAMES.

      An example:

      ^!Set %count%=0
      ^!SetDelimiter ";"

      :loop
      ^!Inc %count%

      ^!Open c:\testing.csv
      ^!SetWordWrap off
      ^!Set %fieldtowork%=^$GetField(^%count%;2)$
      ^!close c:\testing.csv,discard

      ^!If ^%fieldtowork"=9999 final

      DO SOMETING

      ^!Goto loop

      :final

      As you see, I open the file c:\testing.csv, get the field 2 (to work in), and later I close

      Of course, this work. But imagine that in my c:\testing.csv have 50,000 lines. I will open and close c:\testing.csv 50,000 times, bad for the timing and bad for the harddisk heads.

      I am asking if exist a way to use a variable instead of the c:\testing.csv, but with the same text (same lines). In other works, to use a variable instead of a file to open and close.

      Thanks!
    • Ian NTnerd
      I often load a whole file into an array of lines. ^!setlistdelimiter ^p ^!setarray %line%=^$getfiletext(c: testing.csv)$ Then just process each line. I then
      Message 2 of 6 , Oct 14, 2013
      • 0 Attachment
        I often load a whole file into an array of lines.

        ^!setlistdelimiter ^p
        ^!setarray %line%=^$getfiletext(c:\testing.csv)$

        Then just process each line. I then parse the line into fields assuming Tab delimited.

        ^!setlistdelimiter ^t
        ^!setarray %field%=^%line^%count%%
        ^!Set %fieldtowork%=^%field2%


        That should be faster than your approach.

        But for 50,000 lines of data, NT is not the fastest scripting language.

        Can you use RegEx on your data? That would be the fastest by a long way.

        Ian

        On 14/10/2013 11:24 PM, C Brito wrote:
         
        Friends,

        I use several clips to pickup a line from a document, and work in that line. Sometimes I use GetFileFirst, and other no, for exemple to pickup a line that are not a file, as a list of restaurants NAMES.

        An example:

        ^!Set %count%=0
        ^!SetDelimiter ";"

        :loop
        ^!Inc %count%

        ^!Open c:\testing.csv
        ^!SetWordWrap off
        ^!Set %fieldtowork%=^$GetField(^%count%;2)$
        ^!close c:\testing.csv,discard

        ^!If ^%fieldtowork"=9999 final

        DO SOMETING

        ^!Goto loop

        :final

        As you see, I open the file c:\testing.csv, get the field 2 (to work in), and later I close

        Of course, this work. But imagine that in my c:\testing.csv have 50,000 lines. I will open and close c:\testing.csv 50,000 times, bad for the timing and bad for the harddisk heads.

        I am asking if exist a way to use a variable instead of the c:\testing.csv, but with the same text (same lines). In other works, to use a variable instead of a file to open and close.

        Thanks!

      • Axel Berger
        ... There is only one thing that makes NT really slow in my experience: InsertText in a loop. It begins quite fast and then freezes more and more. With several
        Message 3 of 6 , Oct 14, 2013
        • 0 Attachment
          Ian NTnerd wrote:
          > But for 50,000 lines of data, NT is not the fastest scripting
          > language.

          There is only one thing that makes NT really slow in my experience:
          InsertText in a loop. It begins quite fast and then freezes more and
          more. With several thousand lines it goes up to a whole second per
          single Insert command and running time approaches an hour. Unfortunately
          I see no way to eliminate the loop.

          :Loop
          ^!Find "(?s)HREF="(http.+?)"(.*?)>" RS1
          ^!IfError Weiter
          ^!InsertSelect ^$StrReplace(" ";"~";"^$GetSelection$";FALSE;FALSE)$
          ^!InsertSelect ^$FileToUrl("^$GetSelection$")$
          ^!InsertText ^$StrReplace("%25(?=\d{2})";"%";"^$GetSelection$";RA)$
          ^!Goto Loop
          :Weiter

          Axel
        • joy8388608
          As pointed out, using an array is often a good solution, but I d like to point out you do not have to open and close your data file every time you read from
          Message 4 of 6 , Oct 15, 2013
          • 0 Attachment

            As pointed out, using an array is often a good solution, but I'd like to point out you do not have to open and close your data file every time you read from it.


            Without going into details, you could read the data file into a NoteTab doc then do ^!Set %fieldtowork%=^$GetField(^%count%;2)$ You can now do what you need to which includes switching to another NoteTab document you already have open. Then switch back to the document with the data and get fieldtowork from the NEXT line in the same way. Repeat until done.


            If you turn off ScreenUpdating, it should be quite fast.


            Joy 



            ---In ntb-clips@yahoogroups.com, <backup2abet@...> wrote:

            Friends,

            I use several clips to pickup a line from a document, and work in that line. Sometimes I use GetFileFirst, and other no, for exemple to pickup a line that are not a file, as a list of restaurants NAMES.

            An example:

            ^!Set %count%=0
            ^!SetDelimiter ";"

            :loop
            ^!Inc %count%

            ^!Open c:\testing.csv
            ^!SetWordWrap off
            ^!Set %fieldtowork%=^$GetField(^%count%;2)$
            ^!close c:\testing.csv,discard

            ^!If ^%fieldtowork"=9999 final

            DO SOMETING

            ^!Goto loop

            :final

            As you see, I open the file c:\testing.csv, get the field 2 (to work in), and later I close

            Of course, this work. But imagine that in my c:\testing.csv have 50,000 lines. I will open and close c:\testing.csv 50,000 times, bad for the timing and bad for the harddisk heads.

            I am asking if exist a way to use a variable instead of the c:\testing.csv, but with the same text (same lines). In other works, to use a variable instead of a file to open and close.

            Thanks!
          • Art Kocsis
            ... You don t need to eliminate the loop, just the InsertText. I do this all the time, not because of speed but for convenience - such as putting line counts
            Message 5 of 6 , Oct 16, 2013
            • 0 Attachment
              At 10/14/2013 11:43 PM, Axel wrote:
              >Ian NTnerd wrote:
              >> But for 50,000 lines of data, NT is not the fastest scripting
              >> language.
              >
              >There is only one thing that makes NT really slow in my experience:
              >InsertText in a loop. It begins quite fast and then freezes more and
              >more. With several thousand lines it goes up to a whole second per
              >single Insert command and running time approaches an hour. Unfortunately
              >I see no way to eliminate the loop.
              >
              >:Loop
              >^!Find "(?s)HREF="(http.+?)"(.*?)>" RS1
              >^!IfError Weiter
              >^!InsertSelect ^$StrReplace(" ";"~";"^$GetSelection$";FALSE;FALSE)$
              >^!InsertSelect ^$FileToUrl("^$GetSelection$")$
              >^!InsertText ^$StrReplace("%25(?=\d{2})";"%";"^$GetSelection$";RA)$
              >^!Goto Loop
              >:Weiter

              You don't need to eliminate the loop, just the InsertText. I do this all the time, not because of speed but for convenience - such as putting line counts in headers. Yours isn't as trivial as what I typically do but with a little rewrite it will work the same:

              The first thing I would try is to simply change all of your Inserts to in memory commands (and add ^!SetScreenUpdate Off - makes a BIG difference!):

              ^!SetScreenUpdate Off
              :Loop
              ^!Find "(?s)HREF="(http.+?)"(.*?)>" IRS1
              ^!IfError Weiter
              ^!Set %str1%=^$GetSelection$
              ^!Set %str2%=^$StrReplace(" ";"~";"^%str1%";FALSE;FALSE)$
              ^!Set %str2%=^$FileToUrl("^%str2")$
              ^!Set %str2%=^$StrReplace("%25(?=\d{2})";"%";"^%str2%";RA)$
              ^!Replace "^%str1%" >> "^%str2%" H
              ^!Goto Loop
              :Weiter

              Since the FileToUrl command always increases the selection size the final Replace command requires shifting all subsequent text to created the added space for the replacement. For very large documents, this could add significant time. A more drastic rewrite to alternatively concatenate the skipped text and the changed text to an in memory string as you process the document and then doing a single replace for the should help:

              ;Process
              ; Save initial conditions; initialize working variables
              ; location1 = beginning of last skipped text,
              ; location2 = beginning of last found URL
              ; location3 = end of last found URL
              ; Find next URL, save cursor location2 (cursor is at beginning of selection)
              ; Jump to end of URL selection, save cursor location3
              ; Jump back to location1, select skipped text & append to ^%text%
              ; Jump back to location2, reselect URL, process and append to ^%text%
              ; Jump to end of URL selection, update location1
              ; Repeat for all URLs
              ; Select text from end of last URL selection and append to ^%text%
              ; Replace all text from initial condition to Doc_End with ^%text%
              ;
              ^!SetScreenUpdate Off
              ^!set %row0%=^$GetRow$
              ^!set %col0%=^$GetCol$
              ^!set %row1%=^%row0%
              ^!set %col1%=^%col0%
              ^!set %text%=^%Empty%
              :Loop
              ^!Find "(?s)HREF="(http.+?)"(.*?)>" IRS1
              ^!IfError Weiter
              ^!set %row2%=^$GetRow$
              ^!set %col2%=^$GetCol$
              ^!Jump Select_End
              ^!set %row3%=^$GetRow$
              ^!set %col3%=^$GetCol$
              ^!SetCursor ^%row1%:^%col1%
              ^!SelectTo ^%row2%:^%col2%
              ^!set %text%=^%text%^$GetSelection$
              ^!Jump Select_End
              ^!SelectTo ^%row3%:^%col3%
              ^!Set %str1%=^$GetSelection$
              ^!Set %str2%=^$StrReplace(" ";"~";"^%str1%";FALSE;FALSE)$
              ^!Set %str2%=^$FileToUrl("^%str2")$
              ^!Set %str2%=^$StrReplace("%25(?=\d{2})";"%";"^%str2%";RA)$
              ^!set %text%=^%text%^%str2%
              ^!Jump Select_End
              ^!set %row1%=^$GetRow$
              ^!set %col1%=^$GetCol$
              ^!Goto Loop
              :Weiter

              ^!Jump Doc_End
              ^!SelectTo ^%row1%:^%col1%
              ^!set %text%=^%text%^$GetSelection$
              ^!Jump Doc_End
              ^!SelectTo ^%row0%:^%col0%
              ^!InsertText ^%text%
              ;#################### End of Code #################

              Axel:
              Since the open doc is not modified until the very end, an
              easy way to test the clip is to change the last command
              to ^!TextToFile "Filename" ^%text% and compare to original
              Duh!!! Or just save a copy of the original and compare!
              Without your source files I couldn't check execution times but
              it would be interesting to see if there is a significant
              time difference between the two clips for large files.
              Wrap the clips with ^!Set %bgn%=^$GetTime(hh-nn-ss),
              ^!Set %end%=... commands and let us know.
              Did you really want the Find URL pattern to be case sensitive?
              That will miss href=... and HTTP strings. If so, remove the "I"
              You may want to modify your search pattern to include the single
              quote in addition to the double quote you have now.
              Also, you may want to include image links (src="http...) as well.

              HTH,

              Art
            • Ian NTnerd
              ... Good to know that trick. Had never noticed that little gem. ^!SetScreenUpdate [On|Off](added in v4.52) I will try write to a new open file, instead of to
              Message 6 of 6 , Oct 16, 2013
              • 0 Attachment
                On 16/10/2013 12:56 AM, mycroftj@... wrote:


                If you turn off ScreenUpdating, it should be quite fast.


                Joy

                Good to know that trick. Had never noticed that little gem.

                ^!SetScreenUpdate [On|Off] (added in v4.52)

                I will try write to a new open file, instead of to disk. It may be faster.

                Thanks

                Ian
              Your message has been successfully submitted and would be delivered to recipients shortly.