Loading ...
Sorry, an error occurred while loading the content.

[jasspa] ME as download-monster

Expand Messages
  • Detlef Groth
    Sometimes it is difficult to download entire webpages full of advertisment but with many next -buttons. Usually other download-programs are failing here. With
    Message 1 of 2 , Feb 16, 2000
    • 0 Attachment
      Sometimes it is difficult to download entire webpages full of advertisment
      but with many 'next'-buttons. Usually other download-programs are failing
      here. With 11 lines of code ME (with URL-support) beats them all:

      Try find-file "http://www.itknowledge.com/reference/archive/1571690638/ch01/030-033.html"

      and run the macro:

      define-macro download-next
      set-variable $debug 1 ; just for debugging
      set-variable #l0 1
      !while &less #l0 20 ; a loop for twenty pages
      html-show-buffer
      beginning-of-buffer
      !force search-forward " lm[nN]ext le"
      4 backward-char
      0 html-process-link
      set-variable #l0 &add #l0 1 ; incrementing by 1
      !done
      !emacro


      if you are happy, press '!' to avoid debugging and ME will fetch almost
      twenty pages, if they are containing the proper 'next'-links. If something
      goes wrong just press "CTRL+G". If you can't work with the mouse later
      just press again "!" !!!!

      Happy fetching...

      best regards,
      Detlef
    • Detlef Groth
      Hello, the Escape-sequences were lost in space, so it is better you use regexpression e = Escape: Here is revised version of download-next and an other even
      Message 2 of 2 , Feb 18, 2000
      • 0 Attachment
        Hello,

        the Escape-sequences were lost in space, so it is better you use
        regexpression "\e" = Escape:
        Here is revised version of download-next and an other even better
        download-all to access html-pages from a table of contents.


        Best regards, Detlef


        define-macro download-next
        ; starts downloading a thread
        ; if an html-file contains Next-buttons
        ; find-the html-file
        ; execute download-next
        set-variable $debug 1
        set-variable #l0 1
        !while &less #l0 20 ; twenty "next" searches
        html-show-buffer ; process the buffer html-to-ehf
        beginning-of-buffer
        !force search-forward "\elm[nN]ext\ele" ; find-link
        4 backward-char
        0 html-process-link
        set-variable #l0 &add #l0 1 ; increment 1
        !done
        !emacro

        define-macro download-all
        ; starts downloading from a table of contents
        ; after executing html-process-link (C-c C-h)"
        ; Go on the first link you want to fetch
        ; then execute download-all
        ; if all is OK press "!" to end debugging
        set-variable $debug 1
        set-variable #l1 $buffer-bname
        set-variable #l0 1
        !while &less #l0 20 ; for twenty times execution
        ; search first link in ehf-format
        !force search-forward "\ele"
        4 backward-char
        0 html-process-link
        ; go back to table of contents
        find-buffer #l1
        ; move cursor over the already fetched link
        !force search-forward "\ele"
        2 forward-char
        set-variable #l0 &add #l0 1 ; incrementor 1
        !done
        !emacro


        define-macro download-save
        ; for saving of fetched links, this works only
        ; properly if a history-less ME-session
        ; was running
        ; it simply saves twenty buffers
        ; in the ME directory (compare download all)
        set-variable $debug 1
        set-variable #l0 1
        !while &less #l0 20
        ; renaming annd saving the file
        set-variable #l2 $buffer-bname
        set-variable $buffer-fname #l2
        write-buffer #l2
        next-buffer
        set-variable #l0 &add #l0 1
        !done
        !emacro


        detlef groth<dgrot-@...> wrote:
        original article:http://www.egroups.com/group/jasspa/?start=203
        >
        >
        > Sometimes it is difficult to download entire webpages full of
        advertisment
        > but with many 'next'-buttons. Usually other download-programs are
        failing
        > here. With 11 lines of code ME (with URL-support) beats them all:
        >
        > Try find-file "http://www.itknowledge.com/reference/archive/157169063
        8/ch01/030-033.html"
        >
        > and run the macro:
        >
        > define-macro download-next
        > set-variable $debug 1 ; just for debugging
        > set-variable #l0 1
        > !while &less #l0 20 ; a loop for twenty pages
        > html-show-buffer
        > beginning-of-buffer
        > !force search-forward " lm[nN]ext le"
        > 4 backward-char
        > 0 html-process-link
        > set-variable #l0 &add #l0 1 ; incrementing by 1
        > !done
        > !emacro
        >
        >
        > if you are happy, press '!' to avoid debugging and ME will fetch
        almost
        > twenty pages, if they are containing the proper 'next'-links. If
        something
        > goes wrong just press "CTRL+G". If you can't work with the mouse later
        > just press again "!" !!!!
        >
        > Happy fetching...
        >
        > best regards,
        > Detlef
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.