Loading ...
Sorry, an error occurred while loading the content.

Automate operation on web sites

Expand Messages
  • mauro.list
    Hi all, I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser. I would need to: -login
    Message 1 of 11 , Jan 27, 2010
    • 0 Attachment
      Hi all,
      I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser.
      I would need to:
      -login
      -select some radio buttons
      -fill some field in the web pages
      -submit the forms
      -check the result

      Is perl the right choice to do this?
      Is there any CPAN module which could help me doing this?

      Thank You

      Best Regards
    • Jeff Soules
      Hello, ... Perl can be a good choice for this. It depends on your environment and how fancy you re hoping to get. ... I d start with WWW::Mechanize [1]. It s
      Message 2 of 11 , Jan 27, 2010
      • 0 Attachment
        Hello,

        > I would like to automate, by building some ad-hoc scripts, some operations I have
        > to do at the moment using a web browser.
        > I would need to:
        > -login
        > -select some radio buttons
        > -fill some field in the web pages
        > -submit the forms
        > -check the result
        >
        > Is perl the right choice to do this?

        Perl can be a good choice for this. It depends on your environment
        and how fancy you're hoping to get.

        > Is there any CPAN module which could help me doing this?

        I'd start with WWW::Mechanize [1]. It's a great tool, except that it
        doesn't handle JavaScript. There are other options depending on your
        environment that you can use if that won't do; I'm hoping that someone
        else will also reply to this thread and I'll get to learn about even
        more of them!

        Best,
        Jeff



        [1] http://search.cpan.org/~petdance/WWW-Mechanize-1.60/lib/WWW/Mechanize.pm


        On Wed, Jan 27, 2010 at 9:08 AM, mauro.list <mauro.list@...> wrote:
        >
        >
        >
        > Hi all,
        > I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser.
        > I would need to:
        > -login
        > -select some radio buttons
        > -fill some field in the web pages
        > -submit the forms
        > -check the result
        >
        > Is perl the right choice to do this?
        > Is there any CPAN module which could help me doing this?
        >
        > Thank You
        >
        > Best Regards
        >
        >
      • merlyn@stonehenge.com
        ... mauro I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser. mauro I would need to:
        Message 3 of 11 , Jan 27, 2010
        • 0 Attachment
          >>>>> "mauro" == mauro list <mauro.list@...> writes:

          mauro> I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser.
          mauro> I would need to:
          mauro> -login
          mauro> -select some radio buttons
          mauro> -fill some field in the web pages
          mauro> -submit the forms
          mauro> -check the result

          mauro> Is perl the right choice to do this?
          mauro> Is there any CPAN module which could help me doing this?

          This is exactly what a spammer would do to spam forum posts.

          Since we don't know you, helping you may be harming the internet.

          Please elaborate some more about what you are trying to do, and
          why it has to be done through a web interface instead of talking
          directly to your databases, etc.

          --
          Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
          <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
          Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
          See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
        • Shlomi Fish
          ... If you need JavaScript, you can use WWW-Selenium: http://search.cpan.org/perldoc?WWW::Selenium ... Permanent link:
          Message 4 of 11 , Jan 27, 2010
          • 0 Attachment
            On Wednesday 27 Jan 2010 16:18:07 Jeff Soules wrote:
            > Hello,
            >
            > > I would like to automate, by building some ad-hoc scripts, some
            > > operations I have to do at the moment using a web browser.
            > > I would need to:
            > > -login
            > > -select some radio buttons
            > > -fill some field in the web pages
            > > -submit the forms
            > > -check the result
            > >
            > > Is perl the right choice to do this?
            >
            > Perl can be a good choice for this. It depends on your environment
            > and how fancy you're hoping to get.
            >
            > > Is there any CPAN module which could help me doing this?
            >
            > I'd start with WWW::Mechanize [1]. It's a great tool, except that it
            > doesn't handle JavaScript. There are other options depending on your
            > environment that you can use if that won't do; I'm hoping that someone
            > else will also reply to this thread and I'll get to learn about even
            > more of them!
            >

            If you need JavaScript, you can use WWW-Selenium:

            http://search.cpan.org/perldoc?WWW::Selenium

            > Best,
            > Jeff
            >
            >
            >
            > [1]
            > http://search.cpan.org/~petdance/WWW-Mechanize-1.60/lib/WWW/Mechanize.pm
            >

            Permanent link: http://search.cpan.org/dist/WWW-Mechanize/ .

            Regards,

            Shlomi Fish

            --
            -----------------------------------------------------------------
            Shlomi Fish http://www.shlomifish.org/
            http://www.shlomifish.org/humour/ways_to_do_it.html

            Deletionists delete Wikipedia articles that they consider lame.
            Chuck Norris deletes deletionists whom he considers lame.

            Please reply to list if it's a mailing list post - http://shlom.in/reply .
          • Mauro Gatti
            I need to automate an application deploy on websphere. I can access to websphere admin console only (I have no direct access to websphere servers) a I would
            Message 5 of 11 , Jan 27, 2010
            • 0 Attachment
              I need to automate an application deploy on websphere.
              I can access
              to websphere admin console only (I have no direct access to websphere
              servers) a I would like to automate what I have to do manually
              following the deploy procedure suggested in the console:
              -login to console
              -chose the file to upload
              -setup some configuration parameters
              -continue the process
              -confirm
              -check the result.

              The process can take long time and it's very boring. Moreover it happens to do this more then once a day.




              ________________________________
              From: Randal L. Schwartz <merlyn@...>
              To: mauro.list <mauro.list@...>
              Cc: perl-beginner@yahoogroups.com
              Sent: Wed, January 27, 2010 3:44:43 PM
              Subject: Re: [PBML] Automate operation on web sites

              >>>>> "mauro" == mauro list <mauro.list@...> writes:

              mauro> I would like to automate, by building some ad-hoc scripts, some operations I have to do at the moment using a web browser.
              mauro> I would need to:
              mauro> -login
              mauro> -select some radio buttons
              mauro> -fill some field in the web pages
              mauro> -submit the forms
              mauro> -check the result

              mauro> Is perl the right choice to do this?
              mauro> Is there any CPAN module which could help me doing this?

              This is exactly what a spammer would do to spam forum posts.

              Since we don't know you, helping you may be harming the internet.

              Please elaborate some more about what you are trying to do, and
              why it has to be done through a web interface instead of talking
              directly to your databases, etc.

              --
              Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
              <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
              Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
              See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion





              [Non-text portions of this message have been removed]
            • Shlomi Fish
              Hi Randal! ... There is no need for such tone. Implying that mauro may be a spammer is not likely to make him want to learn more about Perl 5. There are many
              Message 6 of 11 , Jan 27, 2010
              • 0 Attachment
                Hi Randal!

                On Wednesday 27 Jan 2010 16:44:43 merlyn@... wrote:
                > >>>>> "mauro" == mauro list <mauro.list@...> writes:
                > mauro> I would like to automate, by building some ad-hoc scripts, some
                > operations I have to do at the moment using a web browser. mauro> I would
                > need to:
                > mauro> -login
                > mauro> -select some radio buttons
                > mauro> -fill some field in the web pages
                > mauro> -submit the forms
                > mauro> -check the result
                >
                > mauro> Is perl the right choice to do this?
                > mauro> Is there any CPAN module which could help me doing this?
                >
                > This is exactly what a spammer would do to spam forum posts.
                >
                > Since we don't know you, helping you may be harming the internet.
                >

                There is no need for such tone. Implying that mauro may be a spammer is not
                likely to make him want to learn more about Perl 5. There are many legitimate
                uses to the use case that they described , and it is unlikely that a spammer
                would ask for help on such a high-profile, public, archived and open, mailing
                list. (And if he does, he's stupid and will be sacked soon.)

                > Please elaborate some more about what you are trying to do, and
                > why it has to be done through a web interface instead of talking
                > directly to your databases, etc.

                Well, if I had a web-app, I wouldn't want to futz directly with the database,
                because the logic behind the forms may be complex and communicating with the
                database directly may cause errors.

                Nevertheless, it is possible one's web-application provides a web API (REST,
                SOAP, XML-RPC, etc.) that can be used instead of HTML-scraping. If so, one
                should investigate how to use it instead.

                Regards,

                Shlomi Fish

                --
                -----------------------------------------------------------------
                Shlomi Fish http://www.shlomifish.org/
                Rethinking CPAN - http://shlom.in/rethinking-cpan

                Deletionists delete Wikipedia articles that they consider lame.
                Chuck Norris deletes deletionists whom he considers lame.

                Please reply to list if it's a mailing list post - http://shlom.in/reply .
              • merlyn@stonehenge.com
                ... Shlomi and it is unlikely that a spammer would ask for help on such a Shlomi high-profile, public, archived and open, mailing list. (And if he Shlomi
                Message 7 of 11 , Jan 27, 2010
                • 0 Attachment
                  >>>>> "Shlomi" == Shlomi Fish <shlomif@...> writes:

                  Shlomi> and it is unlikely that a spammer would ask for help on such a
                  Shlomi> high-profile, public, archived and open, mailing list. (And if he
                  Shlomi> does, he's stupid and will be sacked soon.)

                  Maybe he is, and I'm helping him get sacked before damage is done.

                  Truly, if you don't see the potential damage you do by naively answering
                  questions for dangerous people, you are part of the problem, not part
                  of the solution.

                  I *must* question whether my knowledge is likely being used for good or evil,
                  or I'm being unethical. I ask you to do the same.

                  Perl is not trying to win any popularity contests. But Perl *can* damage the
                  internet. Be very careful with your tools and your assistance.

                  --
                  Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
                  <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                  Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
                  See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
                • Shlomi Fish
                  ... Knowledge can be used for good or evil. So can many other tools. You can club someone with a hammer or devastate their house. You can use a web browser
                  Message 8 of 11 , Jan 27, 2010
                  • 0 Attachment
                    On Wednesday 27 Jan 2010 18:45:26 merlyn@... wrote:
                    > >>>>> "Shlomi" == Shlomi Fish <shlomif@...> writes:
                    > Shlomi> and it is unlikely that a spammer would ask for help on such a
                    > Shlomi> high-profile, public, archived and open, mailing list. (And if he
                    > Shlomi> does, he's stupid and will be sacked soon.)
                    >
                    > Maybe he is, and I'm helping him get sacked before damage is done.
                    >
                    > Truly, if you don't see the potential damage you do by naively answering
                    > questions for dangerous people, you are part of the problem, not part
                    > of the solution.
                    >
                    > I *must* question whether my knowledge is likely being used for good or
                    > evil, or I'm being unethical. I ask you to do the same.
                    >

                    Knowledge can be used for good or evil. So can many other tools. You can club
                    someone with a hammer or devastate their house. You can use a web browser such
                    as Firefox to perform all kinds of nasty activity such as sabotaging wikis or
                    posting hate-speech. You can use perl for doing all sorts of nasty activity
                    such as sending spam. That does not make perl, Firefox or hammers evil - they
                    are just tools.

                    I've written a Perl tutorial and placed it online on
                    http://www.shlomifish.org/lecture/Perl/Newbies/ under the public domain.
                    People can use it to learn Perl and then go on and use that knowledge to
                    become spammers, crackers, or other criminals. Should I make sure that only
                    benevolent people can read my tutorial, so it won't be abused? People can also
                    read your books or other technical and use that knowledge for ill. Trying to
                    enforce an ethical use of our tools and knowledge is ridiculous.

                    > Perl is not trying to win any popularity contests. But Perl *can* damage
                    > the internet. Be very careful with your tools and your assistance.

                    The way I see it, we are trying to make Perl as popular as possible. Perl can
                    damage the Internet, but so can Python, Ruby, C, Lua, Lisp, Haskell, Erlang,
                    Squeak Smalltalk or whatever, and spammers, crackers, phishers and other
                    Internet criminals already have all the necessary knowledge for doing that
                    without our help. But it's no excuse to accuse innocent people who want to get
                    help of having malicious intentions, because this is quite likely going to
                    deter them from wanting to learn Perl.

                    If someone asks for my help and I help him with doing a task that can have
                    many legitimate uses, and they turn out to be a criminal, then I cannot be
                    accused of aiding crime. On the other hand, if someone asks for my help and
                    he's innocent, and I've started interrogating him about his intentions, then I
                    am guilty of not making him feel welcome.

                    Regards,

                    Shlomi Fish

                    --
                    -----------------------------------------------------------------
                    Shlomi Fish http://www.shlomifish.org/
                    "Star Trek: We, the Living Dead" - http://shlom.in/st-wtld

                    Deletionists delete Wikipedia articles that they consider lame.
                    Chuck Norris deletes deletionists whom he considers lame.

                    Please reply to list if it's a mailing list post - http://shlom.in/reply .
                  • Mauro Gatti
                    Just to close the issue... I posted a question which I definitely think doesn t brake any rule of the forum. People is free to answer, ask for clarification or
                    Message 9 of 11 , Jan 27, 2010
                    • 0 Attachment
                      Just to close the issue...
                      I posted a question which I definitely think doesn't brake any rule of the forum.
                      People is free to answer, ask for clarification or be silent.
                      To imply I am a spammer isn't polite of course but I can survive.
                      Anyway I gave my reason without telling anything more (up to now).
                      So now, I would like to come back to the matter of my question full stop.

                      Thanks

                      Regards

                      Mauro






                      ________________________________
                      From: Randal L. Schwartz <merlyn@...>
                      To: Shlomi Fish <shlomif@...>
                      Cc: perl-beginner@yahoogroups.com; mauro.list <mauro.list@...>
                      Sent: Wed, January 27, 2010 5:45:26 PM
                      Subject: Re: [PBML] Automate operation on web sites

                      >>>>> "Shlomi" == Shlomi Fish <shlomif@...> writes:

                      Shlomi> and it is unlikely that a spammer would ask for help on such a
                      Shlomi> high-profile, public, archived and open, mailing list. (And if he
                      Shlomi> does, he's stupid and will be sacked soon.)

                      Maybe he is, and I'm helping him get sacked before damage is done.

                      Truly, if you don't see the potential damage you do by naively answering
                      questions for dangerous people, you are part of the problem, not part
                      of the solution.

                      I *must* question whether my knowledge is likely being used for good or evil,
                      or I'm being unethical. I ask you to do the same.

                      Perl is not trying to win any popularity contests. But Perl *can* damage the
                      internet. Be very careful with your tools and your assistance.

                      --
                      Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
                      <merlyn@...> <URL:http://www.stonehenge.com/merlyn/>
                      Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
                      See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion





                      [Non-text portions of this message have been removed]
                    • Shlomi Fish
                      ... Hi Mauro! See what Jeff and I told you about it here: * http://tech.groups.yahoo.com/group/perl-beginner/message/26775 *
                      Message 10 of 11 , Jan 27, 2010
                      • 0 Attachment
                        On Wednesday 27 Jan 2010 19:57:44 Mauro Gatti wrote:
                        > Just to close the issue...
                        > I posted a question which I definitely think doesn't brake any rule of the
                        > forum. People is free to answer, ask for clarification or be silent.
                        > To imply I am a spammer isn't polite of course but I can survive.
                        > Anyway I gave my reason without telling anything more (up to now).
                        > So now, I would like to come back to the matter of my question full stop.
                        >

                        Hi Mauro!

                        See what Jeff and I told you about it here:

                        * http://tech.groups.yahoo.com/group/perl-beginner/message/26775

                        * http://tech.groups.yahoo.com/group/perl-beginner/message/26777

                        Regards,

                        Shlomi Fish


                        --
                        -----------------------------------------------------------------
                        Shlomi Fish http://www.shlomifish.org/
                        List of Portability Libraries - http://shlom.in/port-libs

                        Deletionists delete Wikipedia articles that they consider lame.
                        Chuck Norris deletes deletionists whom he considers lame.

                        Please reply to list if it's a mailing list post - http://shlom.in/reply .
                      • Mauro Gatti
                        Hi all, I m trying to install via CPAN shell the module DBD::JDBC but I get this error: CPAN.pm: Going to build V/VI/VIZDOM/DBD-JDBC-0.71.tar.gz Checking if
                        Message 11 of 11 , May 19, 2010
                        • 0 Attachment
                          Hi all,
                          I'm trying to install via CPAN shell the module DBD::JDBC but I get this error:

                          CPAN.pm: Going to build V/VI/VIZDOM/DBD-JDBC-0.71.tar.gz

                          Checking if your kit is complete...
                          Looks good
                          Multiple copies of Driver.xst found in: /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/DBI/ /usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/auto/DBI/ at Makefile.PL line 22
                          Using DBI 1.611 (for perl 5.008008 on x86_64-linux-thread-multi) installed in /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/DBI/
                          Writing Makefile for DBD::JDBC
                          cp JDBC.pod blib/lib/DBD/JDBC.pod
                          cp JDBC.pm blib/lib/DBD/JDBC.pm
                          cp lib/Bundle/DBD/JDBC.pm blib/lib/Bundle/DBD/JDBC.pm
                          Manifying blib/man3/DBD::JDBC.3pm
                          Manifying blib/man3/Bundle::DBD::JDBC.3pm
                          VIZDOM/DBD-JDBC-0.71.tar.gz
                          /usr/bin/make -- OK
                          Running make test
                          PERL_DL_NONLAZY=1 CLASSPATH=dbd_jdbc.jar:t/hsqldb/hsqldb-1.8.0.2.jar:t/hsqldb/log4j-1.2.13.jar: /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
                          t/01_env.t ...... ok
                          t/02_connect.t .. 3/5 DBD::JDBC::dr connect warning: No driver found for jdbc:hsqldb:file:t/hsqldb/testdb at /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/DBI.pm line 655.

                          # Failed test 'connected'
                          # at t/02_connect.t line 40.
                          # Connection error: No driver found found



                          Could you help me to solve this issue?

                          Thank You

                          Regards

                          Mauro





                          [Non-text portions of this message have been removed]
                        Your message has been successfully submitted and would be delivered to recipients shortly.