Loading ...
Sorry, an error occurred while loading the content.

Re: [agile-testing] Load/Performance testing

Expand Messages
  • Basim Baassiri
    Hi Chris You mentioned a rails app as part of the application under test I found this blog posting that might be of help
    Message 1 of 28 , Aug 9, 2012
    View Source
    • 0 Attachment
      Hi Chris
      You mentioned a rails app as part of the application under test

      I found this blog posting that might be of help

      Cheers

      Basim

      On Wed, Aug 8, 2012 at 1:16 PM, Basim Baassiri <basim@...> wrote:
      Hi Chris
      Jmeter does offer cookie support

      My approach, is to model the user scenarios in something like UCML

      Then design the tests to satisfy the user scenarios

      another approach is to read in username and passwords and then read in the urls your want to traverse 
      How to read from CSV with Jmeter

      Reading urls from a file

      Once you have your urls as variables, it can be passed to the http sampler and you can further read post data from a file and send it along to the server

      I hope this info might help

      Basim

      On Wed, Aug 8, 2012 at 12:26 PM, Christopher Johnston <chrisj@...> wrote:
       

      Hi Basim,


      Thanks for the link. I am looking into jMeter's access log sampler, but I have a few questions.

      With Nginx logs there is no POST data and the majority of our application is entering data. Is POST data a standard part of an Apache log or do most performance tests only rely on GET requests?

      Also, our entire application is behind user authentication and uses https. As such, without POST data, I am not sure how this method of testing would work at all without the ability to handle cookies in jMeter and have the test log into the application?

      Cheers,
      Chris


      On 2012-08-07, at 11:08 PM, Basim Baassiri wrote:

       

      Hi Chris

      Did you come across this posting in your searching

      Basim

      On Tue, Aug 7, 2012 at 5:49 PM, Christopher Johnston <chrisj@...> wrote:
       

      Hello,

      Sorry for digging up an old topic, but I have been doing a lot of research on this topic and have been playing around with the tools mentioned (Gatling and jMeter).

      The specific thing that I have been looking for over the last few days is a method of feeding log files (nginx or rails) into a tool like jMeter or Gatling. The idea is to snapshot the database and then run a days or weeks worth of real world data through the application as the performance test.

      Is this possible to do? I have yet to find any good information on how to accomplish this, so decided to ask here in are someone has done it and not blogged about it.

      Cheers,
      Chris Johnston


      On 2012-07-19, at 12:26 AM, arnaud.oqube@... wrote:

       

      Basim Baassiri <basim@...> writes:

      > Hi Chris

      Did you have a look at Gatling (http://gatling-tool.org/)?

      Regards,
      --
      Arnaud








    • Tanvir-ul-Islam
      My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium
      Message 2 of 28 , Aug 13, 2012
      View Source
      • 0 Attachment
        My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
         
        Tanvir

        "Consistency is an essential thing in acquiring knowledge"



      • Hassan Murtaza
        HI Tanvir, I have used both of these tools quite extensively. Telerik is very flashy and seems to do everything. However as a matter of fact its very high
        Message 3 of 28 , Aug 13, 2012
        View Source
        • 0 Attachment
          HI Tanvir,

          I have used both of these tools quite extensively. Telerik is very flashy and seems to do everything. However as a matter of fact its very high maintenance tool and very very bugy and unstable. After using for one year i literally throw telerik out the window !
          Selenium on the other hand is awesome, its easy to get used to and can be used with any languauge. I am currently using it with MBUnit and love it. However it does have limitations like silverlight didnot work((atleat last time i checked). I think you will also have issues making it work with third party controls. 
          You should investigate one other option i.e. Watir, although i have never worked with it but lot of people on this forum have.

          Let me know if you need more info.
          Hassan

          On Mon, Aug 13, 2012 at 7:15 PM, Tanvir-ul-Islam <tanvir6@...> wrote:
           

          My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
           
          Tanvir

          "Consistency is an essential thing in acquiring knowledge"




        • Nathan
          Tanvir, Selenium is completely free to use. Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii
          Message 4 of 28 , Aug 14, 2012
          View Source
          • 0 Attachment
            Tanvir,

            Selenium is completely free to use.

            Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii framework available.

            My advice to you is to take some time to try out both of these tools and write some tests which will fully exercise areas of your site which you are going to be testing.

            You should probably time-box a couple of days to evaluate each of the tools.

            This way you are going to find out if the tools will actually be able to deliver what you need from them.

            It's also worth noting that, just because you application is classic asp or .net, that doesn't mean you need to choose a .net testing framework. You could just as easily use Selenium with Ruby as an example.

            For what it's worth, I have used both Selenium and WebAii extensively, and I would choose Selenium every time.

            Good luck..

            --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
            >
            > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
            >  
            > Tanvir
            >
            > "Consistency is an essential thing in acquiring knowledge"
            >
            >
            > ________________________________
            >
          • Lisa Crispin
            I recommend you start by first deciding how YOUR team wants your tests to look. What format works for you? A DSL? BDD style? Tabular? Who will be specifying
            Message 5 of 28 , Aug 14, 2012
            View Source
            • 0 Attachment
              I recommend you start by first deciding how YOUR team wants your tests to look. What format works for you? A DSL? BDD style? Tabular? Who will be specifying test cases? Who will automate the tests?

              Then, look around for drivers such as Selenium and frameworks that can drive Selenium, plus commercial tools like Telerik, and as Nathan recommends, try them out. Do side-by-side comparisons.

              The expense of automating tests is not the initial cost of acquiring the tools. It's in learning how to design them for maintainability so that you get a good return on investment over the long term.

              If I were you, I would first read Markus Gaertner's new book _ATDD by Example_, which will walk you through the process of deciding how you want to automate tests and how to choose the tools.
              -- Lisa

              On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
               

              Tanvir,

              Selenium is completely free to use.

              Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii framework available.

              My advice to you is to take some time to try out both of these tools and write some tests which will fully exercise areas of your site which you are going to be testing.

              You should probably time-box a couple of days to evaluate each of the tools.

              This way you are going to find out if the tools will actually be able to deliver what you need from them.

              It's also worth noting that, just because you application is classic asp or .net, that doesn't mean you need to choose a .net testing framework. You could just as easily use Selenium with Ruby as an example.

              For what it's worth, I have used both Selenium and WebAii extensively, and I would choose Selenium every time.

              Good luck..



              --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
              >
              > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
              >  
              > Tanvir
              >
              > "Consistency is an essential thing in acquiring knowledge"
              >
              >
              > ________________________________
              >




              --
              Lisa Crispin
              Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
              Contributor to _Beautiful Testing_ (O'Reilly 2009)
              http://lisacrispin.com
              @lisacrispin on Twitter
              http://entaggle.com/lisacrispin

            • neena
              Hi Lisa, would you have comparison sheet? Thanks, Neena
              Message 6 of 28 , Aug 14, 2012
              View Source
              • 0 Attachment
                Hi Lisa,
                would you have comparison sheet?
                Thanks,
                Neena

                --- In agile-testing@yahoogroups.com, Lisa Crispin <lisa.crispin@...> wrote:
                >
                > I recommend you start by first deciding how YOUR team wants your tests to
                > look. What format works for you? A DSL? BDD style? Tabular? Who will be
                > specifying test cases? Who will automate the tests?
                >
                > Then, look around for drivers such as Selenium and frameworks that can
                > drive Selenium, plus commercial tools like Telerik, and as Nathan
                > recommends, try them out. Do side-by-side comparisons.
                >
                > The expense of automating tests is not the initial cost of acquiring the
                > tools. It's in learning how to design them for maintainability so that you
                > get a good return on investment over the long term.
                >
                > If I were you, I would first read Markus Gaertner's new book _ATDD by
                > Example_, which will walk you through the process of deciding how you want
                > to automate tests and how to choose the tools.
                > -- Lisa
                >
                > On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
                >
                > > **
                > >
                > >
                > > Tanvir,
                > >
                > > Selenium is completely free to use.
                > >
                > > Telerik provide a 30 day trial of their full testing framework. They also
                > > have free version of their WebAii framework available.
                > >
                > > My advice to you is to take some time to try out both of these tools and
                > > write some tests which will fully exercise areas of your site which you are
                > > going to be testing.
                > >
                > > You should probably time-box a couple of days to evaluate each of the
                > > tools.
                > >
                > > This way you are going to find out if the tools will actually be able to
                > > deliver what you need from them.
                > >
                > > It's also worth noting that, just because you application is classic asp
                > > or .net, that doesn't mean you need to choose a .net testing framework. You
                > > could just as easily use Selenium with Ruby as an example.
                > >
                > > For what it's worth, I have used both Selenium and WebAii extensively, and
                > > I would choose Selenium every time.
                > >
                > > Good luck..
                > >
                > >
                > > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                > > >
                > > > My company is recently moving towards agile and automation. We have a
                > > web-based application and we are currently debating whether to use Telerik
                > > or Selenium for automation purposes. Most of our site uses legacy asp code
                > > and recently we are shifting to .net. We are dealing with frames 100% of
                > > times, our html structures are pretty complex with lot of nested tables,
                > > frames, forms. Based on your experiences, if any of you have used both or
                > > one of the tools(or any other tools) if you can discuss some of the pros
                > > and cons of the tools that would be great.
                > > >
                > > > Tanvir
                > > >
                > > > "Consistency is an essential thing in acquiring knowledge"
                > > >
                > > >
                > > > ________________________________
                > > >
                > >
                > >
                > >
                >
                >
                >
                > --
                > Lisa Crispin
                > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
                > and Agile Teams_ (Addison-Wesley 2009)
                > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                > http://lisacrispin.com
                > @lisacrispin on Twitter
                > http://entaggle.com/lisacrispin
                >
              • Tanvir-ul-Islam
                @Lisa: Thank you for your recommendations. Currently we are following the tabular format. However since we are still moving into agile process we haven t had
                Message 7 of 28 , Aug 14, 2012
                View Source
                • 0 Attachment
                  @Lisa: Thank you for your recommendations. Currently we are following the tabular format. However since we are still moving into agile process we haven't had a planning meeting yet as to who is going to specify the test cases or do the user stories and if it is the same person that would automate the tests. 

                  I would definitely refer to the book you recommended and do more research/side-by-side comparison on the both tools and decide the tool that would provide the best ROI over long term.

                  @Nathan: Your input is greatly appreciated! Thank you.
                   
                  Tanvir-ul Islam

                  Cell: 917-639-7964
                  Skype id: tanvir.islam06

                  "Consistency is an essential thing in acquiring knowledge"


                  From: Lisa Crispin <lisa.crispin@...>
                  To: agile-testing@yahoogroups.com
                  Sent: Tuesday, August 14, 2012 9:07:29 AM
                  Subject: Re: [agile-testing] Re: Telerik vs Selenium

                   
                  I recommend you start by first deciding how YOUR team wants your tests to look. What format works for you? A DSL? BDD style? Tabular? Who will be specifying test cases? Who will automate the tests?

                  Then, look around for drivers such as Selenium and frameworks that can drive Selenium, plus commercial tools like Telerik, and as Nathan recommends, try them out. Do side-by-side comparisons.

                  The expense of automating tests is not the initial cost of acquiring the tools. It's in learning how to design them for maintainability so that you get a good return on investment over the long term.

                  If I were you, I would first read Markus Gaertner's new book _ATDD by Example_, which will walk you through the process of deciding how you want to automate tests and how to choose the tools.
                  -- Lisa

                  On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
                   
                  Tanvir,

                  Selenium is completely free to use.

                  Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii framework available.

                  My advice to you is to take some time to try out both of these tools and write some tests which will fully exercise areas of your site which you are going to be testing.

                  You should probably time-box a couple of days to evaluate each of the tools.

                  This way you are going to find out if the tools will actually be able to deliver what you need from them.

                  It's also worth noting that, just because you application is classic asp or .net, that doesn't mean you need to choose a .net testing framework. You could just as easily use Selenium with Ruby as an example.

                  For what it's worth, I have used both Selenium and WebAii extensively, and I would choose Selenium every time.

                  Good luck..


                  --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                  >
                  > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                  >  
                  > Tanvir
                  >
                  > "Consistency is an essential thing in acquiring knowledge"
                  >
                  >
                  > ________________________________
                  >




                  --
                  Lisa Crispin
                  Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
                  Contributor to _Beautiful Testing_ (O'Reilly 2009)
                  http://lisacrispin.com
                  @lisacrispin on Twitter
                  http://entaggle.com/lisacrispin



                • Lisa Crispin
                  Hi Neena, The Agile Alliance functional test tools group had a spreasheet comparing tools, but Google blocked access to it, I don t know what the status is
                  Message 8 of 28 , Aug 14, 2012
                  View Source
                  • 0 Attachment
                    Hi Neena,
                    The Agile Alliance functional test tools group had a spreasheet comparing tools, but Google blocked access to it, I don't know what the status is right now.
                    -- Lisa

                    On Tue, Aug 14, 2012 at 7:46 AM, neena <g_neena@...> wrote:
                     

                    Hi Lisa,
                    would you have comparison sheet?
                    Thanks,
                    Neena



                    --- In agile-testing@yahoogroups.com, Lisa Crispin <lisa.crispin@...> wrote:
                    >
                    > I recommend you start by first deciding how YOUR team wants your tests to
                    > look. What format works for you? A DSL? BDD style? Tabular? Who will be
                    > specifying test cases? Who will automate the tests?
                    >
                    > Then, look around for drivers such as Selenium and frameworks that can
                    > drive Selenium, plus commercial tools like Telerik, and as Nathan
                    > recommends, try them out. Do side-by-side comparisons.
                    >
                    > The expense of automating tests is not the initial cost of acquiring the
                    > tools. It's in learning how to design them for maintainability so that you
                    > get a good return on investment over the long term.
                    >
                    > If I were you, I would first read Markus Gaertner's new book _ATDD by
                    > Example_, which will walk you through the process of deciding how you want
                    > to automate tests and how to choose the tools.
                    > -- Lisa
                    >
                    > On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
                    >
                    > > **

                    > >
                    > >
                    > > Tanvir,
                    > >
                    > > Selenium is completely free to use.
                    > >
                    > > Telerik provide a 30 day trial of their full testing framework. They also
                    > > have free version of their WebAii framework available.
                    > >
                    > > My advice to you is to take some time to try out both of these tools and
                    > > write some tests which will fully exercise areas of your site which you are
                    > > going to be testing.
                    > >
                    > > You should probably time-box a couple of days to evaluate each of the
                    > > tools.
                    > >
                    > > This way you are going to find out if the tools will actually be able to
                    > > deliver what you need from them.
                    > >
                    > > It's also worth noting that, just because you application is classic asp
                    > > or .net, that doesn't mean you need to choose a .net testing framework. You
                    > > could just as easily use Selenium with Ruby as an example.
                    > >
                    > > For what it's worth, I have used both Selenium and WebAii extensively, and
                    > > I would choose Selenium every time.
                    > >
                    > > Good luck..
                    > >
                    > >
                    > > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                    > > >
                    > > > My company is recently moving towards agile and automation. We have a
                    > > web-based application and we are currently debating whether to use Telerik
                    > > or Selenium for automation purposes. Most of our site uses legacy asp code
                    > > and recently we are shifting to .net. We are dealing with frames 100% of
                    > > times, our html structures are pretty complex with lot of nested tables,
                    > > frames, forms. Based on your experiences, if any of you have used both or
                    > > one of the tools(or any other tools) if you can discuss some of the pros
                    > > and cons of the tools that would be great.
                    > > >
                    > > > Tanvir
                    > > >
                    > > > "Consistency is an essential thing in acquiring knowledge"
                    > > >
                    > > >
                    > > > ________________________________
                    > > >
                    > >
                    > >
                    > >
                    >
                    >
                    >
                    > --
                    > Lisa Crispin
                    > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
                    > and Agile Teams_ (Addison-Wesley 2009)
                    > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                    > http://lisacrispin.com
                    > @lisacrispin on Twitter
                    > http://entaggle.com/lisacrispin
                    >




                    --
                    Lisa Crispin
                    Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
                    Contributor to _Beautiful Testing_ (O'Reilly 2009)
                    http://lisacrispin.com
                    @lisacrispin on Twitter
                    http://entaggle.com/lisacrispin

                  • Nathan
                    Tanvir, If you are new to agile, and testing in an agile environment, then I would also recommend reading Lisa s book - Agile Testing: A Practical Guide for
                    Message 9 of 28 , Aug 14, 2012
                    View Source
                    • 0 Attachment
                      Tanvir,

                      If you are new to agile, and testing in an agile environment, then I would also recommend reading Lisa's book - "Agile Testing: A Practical Guide for Testers and Agile Teams".

                      Nathan..


                      --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                      >
                      > @Lisa: Thank you for your recommendations. Currently we are following the tabular format. However since we are still moving into agile process we haven't had a planning meeting yet as to who is going to specify the test cases or do the user stories and if it is the same person that would automate the tests. 
                      >
                      >
                      > I would definitely refer to the book you recommended and do more research/side-by-side comparison on the both tools and decide the tool that would provide the best ROI over long term.
                      >
                      > @Nathan: Your input is greatly appreciated! Thank you.
                      >  
                      > Tanvir-ul Islam
                      >
                      > Cell: 917-639-7964
                      > Skype id: tanvir.islam06
                      >
                      > "Consistency is an essential thing in acquiring knowledge"
                      >
                      >
                      > ________________________________
                      > From: Lisa Crispin <lisa.crispin@...>
                      > To: agile-testing@yahoogroups.com
                      > Sent: Tuesday, August 14, 2012 9:07:29 AM
                      > Subject: Re: [agile-testing] Re: Telerik vs Selenium
                      >
                      >
                      >  
                      > I recommend you start by first deciding how YOUR team wants your tests to look. What format works for you? A DSL? BDD style? Tabular? Who will be specifying test cases? Who will automate the tests?
                      >
                      > Then, look around for drivers such as Selenium and frameworks that can drive Selenium, plus commercial tools like Telerik, and as Nathan recommends, try them out. Do side-by-side comparisons.
                      >
                      > The expense of automating tests is not the initial cost of acquiring the tools. It's in learning how to design them for maintainability so that you get a good return on investment over the long term.
                      >
                      > If I were you, I would first read Markus Gaertner's new book _ATDD by Example_, which will walk you through the process of deciding how you want to automate tests and how to choose the tools.
                      > -- Lisa
                      >
                      >
                      > On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
                      >
                      >
                      > > 
                      > >Tanvir,
                      > >
                      > >Selenium is completely free to use.
                      > >
                      > >Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii framework available.
                      > >
                      > >My advice to you is to take some time to try out both of these tools and write some tests which will fully exercise areas of your site which you are going to be testing.
                      > >
                      > >You should probably time-box a couple of days to evaluate each of the tools.
                      > >
                      > >This way you are going to find out if the tools will actually be able to deliver what you need from them.
                      > >
                      > >It's also worth noting that, just because you application is classic asp or .net, that doesn't mean you need to choose a .net testing framework. You could just as easily use Selenium with Ruby as an example.
                      > >
                      > >For what it's worth, I have used both Selenium and WebAii extensively, and I would choose Selenium every time.
                      > >
                      > >Good luck..
                      > >
                      > >
                      > >--- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                      > >>
                      > >> My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                      > >>  
                      > >> Tanvir
                      > >>
                      > >> "Consistency is an essential thing in acquiring knowledge"
                      > >>
                      > >>
                      > >> ________________________________
                      > >>
                      > >
                      > >
                      >
                      >
                      > --
                      > Lisa Crispin
                      > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
                      > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                      > http://lisacrispin.com
                      > @lisacrispin on Twitter
                      > http://entaggle.com/lisacrispin
                      >
                    • Tanvir-ul-Islam
                      Nathan, We acquired that book already :) Thank you.   Tanvir-ul Islam Cell: 917-639-7964 Skype id: tanvir.islam06 Consistency is an essential thing in
                      Message 10 of 28 , Aug 14, 2012
                      View Source
                      • 0 Attachment
                        Nathan,
                        We acquired that book already :) Thank you.
                         
                        Tanvir-ul Islam

                        Cell: 917-639-7964
                        Skype id: tanvir.islam06

                        "Consistency is an essential thing in acquiring knowledge"


                        From: Nathan <nathan_bain@...>
                        To: agile-testing@yahoogroups.com
                        Sent: Tuesday, August 14, 2012 4:26:09 PM
                        Subject: [agile-testing] Re: Telerik vs Selenium

                         
                        Tanvir,

                        If you are new to agile, and testing in an agile environment, then I would also recommend reading Lisa's book - "Agile Testing: A Practical Guide for Testers and Agile Teams".

                        Nathan..

                        --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                        >
                        > @Lisa: Thank you for your recommendations. Currently we are following the tabular format. However since we are still moving into agile process we haven't had a planning meeting yet as to who is going to specify the test cases or do the user stories and if it is the same person that would automate the tests. 
                        >
                        >
                        > I would definitely refer to the book you recommended and do more research/side-by-side comparison on the both tools and decide the tool that would provide the best ROI over long term.
                        >
                        > @Nathan: Your input is greatly appreciated! Thank you.
                        >  
                        > Tanvir-ul Islam
                        >
                        > Cell: 917-639-7964
                        > Skype id: tanvir.islam06
                        >
                        > "Consistency is an essential thing in acquiring knowledge"
                        >
                        >
                        > ________________________________
                        > From: Lisa Crispin <lisa.crispin@...>
                        > To: agile-testing@yahoogroups.com
                        > Sent: Tuesday, August 14, 2012 9:07:29 AM
                        > Subject: Re: [agile-testing] Re: Telerik vs Selenium
                        >
                        >
                        >  
                        > I recommend you start by first deciding how YOUR team wants your tests to look. What format works for you? A DSL? BDD style? Tabular? Who will be specifying test cases? Who will automate the tests?
                        >
                        > Then, look around for drivers such as Selenium and frameworks that can drive Selenium, plus commercial tools like Telerik, and as Nathan recommends, try them out. Do side-by-side comparisons.
                        >
                        > The expense of automating tests is not the initial cost of acquiring the tools. It's in learning how to design them for maintainability so that you get a good return on investment over the long term.
                        >
                        > If I were you, I would first read Markus Gaertner's new book _ATDD by Example_, which will walk you through the process of deciding how you want to automate tests and how to choose the tools.
                        > -- Lisa
                        >
                        >
                        > On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@...> wrote:
                        >
                        >
                        > > 
                        > >Tanvir,
                        > >
                        > >Selenium is completely free to use.
                        > >
                        > >Telerik provide a 30 day trial of their full testing framework. They also have free version of their WebAii framework available.
                        > >
                        > >My advice to you is to take some time to try out both of these tools and write some tests which will fully exercise areas of your site which you are going to be testing.
                        > >
                        > >You should probably time-box a couple of days to evaluate each of the tools.
                        > >
                        > >This way you are going to find out if the tools will actually be able to deliver what you need from them.
                        > >
                        > >It's also worth noting that, just because you application is classic asp or .net, that doesn't mean you need to choose a .net testing framework. You could just as easily use Selenium with Ruby as an example.
                        > >
                        > >For what it's worth, I have used both Selenium and WebAii extensively, and I would choose Selenium every time.
                        > >
                        > >Good luck..
                        > >
                        > >
                        > >--- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                        > >>
                        > >> My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                        > >>  
                        > >> Tanvir
                        > >>
                        > >> "Consistency is an essential thing in acquiring knowledge"
                        > >>
                        > >>
                        > >> ________________________________
                        > >>
                        > >
                        > >
                        >
                        >
                        > --
                        > Lisa Crispin
                        > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
                        > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                        > http://lisacrispin.com
                        > @lisacrispin on Twitter
                        > http://entaggle.com/lisacrispin
                        >



                      • neena
                        Thanks Lisa!
                        Message 11 of 28 , Aug 14, 2012
                        View Source
                        • 0 Attachment
                          Thanks Lisa!

                          --- In agile-testing@yahoogroups.com, Lisa Crispin <lisa.crispin@...> wrote:
                          >
                          > Hi Neena,
                          > The Agile Alliance functional test tools group had a spreasheet comparing
                          > tools, but Google blocked access to it, I don't know what the status is
                          > right now.
                          > -- Lisa
                          >
                          > On Tue, Aug 14, 2012 at 7:46 AM, neena <g_neena@...> wrote:
                          >
                          > > **
                          > >
                          > >
                          > > Hi Lisa,
                          > > would you have comparison sheet?
                          > > Thanks,
                          > > Neena
                          > >
                          > >
                          > > --- In agile-testing@yahoogroups.com, Lisa Crispin <lisa.crispin@>
                          > > wrote:
                          > > >
                          > > > I recommend you start by first deciding how YOUR team wants your tests to
                          > > > look. What format works for you? A DSL? BDD style? Tabular? Who will be
                          > > > specifying test cases? Who will automate the tests?
                          > > >
                          > > > Then, look around for drivers such as Selenium and frameworks that can
                          > > > drive Selenium, plus commercial tools like Telerik, and as Nathan
                          > > > recommends, try them out. Do side-by-side comparisons.
                          > > >
                          > > > The expense of automating tests is not the initial cost of acquiring the
                          > > > tools. It's in learning how to design them for maintainability so that
                          > > you
                          > > > get a good return on investment over the long term.
                          > > >
                          > > > If I were you, I would first read Markus Gaertner's new book _ATDD by
                          > > > Example_, which will walk you through the process of deciding how you
                          > > want
                          > > > to automate tests and how to choose the tools.
                          > > > -- Lisa
                          > > >
                          > > > On Tue, Aug 14, 2012 at 6:34 AM, Nathan <nathan_bain@> wrote:
                          > > >
                          > > > > **
                          > >
                          > > > >
                          > > > >
                          > > > > Tanvir,
                          > > > >
                          > > > > Selenium is completely free to use.
                          > > > >
                          > > > > Telerik provide a 30 day trial of their full testing framework. They
                          > > also
                          > > > > have free version of their WebAii framework available.
                          > > > >
                          > > > > My advice to you is to take some time to try out both of these tools
                          > > and
                          > > > > write some tests which will fully exercise areas of your site which
                          > > you are
                          > > > > going to be testing.
                          > > > >
                          > > > > You should probably time-box a couple of days to evaluate each of the
                          > > > > tools.
                          > > > >
                          > > > > This way you are going to find out if the tools will actually be able
                          > > to
                          > > > > deliver what you need from them.
                          > > > >
                          > > > > It's also worth noting that, just because you application is classic
                          > > asp
                          > > > > or .net, that doesn't mean you need to choose a .net testing
                          > > framework. You
                          > > > > could just as easily use Selenium with Ruby as an example.
                          > > > >
                          > > > > For what it's worth, I have used both Selenium and WebAii extensively,
                          > > and
                          > > > > I would choose Selenium every time.
                          > > > >
                          > > > > Good luck..
                          > > > >
                          > > > >
                          > > > > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@>
                          > > wrote:
                          > > > > >
                          > > > > > My company is recently moving towards agile and automation. We have a
                          > > > > web-based application and we are currently debating whether to use
                          > > Telerik
                          > > > > or Selenium for automation purposes. Most of our site uses legacy asp
                          > > code
                          > > > > and recently we are shifting to .net. We are dealing with frames 100%
                          > > of
                          > > > > times, our html structures are pretty complex with lot of nested
                          > > tables,
                          > > > > frames, forms. Based on your experiences, if any of you have used both
                          > > or
                          > > > > one of the tools(or any other tools) if you can discuss some of the
                          > > pros
                          > > > > and cons of the tools that would be great.
                          > > > > >
                          > > > > > Tanvir
                          > > > > >
                          > > > > > "Consistency is an essential thing in acquiring knowledge"
                          > > > > >
                          > > > > >
                          > > > > > ________________________________
                          > > > > >
                          > > > >
                          > > > >
                          > > > >
                          > > >
                          > > >
                          > > >
                          > > > --
                          > > > Lisa Crispin
                          > > > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for
                          > > Testers
                          > > > and Agile Teams_ (Addison-Wesley 2009)
                          > > > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                          > > > http://lisacrispin.com
                          > > > @lisacrispin on Twitter
                          > > > http://entaggle.com/lisacrispin
                          > > >
                          > >
                          > >
                          > >
                          >
                          >
                          >
                          > --
                          > Lisa Crispin
                          > Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers
                          > and Agile Teams_ (Addison-Wesley 2009)
                          > Contributor to _Beautiful Testing_ (O'Reilly 2009)
                          > http://lisacrispin.com
                          > @lisacrispin on Twitter
                          > http://entaggle.com/lisacrispin
                          >
                        • jimholmesoh
                          Hassan, I m the evangelist for Telerik s Test Studio. I m very sorry to hear you had such a bad experience. If you d be willing, I d love to hear more details
                          Message 12 of 28 , Aug 15, 2012
                          View Source
                          • 0 Attachment
                            Hassan,

                            I'm the evangelist for Telerik's Test Studio. I'm very sorry to hear you had such a bad experience.

                            If you'd be willing, I'd love to hear more details about what went so wrong for you. I'm not trying to sway you back to Telerik; I'm trying to ensure we know where we missed, and how we can help other users. You can respond here, or directly in mail: jim.holmes@...

                            Jim

                            --- In agile-testing@yahoogroups.com, Hassan Murtaza <hassan.murtaza@...> wrote:
                            >
                            > HI Tanvir,
                            >
                            > I have used both of these tools quite extensively. Telerik is very flashy
                            > and seems to do everything. However as a matter of fact its very high
                            > maintenance tool and very very bugy and unstable. After using for one year
                            > i literally throw telerik out the window !
                            > Selenium on the other hand is awesome, its easy to get used to and can be
                            > used with any languauge. I am currently using it with MBUnit and love it.
                            > However it does have limitations like silverlight didnot work((atleat last
                            > time i checked). I think you will also have issues making it work with
                            > third party controls.
                            > You should investigate one other option i.e. Watir, although i have never
                            > worked with it but lot of people on this forum have.
                            >
                            > Let me know if you need more info.
                            > Hassan
                            >
                            > On Mon, Aug 13, 2012 at 7:15 PM, Tanvir-ul-Islam <tanvir6@...> wrote:
                            >
                            > > **
                            > >
                            > >
                            > > My company is recently moving towards agile and automation. We have a
                            > > web-based application and we are currently debating whether to use Telerik
                            > > or Selenium for automation purposes. Most of our site uses legacy asp code
                            > > and recently we are shifting to .net. We are dealing with frames 100% of
                            > > times, our html structures are pretty complex with lot of nested tables,
                            > > frames, forms. Based on your experiences, if any of you have used both or
                            > > one of the tools(or any other tools) if you can discuss some of the pros
                            > > and cons of the tools that would be great.
                            > >
                            > > Tanvir
                            > >
                            > > "Consistency is an essential thing in acquiring knowledge"
                            > >
                            > > ------------------------------
                            > >
                            > >
                            > >
                            >
                          • jimholmesoh
                            Tanvir, I m the evangelist for Telerik s Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I m obviously a bit biased in
                            Message 13 of 28 , Aug 15, 2012
                            View Source
                            • 0 Attachment
                              Tanvir,

                              I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.

                              At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/

                              You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.

                              I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.

                              I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)

                              At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.

                              Jim

                              (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)


                              --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                              >
                              > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                              >  
                              > Tanvir
                              >
                              > "Consistency is an essential thing in acquiring knowledge"
                              >
                              >
                              > ________________________________
                              >
                            • tsagilecg1
                              Jim, I m Tanvir s Coworker. I ve worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful
                              Message 14 of 28 , Aug 15, 2012
                              View Source
                              • 0 Attachment
                                Jim,
                                I'm Tanvir's Coworker. I've worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful simulating actions on the browser.

                                I think the question we are struggling with is, what supporting tools can be used with selenium vs maybe the telerik frame work or watin(are there keyword driven framworks, or other approaches, are there reporting tools that can be used, and so on?). that will work best for us. Can you recommend any we can evaluate?

                                I will agree with you, the how its implemented is going to be more important than the APIs used.

                                Thanks
                                Clifton


                                --- In agile-testing@yahoogroups.com, "jimholmesoh" <jim.holmes@...> wrote:
                                >
                                > Tanvir,
                                >
                                > I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.
                                >
                                > At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/
                                >
                                > You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.
                                >
                                > I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.
                                >
                                > I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)
                                >
                                > At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.
                                >
                                > Jim
                                >
                                > (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)
                                >
                                >
                                > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                                > >
                                > > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                > >  
                                > > Tanvir
                                > >
                                > > "Consistency is an essential thing in acquiring knowledge"
                                > >
                                > >
                                > > ________________________________
                                > >
                                >
                              • Janet Gregory
                                Hi folks, A quick note from the moderators on this thread.. We don t want this to be a sales forum, but we encourage everyone to share their actual experiences
                                Message 15 of 28 , Aug 15, 2012
                                View Source
                                • 0 Attachment

                                  Hi folks,

                                   

                                  A quick note from the moderators on this thread…. 

                                   

                                  We don't want this to be a sales forum, but we encourage everyone to share their actual experiences with various tools.  I know I get a lot of questions from people on tools that I can’t answer and we have such a breadth of experience here.

                                   

                                  Janet Gregory

                                   

                                  From: agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com] On Behalf Of tsagilecg1
                                  Sent: August-15-12 9:14 AM
                                  To: agile-testing@yahoogroups.com
                                  Subject: [agile-testing] Re: Telerik vs Selenium

                                   

                                   

                                  Jim,
                                  I'm Tanvir's Coworker. I've worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful simulating actions on the browser.

                                  I think the question we are struggling with is, what supporting tools can be used with selenium vs maybe the telerik frame work or watin(are there keyword driven framworks, or other approaches, are there reporting tools that can be used, and so on?). that will work best for us. Can you recommend any we can evaluate?

                                  I will agree with you, the how its implemented is going to be more important than the APIs used.

                                  Thanks
                                  Clifton

                                  --- In agile-testing@yahoogroups.com, "jimholmesoh" <jim.holmes@...> wrote:
                                  >
                                  > Tanvir,
                                  >
                                  > I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.
                                  >
                                  > At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/
                                  >
                                  > You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.
                                  >
                                  > I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.
                                  >
                                  > I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)
                                  >
                                  > At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.
                                  >
                                  > Jim
                                  >
                                  > (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)
                                  >
                                  >
                                  > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                                  > >
                                  > > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                  > >  
                                  > > Tanvir
                                  > >
                                  > > "Consistency is an essential thing in acquiring knowledge"
                                  > >
                                  > >
                                  > > ________________________________
                                  > >
                                  >

                                • Mark Levison
                                  ... From my point of view it didn t look like Jim was selling at all. It looked like he was making it clear something that we all tend to agree on. Success or
                                  Message 16 of 28 , Aug 15, 2012
                                  View Source
                                  • 0 Attachment
                                    On Wed, Aug 15, 2012 at 11:24 AM, Janet Gregory <janet_gregory@...> wrote:
                                     

                                    Hi folks,

                                     

                                    A quick note from the moderators on this thread…. 

                                     

                                    We don't want this to be a sales forum, but we encourage everyone to share their actual experiences with various tools.  I know I get a lot of questions from people on tools that I can’t answer and we have such a breadth of experience here.


                                    From my point of view it didn't look like Jim was selling at all. It looked like he was making it clear something that we all tend to agree on. Success or failure is more about humans and less about tools. On the upside its nice to have tool vendor who cares enough about this community to subscribe to the list and comment.

                                    Cheers
                                    Mark 
                                  • Jim Holmes
                                    Janet et al., I am extraordinarily sensitive about not doing pitches, which is why I ve previously only monitored this group without subscribing. The last
                                    Message 17 of 28 , Aug 15, 2012
                                    View Source
                                    • 0 Attachment
                                      Janet et al., 

                                      I am extraordinarily sensitive about not doing pitches, which is why I've previously only monitored this group without subscribing. The last thing I want to do is irritate folks by appearing to push my product.

                                      I'd be happy to engage Clifton and Tanvir off the list if you feel that's more appropriate.

                                      Jim Holmes
                                      Evangelist, Testing Tools
                                      Telerik
                                      c:937-416-8903
                                      Twitter: @aJimHolmes
                                      Blog: http://bit.ly/JimAtTestStudio

                                      From: Janet Gregory <janet_gregory@...>
                                      Reply-To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                      Date: Wednesday, August 15, 2012 11:24 AM
                                      To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                      Subject: RE: [agile-testing] Re: Telerik vs Selenium

                                       

                                      Hi folks,

                                       

                                      A quick note from the moderators on this thread…. 

                                       

                                      We don't want this to be a sales forum, but we encourage everyone to share their actual experiences with various tools.  I know I get a lot of questions from people on tools that I can’t answer and we have such a breadth of experience here.

                                       

                                      Janet Gregory

                                       

                                      From: agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com] On Behalf Of tsagilecg1
                                      Sent: August-15-12 9:14 AM
                                      To: agile-testing@yahoogroups.com
                                      Subject: [agile-testing] Re: Telerik vs Selenium

                                       

                                       

                                      Jim,
                                      I'm Tanvir's Coworker. I've worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful simulating actions on the browser.

                                      I think the question we are struggling with is, what supporting tools can be used with selenium vs maybe the telerik frame work or watin(are there keyword driven framworks, or other approaches, are there reporting tools that can be used, and so on?). that will work best for us. Can you recommend any we can evaluate?

                                      I will agree with you, the how its implemented is going to be more important than the APIs used.

                                      Thanks
                                      Clifton

                                      --- In agile-testing@yahoogroups.com, "jimholmesoh" <jim.holmes@...> wrote:
                                      >
                                      > Tanvir,
                                      >
                                      > I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.
                                      >
                                      > At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/
                                      >
                                      > You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.
                                      >
                                      > I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.
                                      >
                                      > I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)
                                      >
                                      > At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.
                                      >
                                      > Jim
                                      >
                                      > (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)
                                      >
                                      >
                                      > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                                      > >
                                      > > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                      > >  
                                      > > Tanvir
                                      > >
                                      > > "Consistency is an essential thing in acquiring knowledge"
                                      > >
                                      > >
                                      > > ________________________________
                                      > >
                                      >

                                    • Jim Holmes
                                      To me this is a very common, very general issue here: integration between any API and the rest of the environment. Telerik s Testing Framework has a command
                                      Message 18 of 28 , Aug 15, 2012
                                      View Source
                                      • 0 Attachment
                                        To me this is a very common, very general issue here: integration between any API and the rest of the environment.

                                        Telerik's Testing Framework has a command line test runner for kicking off test runs. If you're using Nunit or something similar as your overall testing approach then you'll have our APIs wrapped inside their infrastructure and you'll use their command line runner. Ergo, the appropriate command line runner is the integration point with your CI or build system. We've got customers running Telerik tests via TFS, Hudson, Team City, and a couple other common servers.

                                        If you're looking to wrap inside Fit/Fitness, Cucumber, or something similar then you'll use the same approach as with other tooling: the backing fixtures wrap the API's calls regardless of whether you're using Telerik or WebDriver/etc. (FWIW, we have customers using our Testing Framework with Fitness and SpecFlow. I've been excited about seeing what I can do myself in those environments, but haven't yet made time for it.)

                                        The final piece is reporting, and that goes back to the environment you're in. If you're using our runner and working in TeamCity or anther build environment, then you'll need to figure out how to transform the output from the runner (ours, Nunit, MbUnit, etc.). I don’t believe we have any canned XSLT for those CI servers, but Nunit and other output is consumed straight in to the reporting infrastructure in those servers. (TFS, TeamCity, CruiseControl, etc.)

                                        Please let me know if I actually answered the questions you were asking. :)

                                        Jim Holmes
                                        Evangelist, Testing Tools
                                        Telerik
                                        c:937-416-8903
                                        Twitter: @aJimHolmes
                                        Blog: http://bit.ly/JimAtTestStudio

                                        From: tsagilecg1 <clifton.guzman@...>
                                        Reply-To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                        Date: Wednesday, August 15, 2012 11:13 AM
                                        To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                        Subject: [agile-testing] Re: Telerik vs Selenium

                                         

                                        Jim,
                                        I'm Tanvir's Coworker. I've worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful simulating actions on the browser.

                                        I think the question we are struggling with is, what supporting tools can be used with selenium vs maybe the telerik frame work or watin(are there keyword driven framworks, or other approaches, are there reporting tools that can be used, and so on?). that will work best for us. Can you recommend any we can evaluate?

                                        I will agree with you, the how its implemented is going to be more important than the APIs used.

                                        Thanks
                                        Clifton

                                        --- In agile-testing@yahoogroups.com, "jimholmesoh" <jim.holmes@...> wrote:
                                        >
                                        > Tanvir,
                                        >
                                        > I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.
                                        >
                                        > At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/
                                        >
                                        > You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.
                                        >
                                        > I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.
                                        >
                                        > I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)
                                        >
                                        > At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.
                                        >
                                        > Jim
                                        >
                                        > (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)
                                        >
                                        >
                                        > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                                        > >
                                        > > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                        > >  
                                        > > Tanvir
                                        > >
                                        > > "Consistency is an essential thing in acquiring knowledge"
                                        > >
                                        > >
                                        > > ________________________________
                                        > >
                                        >

                                      • erfanasikder
                                        Hi Tanvir , I have been using Telerik for almost a year , before purchasing it I evaluated it along with Selenium , QTP and many other tools for a long time.
                                        Message 19 of 28 , Aug 16, 2012
                                        View Source
                                        • 0 Attachment
                                          Hi Tanvir ,

                                          I have been using Telerik for almost a year , before purchasing it I evaluated it along with Selenium , QTP and many other tools for a long time.
                                          My app is a .NET app with lots of frames in each page. Ajax is used hugely everywhere , we also have some silverlight parts.
                                          Telerik can handle these easily.
                                          The main pros of it I think - the ease of use.
                                          It takes very short time to write a test case.
                                          The Test case maintenance part is also very easy. I don't have to write a new case from scratch for a small change. It has a feature which automatically updates the existing cases if anything is changed- for e.g. a variable name is changed.
                                          I do scheduled performance test of my portals on different VMs located in different countries. The performance result sheet is very useful.
                                          I also run some functional scheduled tests everyday .
                                          I have integrated it with my Visual Studio Team System. It enables me to share my tests with developers.
                                          Another pros would be helpful support.Several times they did video conferencing with me to understand my problem and they resolved it by providing a patch or showing me the way to handle it.

                                          The cons of this product is it's extremely buggy , it's heavyweight and it's not free!
                                          Another cons would be the editor does not give you freedom to modify the code behind. They have provided lots of features to make sure that you dont need to write a single line code. Therefore sometimes I feel restricted. But in that case I can take my testcase to VSTS - where I can modify it according to my need.



                                          -Regards
                                          Erfana Sikder

                                          --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                                          >
                                          > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                          >  
                                          > Tanvir
                                          >
                                          > "Consistency is an essential thing in acquiring knowledge"
                                          >
                                          >
                                          > ________________________________
                                          >
                                        • Jim Holmes
                                          Erfana, There s actually a complete, full-featured code editor in Test Studio s Standalone edition (See documentation
                                          Message 20 of 28 , Aug 16, 2012
                                          View Source
                                          • 0 Attachment
                                            Erfana,

                                            There's actually a complete, full-featured code editor in Test Studio's Standalone edition (See documentation here). You're able to completely customize or extend any test or test step as you need. There's also Test Studio Express, a plug in for Visual Studio, where you can use all the other coding tools you've got in your tool belt.

                                            We try to be very forthright that any serious automation effort /will/ require some level of code. It's simply unrealistic to approach testing without expecting to drop to code for customized teardown/setup steps, test oracles, or complex actions such as Javascript event invocation or querying in to tables.

                                            I'm sorry to hear about the stability issues you had. As I mentioned elsewhere, I'd love to get more details on your bad experience. Feel free to contact me via mail (jim.holmes@...) if you're willing to share.

                                            Jim Holmes
                                            Evangelist, Testing Tools
                                            Telerik
                                            c:937-416-8903
                                            Twitter: @aJimHolmes
                                            Blog: http://bit.ly/JimAtTestStudio

                                            From: erfanasikder <erfanasikder@...>
                                            Reply-To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                            Date: Thursday, August 16, 2012 7:50 AM
                                            To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                            Subject: [agile-testing] Re: Telerik vs Selenium

                                             



                                            Hi Tanvir ,

                                            I have been using Telerik for almost a year , before purchasing it I evaluated it along with Selenium , QTP and many other tools for a long time.
                                            My app is a .NET app with lots of frames in each page. Ajax is used hugely everywhere , we also have some silverlight parts.
                                            Telerik can handle these easily.
                                            The main pros of it I think - the ease of use.
                                            It takes very short time to write a test case.
                                            The Test case maintenance part is also very easy. I don't have to write a new case from scratch for a small change. It has a feature which automatically updates the existing cases if anything is changed- for e.g. a variable name is changed.
                                            I do scheduled performance test of my portals on different VMs located in different countries. The performance result sheet is very useful.
                                            I also run some functional scheduled tests everyday .
                                            I have integrated it with my Visual Studio Team System. It enables me to share my tests with developers.
                                            Another pros would be helpful support.Several times they did video conferencing with me to understand my problem and they resolved it by providing a patch or showing me the way to handle it.

                                            The cons of this product is it's extremely buggy , it's heavyweight and it's not free!
                                            Another cons would be the editor does not give you freedom to modify the code behind. They have provided lots of features to make sure that you dont need to write a single line code. Therefore sometimes I feel restricted. But in that case I can take my testcase to VSTS - where I can modify it according to my need.

                                            -Regards
                                            Erfana Sikder

                                            --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@...> wrote:
                                            >
                                            > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                            >  
                                            > Tanvir
                                            >
                                            > "Consistency is an essential thing in acquiring knowledge"
                                            >
                                            >
                                            > ________________________________
                                            >

                                          • Lisa Crispin
                                            Jim, we weren t at all worried about you, you are such a great resource on test automation in general! But we didn t want any members to think we were letting
                                            Message 21 of 28 , Aug 16, 2012
                                            View Source
                                            • 0 Attachment
                                              Jim, we weren't at all worried about you, you are such a great resource on test automation in general! But we didn't want any members to think we were letting people "sell" on agile-testing. You were sharing your experiences, and helping folks with questions, and that is all totally cool. Our note wasn't directed at you, but more to make sure people understand the purpose of the list.
                                              -- Lisa

                                              On Wed, Aug 15, 2012 at 10:02 AM, Jim Holmes <jim.holmes@...> wrote:
                                               

                                              Janet et al., 

                                              I am extraordinarily sensitive about not doing pitches, which is why I've previously only monitored this group without subscribing. The last thing I want to do is irritate folks by appearing to push my product.

                                              I'd be happy to engage Clifton and Tanvir off the list if you feel that's more appropriate.

                                              Jim Holmes
                                              Evangelist, Testing Tools
                                              Telerik
                                              Twitter: @aJimHolmes

                                              From: Janet Gregory <janet_gregory@...>
                                              Reply-To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                              Date: Wednesday, August 15, 2012 11:24 AM
                                              To: "agile-testing@yahoogroups.com" <agile-testing@yahoogroups.com>
                                              Subject: RE: [agile-testing] Re: Telerik vs Selenium

                                               

                                              Hi folks,

                                               

                                              A quick note from the moderators on this thread…. 

                                               

                                              We don't want this to be a sales forum, but we encourage everyone to share their actual experiences with various tools.  I know I get a lot of questions from people on tools that I can’t answer and we have such a breadth of experience here.

                                               

                                              Janet Gregory

                                               

                                              From: agile-testing@yahoogroups.com [mailto:agile-testing@yahoogroups.com] On Behalf Of tsagilecg1
                                              Sent: August-15-12 9:14 AM
                                              To: agile-testing@yahoogroups.com
                                              Subject: [agile-testing] Re: Telerik vs Selenium

                                               

                                               

                                              Jim,
                                              I'm Tanvir's Coworker. I've worked with the telerik framework for some time and also like it very much, I also have no doubt that selenium is as powerful simulating actions on the browser.

                                              I think the question we are struggling with is, what supporting tools can be used with selenium vs maybe the telerik frame work or watin(are there keyword driven framworks, or other approaches, are there reporting tools that can be used, and so on?). that will work best for us. Can you recommend any we can evaluate?

                                              I will agree with you, the how its implemented is going to be more important than the APIs used.

                                              Thanks
                                              Clifton

                                              --- In agile-testing@yahoogroups.com, "jimholmesoh" <jim.holmes@...> wrote:
                                              >
                                              > Tanvir,
                                              >
                                              > I'm the evangelist for Telerik's Test Studio (and the underlying Telerik Testing Framework, formerly known as WebAii), so I'm obviously a bit biased in my opinions.
                                              >
                                              > At the end of the day, you need to find a tool/framework/API combination that works for you and your team's environment and skill sets. I highly, HIGHLY recommend you read Elisabeth Hendrickson's awesome post on selecting a test automation toolset: http://testobsessed.com/2011/12/selecting-test-automation-tools/
                                              >
                                              > You need to set expectations with your team and leadership that you need time *on the schedule* for evaluating various tools and approaches. Put Telerik's Testing Framework up against WebDriver, WatiN, Robot Framework, or Jeff Morgan's pageobject gem in Ruby. See what works out best for your team.
                                              >
                                              > I've used WebDriver, Watir, Watin, and a number of other tools. I still do UI automation workshops and talks using WebDriver as an example for my demos. It's a great set of APIs.
                                              >
                                              > I favor Telerik's Testing Framework for great support of frames, ease of dealing with AJAX or other dynamic/asynch content, and cross-browser support. You absolutely can get all of those things done, and done handily, with WebDriver or other APIs/frameworks. (The Testing Framework also has a paid option for support when you need it.)
                                              >
                                              > At the end of the day, frankly, your team's success won't hinge on the API or toolset you've chosen. Your team's success will hinge more upon the communication and practices you build up around HOW you're doing your testing. You can do great things with WebDriver or other APIs, and you can fail with Telerik's Testing Framework. It's the human side of things that are more critical.
                                              >
                                              > Jim
                                              >
                                              > (Sorry for the preachy rant. Telerik isn't a silver bullet for the hard problem of automating your UI tests, nor are WebDriver or other APIs/tools!)
                                              >
                                              >
                                              > --- In agile-testing@yahoogroups.com, Tanvir-ul-Islam <tanvir6@> wrote:
                                              > >
                                              > > My company is recently moving towards agile and automation. We have a web-based application and we are currently debating whether to use Telerik or Selenium for automation purposes. Most of our site uses legacy asp code and recently we are shifting to .net. We are dealing with frames 100% of times, our html structures are pretty complex with lot of nested tables, frames, forms. Based on your experiences, if any of you have used both or one of the tools(or any other tools) if you can discuss some of the pros and cons of the tools that would be great.
                                              > >  
                                              > > Tanvir
                                              > >
                                              > > "Consistency is an essential thing in acquiring knowledge"
                                              > >
                                              > >
                                              > > ________________________________
                                              > >
                                              >




                                              --
                                              Lisa Crispin
                                              Co-author with Janet Gregory, _Agile Testing: A Practical Guide for Testers and Agile Teams_ (Addison-Wesley 2009)
                                              Contributor to _Beautiful Testing_ (O'Reilly 2009)
                                              http://lisacrispin.com
                                              @lisacrispin on Twitter
                                              http://entaggle.com/lisacrispin

                                            Your message has been successfully submitted and would be delivered to recipients shortly.