Loading ...
Sorry, an error occurred while loading the content.

20827Re: [fitnesse] FitNesse parsing limit?

Expand Messages
  • Gregor Gramlich
    Jul 16, 2014
      Hi all,

      as far as I have seen, nobody has mentioned it yet, but this exactly matches your error message:
      Error while executing SLIM instructions: protocol error: expected colon after message length field: n

      The Slim protocol only allows messages with approx 10^6 bytes. Usually one message is used for one table.

      Every other message that slim sends will be prefixed by a <length> in bytes, followed by a colon. NOTE: Every other length in this document is in UTF-8 characters. This one length is in bytes.

      See also

      Gregor



      2014-07-16 8:59 GMT+02:00 Barre Dijkstra barre.dijkstra@... [fitnesse] <fitnesse@yahoogroups.com>:
       

      Hi,

      I just went through the relevant slim code and I double checked if there were no artificial limits in place.
      The error message is that the slim protocol (http://fitnesse.org/FitNesse.FullReferenceGuide.UserGuide.WritingAcceptanceTests.SliM.SlimProtocol) is expecting either a tag or a value of a certain length and that the length is not as expected or resulted in an invalid command.

      Disclaimer: I scanned through the relevant code as far as I could see or where I knew where parsing took place, but I haven't worked on slim for at least the last half year. It might very well be that I missed some parts and I did by no means do an extensive analysis nor did I ran any tests to reproduce the problem.

      The only hard restriction I found in the serialization/deserialization and execution in slim and its protocol is the length of a set of instructions is limited by the length of char array in Java  (2^31-1 characters) since a StringBuffer is used for creating the message being sent to the slim server for execution.
      My guess is that that limit is hit and the rest is truncated.

      If that is the case, 3 options going forward are;
      1. Revisit your test setup (do you need all the includes, etc.?). This might also have a positive on maintainability/clarity of the test set as side-effect.
      2. Implementing a streaming transfer from the slim client and the slim server. This will be tricky, especially in terms of either backwards compatibility and compatibility with various implementations in different programming languages.
      3. Implementing a chunk based transfer from the slim client and the slim server. This will require some looking into in terms of result correlation due to the decoupled/multi-threaded execution of chunks.

      I have no direct solution, but I hope I at least shed some light on the issue from the implementation perspective.

      Regards,

      Barre




      On Sun, Jul 13, 2014 at 9:12 PM, roboaks@... [fitnesse] <fitnesse@yahoogroups.com> wrote:
       

       For our automated testing consulting practice we have created a generalized AFT platform that is based on the FitNesse/Xebium/Selenium stack. This platform is embodied in a 3-tier hierarchy of FitNesse scenario tables (along with a set of primitives that abstract away many Selenium nuances). This platform has been quite successful and has greatly increased the efficiency with which we can create and maintain our clients’ testing suites.

       

      Unfortunately, it appears that we may have bumped up against a serious FitNesse parsing/processing limitation in certain circumstances. For example, we have a simple test that is actually designed to exercise a complex bottom tier element (something we call a “mechanic”). The test looks something like this:

       

      '''Include common and SUT primitives'''

      !include -c . .CommonRoot.ScenarioInclude

      !include -c . .SutRoot.ScenarioInclude

       

      !include -c .ClientMechanics.QuotePolicyMechanics.PolicyMechanics.HoPolicyMechanic

       

      !| script |

      | HoPolicyMechanic.createSetupRunKeep |


      Now, the included page, HoPolicyMechanic, in addition to its own scenarios, has this include:


      !include -c .ClientMechanics.QuotePolicyMechanics.ApplicationMechanics.HoApplicationMechanic

      And HoApplicationMechanic, in turn, has:

      !include -c .ClientMechanics.QuotePolicyMechanics.QuoteMechanics.HoQuoteMechanic

      Finally, HoQuoteMechanic has:


      !include -c .ClientMechanics.QuotePolicyMechanics.CustomerMechanic


      While the overall page hierarchy/structure is certainly on the complex side, my guess is that the test requires FitNesse to parse less than 5,000 table rows in total. Tthe vast majority of those rows are in scenario tables. A total of perhaps 10 includes are used. We use symbols and variables extensively; for this test perhaps 200 variables and 150 symbols come into play.


      First, here’s the ErrorLog page:

       


      Error while executing SLIM instructions: protocol error: expected colon after message length field: 2


      fitnesse.slim.SlimError: protocol error: expected colon after message length field: 2


                      at fitnesse.slim.SlimServer.getInstructionsFromClient(SlimServer.java:84)


                      at fitnesse.slim.SlimServer.processOneSetOfInstructions(SlimServer.java:66)


                      at fitnesse.slim.SlimServer.tryProcessInstructions(SlimServer.java:54)


                      at fitnesse.slim.SlimServer.serve(SlimServer.java:40)


                      at fitnesse.slim.SlimService.accept(SlimService.java:112)


                      at fitnesse.slim.SlimService.startWithFactory(SlimService.java:37)


                      at fitnesse.slim.SlimService.main(SlimService.java:24)


       

      While I have no idea what this error signifies, here’s why I’m convinced it’s a FitNesse parsing/processing limitation.


      Once I reduce the cumulative number of rows (across scenario tables, script tables, etc.) below a certain value, the test starts working again. If I then add a single row back to any table anywhere in the hierarchy, the test fails with the error above.


      The only table row that I can add without causing the error is a comment row (using # or note). Even adding an innocuous row such as | show | echo | Hello World | causes the error.


      Note that there is one part of the error that seems to vary a bit depending on which row is added. Sometime I see:


      Error while executing SLIM instructions: protocol error: expected colon after message length field: 2


      And sometime I see


      Error while executing SLIM instructions: protocol error: expected colon after message length field: 7


      I have seen numbers besides 2 and 7 as well.


      Additional information that may be relevant:


      • The problem originally manifested in FitNesse v20140416, which we have been using from the start. Today, I updated to the latest version, v20140630, but unfortunately the problem remains.

      • I am running under Windows 7 and using Firefox 30.0.

      • We use no fixtures other than echo and Xebium fixture

      • Thinking that perhaps it was a heap or stack size issue (though there was no such error or other indication), I added the following parameters: -Xms256m -Xmx1024m -Xss8000k. I added these parameters to both the FitNesse startup command line:

        java  -Xms256m -Xmx1024m -Xss8000k -jar %FitNesseJar% -p %FitNessePort% -d %ProjectRootFolder% -r %ProjectRoot% -e 0 –v

        and to the root page:

        !define COMMAND_PATTERN {java -Xms256m -Xmx1024m -Xss8000k -cp %p %m}

        I assume COMMAND_PATTERN is only relevant to fixtures but I confess my ignorance in this area and, in any event, it didn’t seem to make a difference.

      This is a very serious situation for us. Our entire strategy assumes that FitNesse is sufficiently mature to support the deeply architectural approach that we’ve employed. Indeed, until now, I’ve encountered no significant limitations that would lead me to conclude otherwise. It is important enough that, if required, we would be willing to establish a paid engagement for the purpose of solving this issue.


       


      Thanks so much for you help.





    • Show all 6 messages in this topic