Loading ...
Sorry, an error occurred while loading the content.

Pool exhausted error

Expand Messages
  • debelyoo
    Hello, I m developing a statistics logger module with blueeyes. It basically receives statistics from numerous clients (POST requests) and stores them in a
    Message 1 of 4 , Jan 19, 2012
    • 0 Attachment
      Hello,

      I'm developing a statistics logger module with blueeyes. It basically receives statistics from numerous clients (POST requests) and stores them in a Cassandra DB. I'm using the Cascal library (https://github.com/shorrockin/cascal) to communicate with the Cassandra DB.

      When my module receives a lot of statistics at the same time (or almost) I get a "Pool exhausted" error (as shown below).

      Is there anything that I can tweak to avoid this error ?

      Thanks,
      Jean

      ERR [20120119-15:30:35.107] blueeyes: Error handling request
      ERR [20120119-15:30:35.107] blueeyes: java.util.NoSuchElementException: Pool exhausted
      ERR [20120119-15:30:35.107] blueeyes: at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1096)
      ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.checkout(SessionPool.scala:104)
      ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.borrow(SessionPool.scala:83)
      ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.CassandraHandler$.doBatchInsert(CassandraHandler.scala:27)
      ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.insertNode(NodeHandler.scala:23)
      ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.handleNodePost(NodeHandler.scala:17)
      ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:52)
      ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:50)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:132)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:129)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:63)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:59)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
      ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:45)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:357)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:354)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
      ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:46)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpServer$class.apply(HttpServer.scala:68)
      ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
      ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
      ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.engines.NettyRequestHandler.messageReceived(NettyRequestHandler.scala:43)
      ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
      ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
      ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
      ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
      ERR [20120119-15:30:35.107] blueeyes: (...more...)
    • John A. De Goes
      This error is occurring deep inside Cascal. Have you contacted the author to see if he has any suggestions? -- John A. De Goes CEO / CTO 303-921-5784 -
      Message 2 of 4 , Jan 19, 2012
      • 0 Attachment

        This error is occurring deep inside Cascal. Have you contacted the author to see if he has any suggestions?
        -- 
        John A. De Goes
        CEO / CTO
        303-921-5784 - @jdegoes

        ReportGrid, Inc - http://reportgrid.com






        On Jan 19, 2012, at 7:48 AM, debelyoo wrote:

         

        Hello,

        I'm developing a statistics logger module with blueeyes. It basically receives statistics from numerous clients (POST requests) and stores them in a Cassandra DB. I'm using the Cascal library (https://github.com/shorrockin/cascal) to communicate with the Cassandra DB.

        When my module receives a lot of statistics at the same time (or almost) I get a "Pool exhausted" error (as shown below).

        Is there anything that I can tweak to avoid this error ?

        Thanks,
        Jean

        ERR [20120119-15:30:35.107] blueeyes: Error handling request
        ERR [20120119-15:30:35.107] blueeyes: java.util.NoSuchElementException: Pool exhausted
        ERR [20120119-15:30:35.107] blueeyes: at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1096)
        ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.checkout(SessionPool.scala:104)
        ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.borrow(SessionPool.scala:83)
        ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.CassandraHandler$.doBatchInsert(CassandraHandler.scala:27)
        ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.insertNode(NodeHandler.scala:23)
        ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.handleNodePost(NodeHandler.scala:17)
        ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:52)
        ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:50)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:132)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:129)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:63)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:59)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
        ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:45)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:357)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:354)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
        ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:46)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpServer$class.apply(HttpServer.scala:68)
        ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
        ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
        ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.engines.NettyRequestHandler.messageReceived(NettyRequestHandler.scala:43)
        ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
        ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
        ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
        ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
        ERR [20120119-15:30:35.107] blueeyes: (...more...)


      • debelyoo
        No not yet. I write him an email now. Thanks for the hint. Jean
        Message 3 of 4 , Jan 19, 2012
        • 0 Attachment
          No not yet.

          I write him an email now. Thanks for the hint.

          Jean

          --- In blueeyes-web@yahoogroups.com, "John A. De Goes" <john@...> wrote:
          >
          >
          > This error is occurring deep inside Cascal. Have you contacted the author to see if he has any suggestions?
          > --
          > John A. De Goes
          > CEO / CTO
          > 303-921-5784 - @jdegoes
          >
          > ReportGrid, Inc - http://reportgrid.com
          >
          >
          >
          >
          >
          >
          > On Jan 19, 2012, at 7:48 AM, debelyoo wrote:
          >
          > > Hello,
          > >
          > > I'm developing a statistics logger module with blueeyes. It basically receives statistics from numerous clients (POST requests) and stores them in a Cassandra DB. I'm using the Cascal library (https://github.com/shorrockin/cascal) to communicate with the Cassandra DB.
          > >
          > > When my module receives a lot of statistics at the same time (or almost) I get a "Pool exhausted" error (as shown below).
          > >
          > > Is there anything that I can tweak to avoid this error ?
          > >
          > > Thanks,
          > > Jean
          > >
          > > ERR [20120119-15:30:35.107] blueeyes: Error handling request
          > > ERR [20120119-15:30:35.107] blueeyes: java.util.NoSuchElementException: Pool exhausted
          > > ERR [20120119-15:30:35.107] blueeyes: at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1096)
          > > ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.checkout(SessionPool.scala:104)
          > > ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.borrow(SessionPool.scala:83)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.CassandraHandler$.doBatchInsert(CassandraHandler.scala:27)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.insertNode(NodeHandler.scala:23)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.dbHandler.NodeHandler$.handleNodePost(NodeHandler.scala:17)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:52)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatisticService$$anonfun$1$$anonfun$apply$4$$anonfun$apply$9.apply(StatisticService.scala:50)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:132)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$6.apply(HttpRequestHandlerCombinators.scala:129)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:63)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$method$1$$anon$2.apply(HttpRequestHandlerCombinators.scala:59)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
          > > ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:45)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:357)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anon$11.apply(HttpRequestHandlerCombinators.scala:354)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:36)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpRequestHandlerCombinators$$anonfun$path$1$$anon$1.apply(HttpRequestHandlerCombinators.scala:28)
          > > ERR [20120119-15:30:35.107] blueeyes: at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:46)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.HttpServer$class.apply(HttpServer.scala:68)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
          > > ERR [20120119-15:30:35.107] blueeyes: at code.RESTHandler.StatsDbInterface$.apply(StatsDbInterface.scala:11)
          > > ERR [20120119-15:30:35.107] blueeyes: at blueeyes.core.service.engines.NettyRequestHandler.messageReceived(NettyRequestHandler.scala:43)
          > > ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)
          > > ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:545)
          > > ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:754)
          > > ERR [20120119-15:30:35.107] blueeyes: at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302)
          > > ERR [20120119-15:30:35.107] blueeyes: (...more...)
          > >
          > >
          >
        • Noel Welsh
          This suggests Cascal has some pool of sessions (maybe these are the same as DB connections?) and you re running out of them. Assuming this is correct you need
          Message 4 of 4 , Jan 19, 2012
          • 0 Attachment
            This suggests Cascal has some pool of sessions (maybe these are the same as DB connections?) and you're running out of them. Assuming this is correct you need to queue your events till a session is free, and/or batch them up to use fewer sessions.

            HTH,
            N.

            ERR [20120119-15:30:35.107] blueeyes: java.util.NoSuchElementException: Pool exhausted
            ERR [20120119-15:30:35.107] blueeyes: at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1096)
            ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.checkout(SessionPool.scala:104)
            ERR [20120119-15:30:35.107] blueeyes: at com.shorrockin.cascal.session.SessionPool.borrow(SessionPool.scala:83)


          Your message has been successfully submitted and would be delivered to recipients shortly.