Loading ...
Sorry, an error occurred while loading the content.

Data layer for speed in fat-ajax applications

Expand Messages
  • Erol KABADAYI
    Hi There, Nowadays, we are in trouble and hacking about a performance issue. We have a calendar application which is storing about 500.000 records. We are
    Message 1 of 8 , Jul 12, 2008
    View Source
    • 0 Attachment
      Hi There,

      Nowadays, we are in trouble and hacking about a performance issue.

      We have a calendar application which is storing about 500.000 records.
      We are getting the records which we need by yui-connection-manager and
      showing/creating the items in for between certain dates for a certain
      user in the screen by using DOM methods. Apart from that we are
      applying some kinds of pagination. (Datas coming from server as JSON
      string.)

      Assume that we will have too much items to show them in the screen for
      a lot of people. ( I mean, we got nearly 1.200 items as a response
      from server. )

      Currently, we are getting all the 1200 items and creating all of them
      as an appointment item by DOM commands. Then we are hiding (with css
      display:none) the items except in the first page. If user click next
      page, we show the items in next page and hide previous page by use
      display property.

      But we know that this way is not best one. It causes momory and
      performance problems. So, we decided to change infrastructure of the
      applications after customer's negative feedback about speed.

      And now, we want create a new client-side data layer and place it
      between live-server and client-side. This layer will behave as a
      recordset/dataset/cache and we will easily the operations add,
      delete,update, search, navigate on records.

      This layer will have 2 scheduled ajax connections. One of them for
      fetching new/changes records into data-layer from server and the other
      one for pushing new records on data-layer to the server.

      If we create a new appointment, datas will insert firstly into this
      layer. After a while, changes will be send from data-layer to the server.

      If there is a new records or changes on server,in same way, they will
      be fetched from server to the data layer.

      When we navigate between pages we will look at data-layer first. We
      will just connect to server to get record in case we couldn't find the
      record in data-layer.

      We suppose we can improve the overall performance thanks to this way.

      Shortly, is there anything in Yui like we need?

      Thanks in advance,
      Best regards,

      Erol KabadayĆ½
    • Peter Michaux
      ... You definitely can. I ve done roughly what you have described above in one situation for a data table. I found building the table rows as HTML text and
      Message 2 of 8 , Jul 26, 2008
      View Source
      • 0 Attachment
        On Sat, Jul 12, 2008 at 1:41 AM, Erol KABADAYI <erolkabadayi@...> wrote:
        > Hi There,
        >
        > Nowadays, we are in trouble and hacking about a performance issue.
        >
        > We have a calendar application which is storing about 500.000 records.
        > We are getting the records which we need by yui-connection-manager and
        > showing/creating the items in for between certain dates for a certain
        > user in the screen by using DOM methods. Apart from that we are
        > applying some kinds of pagination. (Datas coming from server as JSON
        > string.)
        >
        > Assume that we will have too much items to show them in the screen for
        > a lot of people. ( I mean, we got nearly 1.200 items as a response
        > from server. )
        >
        > Currently, we are getting all the 1200 items and creating all of them
        > as an appointment item by DOM commands. Then we are hiding (with css
        > display:none) the items except in the first page. If user click next
        > page, we show the items in next page and hide previous page by use
        > display property.
        >
        > But we know that this way is not best one. It causes momory and
        > performance problems. So, we decided to change infrastructure of the
        > applications after customer's negative feedback about speed.
        >
        > And now, we want create a new client-side data layer and place it
        > between live-server and client-side. This layer will behave as a
        > recordset/dataset/cache and we will easily the operations add,
        > delete,update, search, navigate on records.
        >
        > This layer will have 2 scheduled ajax connections. One of them for
        > fetching new/changes records into data-layer from server and the other
        > one for pushing new records on data-layer to the server.
        >
        > If we create a new appointment, datas will insert firstly into this
        > layer. After a while, changes will be send from data-layer to the server.
        >
        > If there is a new records or changes on server,in same way, they will
        > be fetched from server to the data layer.
        >
        > When we navigate between pages we will look at data-layer first. We
        > will just connect to server to get record in case we couldn't find the
        > record in data-layer.
        >
        > We suppose we can improve the overall performance thanks to this way.

        You definitely can. I've done roughly what you have described above in
        one situation for a data table. I found building the table rows as
        HTML text and then inserting with innerHTML was the fastest response
        time. I also found that using event delegates for any listeners on the
        rows was handy because as the user cycled through the pagination, I
        didn't need to attach any new listeners. I only had to regenerate the
        HTML for the rows.

        > Shortly, is there anything in Yui like we need?

        Your application is very specific. I'm doubtful there is anything
        prefabricated and ready to go anywhere on the Internet that you could
        use.There are building blocks in YUI which you can use, of course.

        Peter
      • y_lsmith
        ... server. ... DataTable also does a similar thing. Rather than listening for events on the rows, however, a single event listener is attached to the table
        Message 3 of 8 , Jul 27, 2008
        View Source
        • 0 Attachment
          --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
          <petermichaux@...> wrote:
          >
          > On Sat, Jul 12, 2008 at 1:41 AM, Erol KABADAYI <erolkabadayi@...> wrote:
          > > Hi There,
          > >
          > > Nowadays, we are in trouble and hacking about a performance issue.
          > >
          > > We have a calendar application which is storing about 500.000 records.
          > > We are getting the records which we need by yui-connection-manager and
          > > showing/creating the items in for between certain dates for a certain
          > > user in the screen by using DOM methods. Apart from that we are
          > > applying some kinds of pagination. (Datas coming from server as JSON
          > > string.)
          > >
          > > Assume that we will have too much items to show them in the screen for
          > > a lot of people. ( I mean, we got nearly 1.200 items as a response
          > > from server. )
          > >
          > > Currently, we are getting all the 1200 items and creating all of them
          > > as an appointment item by DOM commands. Then we are hiding (with css
          > > display:none) the items except in the first page. If user click next
          > > page, we show the items in next page and hide previous page by use
          > > display property.
          > >
          > > But we know that this way is not best one. It causes momory and
          > > performance problems. So, we decided to change infrastructure of the
          > > applications after customer's negative feedback about speed.
          > >
          > > And now, we want create a new client-side data layer and place it
          > > between live-server and client-side. This layer will behave as a
          > > recordset/dataset/cache and we will easily the operations add,
          > > delete,update, search, navigate on records.
          > >
          > > This layer will have 2 scheduled ajax connections. One of them for
          > > fetching new/changes records into data-layer from server and the other
          > > one for pushing new records on data-layer to the server.
          > >
          > > If we create a new appointment, datas will insert firstly into this
          > > layer. After a while, changes will be send from data-layer to the
          server.
          > >
          > > If there is a new records or changes on server,in same way, they will
          > > be fetched from server to the data layer.
          > >
          > > When we navigate between pages we will look at data-layer first. We
          > > will just connect to server to get record in case we couldn't find the
          > > record in data-layer.
          > >
          > > We suppose we can improve the overall performance thanks to this way.
          >
          > You definitely can. I've done roughly what you have described above in
          > one situation for a data table. I found building the table rows as
          > HTML text and then inserting with innerHTML was the fastest response
          > time. I also found that using event delegates for any listeners on the
          > rows was handy because as the user cycled through the pagination, I
          > didn't need to attach any new listeners. I only had to regenerate the
          > HTML for the rows.

          DataTable also does a similar thing. Rather than listening for events
          on the rows, however, a single event listener is attached to the table
          element and an internal event bubbling mechanism exposes the custom
          events such as 'rowClickEvent'. However, innerHTML is read-only in IE
          for nearly all table elements so I'm curious how you got that working.
          Personally I've found using cloneNode and DocumentFragment are
          optimal for tables given so many of the constituent elements are at
          the same level in the DOM tree.

          >
          > > Shortly, is there anything in Yui like we need?
          >
          > Your application is very specific. I'm doubtful there is anything
          > prefabricated and ready to go anywhere on the Internet that you could
          > use.There are building blocks in YUI which you can use, of course.
          >
          > Peter
          >

          Agreed. You can look at DataTable and its supporting classes
          RecordSet, Record, Paginator, as well as how it relates (and delegates
          caching) to the DataSource. But in the end, you'll need to piece
          together your application. As your development progresses, we'd love
          to see updates, too! Always nice to showcase interesting applications.

          Luke
        • Peter Michaux
          ... It is a roundabout way to use innerHTML. Suppose a table in the actual pages needs to be updated. If a variable rows holds the html to be used as the new
          Message 4 of 8 , Jul 30, 2008
          View Source
          • 0 Attachment
            On Sun, Jul 27, 2008 at 12:57 PM, y_lsmith <lsmith@...> wrote:
            > --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
            > <petermichaux@...> wrote:

            >> You definitely can. I've done roughly what you have described above in
            >> one situation for a data table. I found building the table rows as
            >> HTML text and then inserting with innerHTML was the fastest response
            >> time. I also found that using event delegates for any listeners on the
            >> rows was handy because as the user cycled through the pagination, I
            >> didn't need to attach any new listeners. I only had to regenerate the
            >> HTML for the rows.
            >
            > DataTable also does a similar thing. Rather than listening for events
            > on the rows, however, a single event listener is attached to the table
            > element and an internal event bubbling mechanism exposes the custom
            > events such as 'rowClickEvent'. However, innerHTML is read-only in IE
            > for nearly all table elements so I'm curious how you got that working.
            > Personally I've found using cloneNode and DocumentFragment are
            > optimal for tables given so many of the constituent elements are at
            > the same level in the DOM tree.

            It is a roundabout way to use innerHTML.

            Suppose a table in the actual pages needs to be updated. If a variable
            "rows" holds the html to be used as the new rows of the table then do
            this

            var d = document.createElement('div')
            d.innerHTML = '<table><tbody>' + rows + '</tbody></table>';
            var tbody = d.childNodes[0].childNodes[0]

            This tbody element can replace the current tbody element actually in the page.

            This procedure using innerHTML is much faster than using createElement
            and appendChild to build up the cells and rows of a table.

            ----

            I've wrapped this type of functionality up in a library:

            http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83
            http://forkjavascript.org/mutate/docs

            Peter
          • y_lsmith
            ... [snip] ... the page. ... http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83 ... This method is a good
            Message 5 of 8 , Jul 31, 2008
            View Source
            • 0 Attachment
              --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
              [snip]
              >
              > It is a roundabout way to use innerHTML.
              >
              > Suppose a table in the actual pages needs to be updated. If a variable
              > "rows" holds the html to be used as the new rows of the table then do
              > this
              >
              > var d = document.createElement('div')
              > d.innerHTML = '<table><tbody>' + rows + '</tbody></table>';
              > var tbody = d.childNodes[0].childNodes[0]
              >
              > This tbody element can replace the current tbody element actually in
              the page.
              >
              > This procedure using innerHTML is much faster than using createElement
              > and appendChild to build up the cells and rows of a table.
              >
              > ----
              >
              > I've wrapped this type of functionality up in a library:
              >
              >
              http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83
              > http://forkjavascript.org/mutate/docs
              >
              > Peter
              >

              This method is a good solution for moderately sized tables containing
              content free of event handlers. In DataTable's case, we didn't have
              that option because the tables can get quite large, and cell contents
              can be rendered using implementer specific formatters, which are free
              to add event handlers.

              Luke
            • Peter Michaux
              ... Why do you write moderately sized ? I specifically used this for a very large table. The larger the table the bigger the benefit. Using event delegation
              Message 6 of 8 , Jul 31, 2008
              View Source
              • 0 Attachment
                On Thu, Jul 31, 2008 at 9:21 AM, y_lsmith <lsmith@...> wrote:
                > --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
                > [snip]
                >>
                >> It is a roundabout way to use innerHTML.
                >>
                >> Suppose a table in the actual pages needs to be updated. If a variable
                >> "rows" holds the html to be used as the new rows of the table then do
                >> this
                >>
                >> var d = document.createElement('div')
                >> d.innerHTML = '<table><tbody>' + rows + '</tbody></table>';
                >> var tbody = d.childNodes[0].childNodes[0]
                >>
                >> This tbody element can replace the current tbody element actually in
                > the page.
                >>
                >> This procedure using innerHTML is much faster than using createElement
                >> and appendChild to build up the cells and rows of a table.
                >>
                >> ----
                >>
                >> I've wrapped this type of functionality up in a library:
                >>
                >>
                > http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83
                >> http://forkjavascript.org/mutate/docs
                >>
                >> Peter
                >>
                >
                > This method is a good solution for moderately sized tables containing
                > content free of event handlers.

                Why do you write "moderately sized"? I specifically used this for a
                very large table. The larger the table the bigger the benefit.

                Using event delegation is a good solution for this problem; however,
                It is possible to use event handlers with this system. For example,
                particular elements can be found after the innerHTML operation and
                handlers attached. Another method to use event handlers is to have
                inline event handler attributes in the HTML and use the flyweight
                pattern where the cell identifies itself in a call to some other
                functions.

                Peter
              • y_lsmith
                ... variable ... createElement ... http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83 ... If you have a large
                Message 7 of 8 , Jul 31, 2008
                View Source
                • 0 Attachment
                  --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
                  <petermichaux@...> wrote:
                  >
                  > On Thu, Jul 31, 2008 at 9:21 AM, y_lsmith <lsmith@...> wrote:
                  > > --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
                  > > [snip]
                  > >>
                  > >> It is a roundabout way to use innerHTML.
                  > >>
                  > >> Suppose a table in the actual pages needs to be updated. If a
                  variable
                  > >> "rows" holds the html to be used as the new rows of the table then do
                  > >> this
                  > >>
                  > >> var d = document.createElement('div')
                  > >> d.innerHTML = '<table><tbody>' + rows + '</tbody></table>';
                  > >> var tbody = d.childNodes[0].childNodes[0]
                  > >>
                  > >> This tbody element can replace the current tbody element actually in
                  > > the page.
                  > >>
                  > >> This procedure using innerHTML is much faster than using
                  createElement
                  > >> and appendChild to build up the cells and rows of a table.
                  > >>
                  > >> ----
                  > >>
                  > >> I've wrapped this type of functionality up in a library:
                  > >>
                  > >>
                  > >
                  http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83
                  > >> http://forkjavascript.org/mutate/docs
                  > >>
                  > >> Peter
                  > >>
                  > >
                  > > This method is a good solution for moderately sized tables containing
                  > > content free of event handlers.
                  >
                  > Why do you write "moderately sized"? I specifically used this for a
                  > very large table. The larger the table the bigger the benefit.

                  If you have a large table, I would think that appending a docfrag to
                  the existing tbody would render faster because the entire DOM
                  structure in the tbody isn't being recreated, only reflowed. Less
                  memory consumption by avoiding the lengthy string concat as well. If
                  event handlers in the content weren't a concern, I might see if I
                  could use that method to create the additional rows to populate the
                  docfrag, though. That said, I've not done benchmarks because the
                  event handler issue makes the point moot for now, imo. It would
                  certainly be interesting to see some numbers. I might put something
                  together if I ever get a spare moment :)


                  >
                  > Using event delegation is a good solution for this problem; however,
                  > It is possible to use event handlers with this system. For example,
                  > particular elements can be found after the innerHTML operation and
                  > handlers attached. Another method to use event handlers is to have
                  > inline event handler attributes in the HTML and use the flyweight
                  > pattern where the cell identifies itself in a call to some other
                  > functions.

                  Event delegation is generally a good idea and we strongly encourage
                  using it. Declaring inline handlers is something we discourage,
                  though. There are cases when they may make sense pragmatically, but
                  generally speaking we recommend against their use. As to
                  (re)attaching event handlers after writing innerHTML, when nesting
                  widgets that maintain their own state, UI, and event infrastructure,
                  it's non-trivial (if possible) to rewire them after clobbering a
                  parent container's innerHTML, and certainly nothing we'd expect
                  implementers to have to deal with. We maintain the existing DOM as
                  much as possible to avoid conflicts with implementer code/style.

                  Luke
                • Peter Michaux
                  ... My situation is a complete replace of the tbody where all rows are changing. This is used in pagination or a very big dataset. Peter
                  Message 8 of 8 , Jul 31, 2008
                  View Source
                  • 0 Attachment
                    On Thu, Jul 31, 2008 at 11:38 AM, y_lsmith <lsmith@...> wrote:
                    > --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
                    > <petermichaux@...> wrote:
                    >>
                    >> On Thu, Jul 31, 2008 at 9:21 AM, y_lsmith <lsmith@...> wrote:
                    >> > --- In ydn-javascript@yahoogroups.com, "Peter Michaux"
                    >> > [snip]
                    >> >>
                    >> >> It is a roundabout way to use innerHTML.
                    >> >>
                    >> >> Suppose a table in the actual pages needs to be updated. If a
                    > variable
                    >> >> "rows" holds the html to be used as the new rows of the table then do
                    >> >> this
                    >> >>
                    >> >> var d = document.createElement('div')
                    >> >> d.innerHTML = '<table><tbody>' + rows + '</tbody></table>';
                    >> >> var tbody = d.childNodes[0].childNodes[0]
                    >> >>
                    >> >> This tbody element can replace the current tbody element actually in
                    >> > the page.
                    >> >>
                    >> >> This procedure using innerHTML is much faster than using
                    > createElement
                    >> >> and appendChild to build up the cells and rows of a table.
                    >> >>
                    >> >> ----
                    >> >>
                    >> >> I've wrapped this type of functionality up in a library:
                    >> >>
                    >> >>
                    >> >
                    > http://dev.forkjavascript.org/trac/browser/branches/RB-0.1/public/javascripts/fork/mutate.js#L83
                    >> >> http://forkjavascript.org/mutate/docs
                    >> >>
                    >> >> Peter
                    >> >>
                    >> >
                    >> > This method is a good solution for moderately sized tables containing
                    >> > content free of event handlers.
                    >>
                    >> Why do you write "moderately sized"? I specifically used this for a
                    >> very large table. The larger the table the bigger the benefit.
                    >
                    > If you have a large table, I would think that appending a docfrag to
                    > the existing tbody would render faster because the entire DOM
                    > structure in the tbody isn't being recreated, only reflowed.

                    My situation is a complete replace of the tbody where all rows are
                    changing. This is used in pagination or a very big dataset.

                    Peter
                  Your message has been successfully submitted and would be delivered to recipients shortly.