Loading ...
Sorry, an error occurred while loading the content.

Re: Persistence

Expand Messages
  • Ralph Daugherty
    Hi Paul, I will start coding what I m describing and then open source it. Can t tell from comments if there is something in what I m describing that someone
    Message 1 of 32 , May 1 5:06 AM
    • 0 Attachment
      Hi Paul, I will start coding what I'm describing and then open source
      it. Can't tell from comments if there is something in what I'm
      describing that someone thinks can't or wouldn't work for some
      specific reason, but if I run across a major stumbling block I guess
      I'll find out. :)

      rd


      --- In Easy400Group@yahoogroups.com, "DECOURCY ELLIS Paul AUS GUEST"
      <paul.decourcyellis@...> wrote:
      >
      > Hi All,
      >
      > I have been following this thread and I will put my hand up to say I
      am a little confused. I also seem to be able to agree with both yours
      and Kevin's post
      > each time I read them. A sure sign I don't fully understand the
      underlying concept.
      >
      > Ralph, if you get around to writing a White paper on this approach
      could you please post to this forum. I am always looking for new ways
      to to add "stateful" logic to my systems.
      >
      >
      > Ciao
      >
      >
      > Paul deCourcy Ellis
      > Application Support
      > Ferrero Australia Pty Limited
      > * Locked Bag 902, North Sydney, NSW, 2060
      > * (02) 9409 8451
      > 6 (02) 9409 8861
      > Mob 0423196525
      > Before you print this message, be sure that it is necessary.
      The environment is in our hands!
      >
      > Notice: The information contained in this e-mail and/or documents
      which accompany it is confidential, may be privileged, proprietary and
      exempt from disclosure, and is intended for the exclusive use of the
      person or persons to whom it is addressed. If you are not the intended
      recipient or if you have received this message in error for any
      reason, please advise us immediately by reply e-mail and delete both
      the message and the copy thereof contained in reply. Thank you.
      >
    • Ralph Daugherty
      Anyway......perhaps we are not really signing off the same song sheet.....and I wonder if the person who started this thread is still reading it or has lost
      Message 32 of 32 , May 13 10:32 AM
      • 0 Attachment
        "Anyway......perhaps we are not really signing off the same song
        sheet.....and I wonder if the person who started this thread is still
        reading it or has lost the will to live J"

        lol I don't know, but he started a good discussion. I'm pretty sure
        we'll come up with at least one good alternative out of this.

        ok, I took a look at a description of user index again and I see
        the key values within the user index you're talking about. Thanks for
        the clarification.

        I agree, with minimal state values, just the key values associated
        with the session and some status codes and such as to what is going
        on, the user index code you have is probably the best combination of
        fast and flexible.

        Where I'm going is an alternative to preserving a few key values to
        use to rebuild state by using as keys to go refigure out whatever is
        needed, or to storing away all the globals as is done with save
        session type API in Java and PHP and the like.

        Instead, all the globals used in the various business logic
        procedures, and all we have to do is look at any current program and
        see how many standalone globals, data structures, and arrays there are
        to see what they would be, would reside in data structures (or arrays
        of data structures) instead of standalones and accessed by a session
        index.

        That session index would be the one value that I would need to
        set/get with user index API to associate with a session id (session id
        being given to me by interface with the environment I'm operating in,
        such as Rennaissance), then use the session index (1 to MAX_SESSIONS)
        as a data structure array index to access variables with normal
        programming which accesses global variables across UI calls, the only
        difference being that each variable would have (sess_idx) after it to
        index into a data structure array (as in last_item_key(sess_idx)
        versus last_item_key).

        I hope that no one finds this too redundant, but since Nathan
        wasn't sure what I was talking about (and maybe no one else either :)
        I hope it helps. Of course demo source will help further which I will
        make available.

        Certainly all the hard work has been done by the CGIDEV2 and
        Rennaisace guys, this is just a tweak for te business logic that we'll
        see if it's as screaming fast as I think it will be.

        rd


        --- In Easy400Group@yahoogroups.com, "Kevin Turner" <kevin.turner@...>
        wrote:
        >
        > Hi Ralph
        >
        >
        >
        > "You could do that, but with a 10 digit index the key is already the
        > session index, not each state variable of a session."
        > Not quite true, which may explain the misunderstanding. If I explain it
        > a little further: 10 users log on. Each gets a unique 10 character
        > session id allocated which is carried around on every request. For each
        > session id, a user index object is created (in a work library) of the
        > same name - so there would be 10 individual user index objects in the
        > work library. When a request arrives from a user with session id
        > ABCDE12345 and a program does a SES_Set('myvariable':myvalue), then a
        > record gets created in user index ABCDE12345 with a key of "myvariable"
        > and a value of myvalue. When another program does a
        > SES_Get("myvariable") then the service programs goes to user index
        > ABCDE12345 and "gets" the record keyed by "myvariable" and returns
        > myvalue. So the session id just tells you which object contains the
        > data, but the retrieval of the data is by variable name. There is no
        > parsing in such a scenario - myvalue might be just a Boolean value after
        > all. I guess you could compare this with the ability to chain a field in
        > a physical file rather than chaining a record (to get a buffer). Of
        > course, the programmer may choose to store a complete record retrieved
        > from a file (or an HTML page) in a single session variable rather than
        > store each field individually. In that case they would indeed have to
        > overlay the retrieved data onto a DS or parse it perhaps - but that is
        > down to the way an application programmer chooses to use the session
        > persistence facility.
        >
        >
        >
        > I might have mentioned it before, but if this user starts getting odd
        > behaviour that needs diagnosis, we can just do a DMPOBJ ABCDE12345
        > *USRIDX and see their session state information in spooled format (in
        > fact this happens automatically if a PSSR routine kicks in). This is
        > something that we would need to be able to replicate in an alternative
        > memory-only based solution. Not easy if the program that has the data in
        > memory is the one that has crashed.
        >
        >
        >
        > Anyway......perhaps we are not really signing off the same song
        > sheet.....and I wonder if the person who started this thread is still
        > reading it or has lost the will to live J
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        >
        > From: Easy400Group@yahoogroups.com [mailto:Easy400Group@yahoogroups.com]
        > On Behalf Of Ralph Daugherty
        > Sent: 13 May 2008 12:58
        > To: Easy400Group@yahoogroups.com
        > Subject: [Easy400Group] Re: Persistence
        >
        >
        >
        >
        > Hi Kevin, I call:
        >
        > "SES_Set(`myvariable':myvalue) where "myvariable" is absolutely any
        > alphanumeric value they choose up to 256 bytes in length, and myvalue
        > is any alphanumeric value up to 32768 bytes in length."
        >
        > parsing through a user index for data. :) The data is stored as
        > alpha, and it is parsed after being accessed by a user index to
        > extract the various elements from a buffer. Unless I misunderstood the
        > user index doc, the index is to a buffer, of which you determine
        > makeup and store away and parse to extract ( or move to a DS, etc.).
        >
        > "You access each data element directly using a key, in the same way
        > that you access data in a table (or logical view)."
        >
        > You could do that, but with a 10 digit index the key is already the
        > session index, not each state variable of a session.
        >
        > Oh, I agree wholeheartedly with the rest of your post. %scan is
        > parsing. The difference in what I'm talking about is your get/set one
        > state variable at a time versus my get/set the entire set of state
        > variables for a task with a data structure. The data in the server is
        > stored as type (packed, integer, text, varying, etc.) and filled into
        > data structure fields with an = per field as the overhead.
        >
        > Any access to text potentially larger than 256 should be a separate
        > get/set call. You wouldn't mix unstructured text buffers in with the
        > fixed type fields.
        >
        > I've done all this for the Jobs/400 web site for News/400 back in
        > 2000, which wasn't used much because there weren't any jobs for /400
        > people. :( The functionality contained the best features of every
        > existing job site, so it was quite complex. However I wrote that as
        > traditional stateless web serving logic, and I am proposing the
        > arrayed data structures to change all that.
        >
        > In the end, either setting/getting as we discussed last or fully
        > processing in the dataq servers against the arrayed data structures,
        > the UI independent data is exchanged in dataq's with different DS
        > layouts based on an opcode, or task code, of what the dataq buffer
        > contains. This I do understand is quite different in concept from say
        > a buffer of HTML that may be output.
        >
        > I appreciate the insights, Kevin.
        >
        > rd
        >
        > --- In Easy400Group@yahoogroups.com
        > <mailto:Easy400Group%40yahoogroups.com> , "Kevin Turner"
        > <kevin.turner@>
        > wrote:
        > >
        > > Ralph
        > >
        > >
        > >
        > > If I may be so bold, before getting carried away with the premise that
        > > the data structure array "is probably a lot faster than parsing
        > through
        > > a users index for data" we need to back that up with some evidence.
        > For
        > > starters, you don't parse a user index to retrieve data. You access
        > each
        > > data element directly using a key, in the same way that you access
        > data
        > > in a table (or logical view). The difference is that instead of having
        > > convenient RPG operation codes (like CHAIN) you have to use the APIs
        > > provided - which basically means you create your own friendly wrappers
        > > around the APIs to give something more friendly to use. It is
        > extremely
        > > fast. I am toying with experimenting with a large DS array to try the
        > > same sort of thing, but the more I get into it the more I am realising
        > > that there are some flaws in it that could probably make it slower for
        > > the way we implement session persistence. In Renaissance, a CGI
        > > programmer can do this:
        > >
        > > SES_Set('myvariable':myvalue) where "myvariable" is absolutely any
        > > alphanumeric value they choose up to 256 bytes in length, and myvalue
        > is
        > > any alphanumeric value up to 32768 bytes in length. This stores (in a
        > > user index named after the session id) a record keyed by 'myvariable'.
        > > At any time in the session the programmer can retrieve the value using
        > > myvalue=SES_Get('myvariable'); There is no limit on the number of
        > > session variables one can set or what you use them for, so we use them
        > > for all sorts of things - including replicating a viewstate mechanism
        > > you see in .net applications.
        > >
        > >
        > >
        > > If I try to do this sort of thing is a DS array, with each dim of the
        > DS
        > > representing a session, I quickly come unstuck. I can do something
        > that
        > > makes it easy for me to retrieve the DS array element representing the
        > > session quickly, but I cannot think of a good way of storing the
        > session
        > > data in the DS itself in a way that would make it fast to retrieve.
        > The
        > > DS would consist of nothing but a long string of name, value pairs, so
        > > when I do a SES_Get('myvariable') I am going to have to %scan the
        > string
        > > to find 'myvariable' in order to get myvalue. This all ignores the
        > fact
        > > that I am also limited in terms of size using a datastructure, but
        > with
        > > a user index I am not (there is a limit with the record length, but
        > > variables can be split into several records if required).
        > >
        > >
        > >
        > > I think I would need to see a specific example of the DS technique to
        > be
        > > able to replicate the functionality we need from session persistence.
        > > Then I can set about doing a good comparison test. It could be that
        > > what you are describing/proposing works for the way you intend to
        > > implement session persistence, but doesn't work for the way we have
        > > implemented it in RNS. Let me know when you have some code to play
        > > with J
        > >
        > >
        > >
        > > Cheers
        > >
        > > Kevin
        > >
        > >
        > >
        > > From: Easy400Group@yahoogroups.com
        > <mailto:Easy400Group%40yahoogroups.com>
        > [mailto:Easy400Group@yahoogroups.com
        > <mailto:Easy400Group%40yahoogroups.com> ]
        > > On Behalf Of Ralph Daugherty
        > > Sent: 13 May 2008 01:19
        > > To: Easy400Group@yahoogroups.com
        > <mailto:Easy400Group%40yahoogroups.com>
        > > Subject: [Easy400Group] Re: Persistence
        > >
        > >
        > >
        > >
        > > shoot, I had a response typed up, hit one wrong key or something
        > > and the whole thing disappeared and dropped me back out of editing. Oh
        > > how I love web pages. :D
        > >
        > > I was not suggesting this as a faster way to store state data, but
        > > you know what, it probably is a lot faster than parsing through user
        > > index data for significant amounts of data. Also much more powerful in
        > > returning only the appropriate state data for a given request using
        > > task opcodes and returning in a different data structure per opcode.
        > >
        > > I would say next best thing to my suggestion. the only thing
        > > missing is merging the business logic in there with the data. :)
        > >
        > > To picture what I'm talking about, just take any 5250 interactive
        > > program, put a DS around the standalones, and dimension to whatever
        > > sessions your architecture is to handle. This would be equivalent to
        > > how many interactive sessions you expected your box to handle,
        > > although you normally wouldn't get close to having all those sessions
        > > going but if you had a lot of remote access you could handle it if
        > > necessary, for example.
        > >
        > > The code here is different than interactive in two respects. It's
        > > UI independent, writing to a data queue as data fields rather than
        > > doing an EXFMT or outputting a buffer of HTML or XML. CGI, 5250, or
        > > client would handle UI.
        > >
        > > The other difference is that logic is broken up into sections
        > > starting with an input dataq and ending with writing a data. It would
        > > be the code between two EXFMT's at a minimum to roughly envision it.
        > > It's the code that handles input, or Function keys, and every other
        > > task business logic is provided for.
        > >
        > > Obviously, all logic that can be isolated to subprocedures in
        > > service programs would be done, and business logic calls for SOA and
        > > batch would also be used by the UI oriented calls.
        > >
        > > When a page goes out, the data in every variable is still there as
        > > in interactive. When a page comes back in, any access to variables is
        > > available as we do in 5250 programming. It's normal state programming
        > > versus stateless programming and the need to go retrieve that data
        > > having been written away last time that data was used for this
        > > session, and even more importantly than the overhead of doing that, is
        > > the special stateless programming that must be done versus normal
        > > programming.
        > >
        > > The other difference is that the standalones would have a DS
        > > statement right above them (and S removed from all of them), and the
        > > data structure DIMensioned to MAX_SESSIONS, a constant used throughout
        > > all the logic servers for that app system. Let's say 10,000 to show
        > > what kind of scaling capability we have here.
        > >
        > > Or should I say it's already scaled, just a matter of scaling the
        > > CGI jobs as is already done. The single instance server will blast
        > > back anything you want to throw at it with state already in place.
        > >
        > > rd
        > >
        > > --- In Easy400Group@yahoogroups.com
        > <mailto:Easy400Group%40yahoogroups.com>
        > > <mailto:Easy400Group%40yahoogroups.com> , "nandelin" <nandelin@>
        > > wrote:
        > > >
        > > > Kevin,
        > > >
        > > > I'm not sure if I really understood Ralph's idea either, but it's
        > > > clear that you understand my interpretation of it. A separate "state
        > > > server". Keep session data in memory. Getters and setters performed
        > > > over a data queue.
        > > >
        > > > My gut feel is that there wouldn't be much performance difference
        > > > compared to user indexes, but I wouldn't be concerned about the
        > server
        > > > going down. The design is so simple, that it would likely be more
        > > > reliable than an HTTP server, which is reliable enough.
        > > >
        > > > Talking about reliability, your design of pairing CGI jobs with
        > > > business logic servers not only addresses the scalability issue, it
        > > > can also be more reliable. The CGI jobs do little more than screen
        > > > I/O. The complex logic is in the business logic servers. There have
        > > > been a number of reports where failing CGI jobs cause the entire
        > HTTP
        > > > server to lock up. So it helps to keep the CGI as simple as
        > possible.
        > > >
        > > > --
        > > >
        > > > Nathan M. Andelin
        > > >
        > >
        > >
        > >
        > >
        > > NOTICE: The information in this electronic mail transmission is
        > intended by CoralTree Systems Ltd for the use of the named individuals
        > or entity to which it is directed and may contain information that is
        > privileged or otherwise confidential. If you have received this
        > electronic mail transmission in error, please delete it from your
        > system without copying or forwarding it, and notify the sender of the
        > error by reply email or by telephone, so that the sender's address
        > records can be corrected.
        > >
        > > CoralTree Systems Limited
        > > Company Registration Number 5021022.
        > > Registered Office:
        > > 12-14 Carlton Place
        > > Southampton
        > > Hampshire
        > > SO15 2EA
        > > UK
        > > VAT Registration Number 834 1020 74.
        > >
        >
      Your message has been successfully submitted and would be delivered to recipients shortly.