Loading ...
Sorry, an error occurred while loading the content.

Implementing implementation upgrade service RESTfully.

Expand Messages
  • mark69_fnd
    Dear ladies and sirs. Please, forgive me for this long post, but I have to present the full context, before actually asking my question. Whoever is going to
    Message 1 of 3 , Jul 10 12:36 AM
    • 0 Attachment
      Dear ladies and sirs.

      Please, forgive me for this long post, but I have to present the full context, before actually asking my question. Whoever is going to read this post through - thank you for your patience.

      Our product consists of a server and multiple agents and we wish to implement an agent upgrade service, so that agents can upgrade their implementation silently without human intervention.

      I would also like to make the upgrade process in one round-trip to the service, otherwise I have to deal with issues like an agent getting partial implementation and I really really do not want to allow it.

      My current implementation uses overloaded POST method with the following semantics:
      • Agent periodically sends the server a mapping between the names of the files constituting the current implementation and their respective MD5 hash values. The mapping is represented in JSON.
      • The service compares the snapshot received from the agent with the local repository containing the most up-to-date agent implementation. The service then composes the mapping between the names of the modified/new files and their respective contents (zipped) and includes it in the response. Deleted files are mapped with no content. The response is represented in JSON again.

      For instance, here is a sample request from an agent:

      POST http://il-mark-lt/NC/AgentImplService/RepositoryDelta HTTP/1.1
      Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml, application/json
      User-Agent: RestSharp 101.3.0.0
      Content-Type: application/json
      Host: il-mark-lt
      Content-Length: 4013
      Expect: 100-continue
      Accept-Encoding: gzip, deflate

      {
        "Lib1.dll": "Ud/QDKYI8CPcWTxwBhfLjQ==",
        "Lib2.dll": "2yjGP63c4PcdtkK+MQS+5g==",
        "Lib3.dll": "1slfkwOzqfj/RDEYSKDE/Q==",
        "Lib4.dll": "l7Q8q4Tn2P6DSPkZS7GMQA==",
      }

      If nothing has changed, then the service response would be:

      HTTP/1.1 200 OK
      Content-Length: 5
      Content-Type: application/json
      Server: Microsoft-HTTPAPI/2.0
      Date: Thu, 07 Jul 2011 13:51:50 GMT

      {}

      Otherwise, if for instance, Lib2 was changed, Lib3 was deleted and Lib5 was added:

      HTTP/1.1 200 OK
      Content-Length: 591
      Content-Type: application/json
      Server: Microsoft-HTTPAPI/2.0
      Date: Thu, 07 Jul 2011 13:59:16 GMT

      {
        "Lib2.dll": "H4sIAAAA........."
        "Lib3.dll": null
        "Lib5.dll": "OPUguh89........."
      }

      (I have omitted the actual contents for the sake of clarity)

      There is a mechanism on the server side that prevents the service from serving requests while the local repository is being updated, so everything is fine on that end.

      Another note to make is that it is highly unlikely that anyone else, but our agents will ever use this service.

      What I am interested to know is whether it is feasible to implement it with a single GET request utilizing some clever conditional and/or partial GET semantics?

      Thank you very much.
    • Glenn Block
      What about using ETags? With ETags the client can send an if-none-match header to do a conditional GET it will receive 304 Not Modified status as long as the
      Message 2 of 3 , Jul 10 7:45 PM
      • 0 Attachment
        What about using ETags? With ETags the client can send an if-none-match header to do a conditional GET it will receive 304 Not Modified status as long as the server files have not changed. If files have changed, the server issues an ETag header along with the response which the client holds.

        Ideally the server would then return the full file list in order to avoid server resources. An alternative would be to store the deltas for each etag. Then when a client request comes in you could determine based on the etah which files have change. As long as only deltas are stored the storage cost would be minimal.

        Sent from my Windows Phone

        From: mark69_fnd
        Sent: Sunday, July 10, 2011 6:51 PM
        To: rest-discuss@yahoogroups.com
        Subject: [rest-discuss] Implementing implementation upgrade service RESTfully.

         

        Dear ladies and sirs.

        Please, forgive me for this long post, but I have to present the full context, before actually asking my question. Whoever is going to read this post through - thank you for your patience.

        Our product consists of a server and multiple agents and we wish to implement an agent upgrade service, so that agents can upgrade their implementation silently without human intervention.

        I would also like to make the upgrade process in one round-trip to the service, otherwise I have to deal with issues like an agent getting partial implementation and I really really do not want to allow it.

        My current implementation uses overloaded POST method with the following semantics:

        • Agent periodically sends the server a mapping between the names of the files constituting the current implementation and their respective MD5 hash values. The mapping is represented in JSON.
        • The service compares the snapshot received from the agent with the local repository containing the most up-to-date agent implementation. The service then composes the mapping between the names of the modified/new files and their respective contents (zipped) and includes it in the response. Deleted files are mapped with no content. The response is represented in JSON again.

        For instance, here is a sample request from an agent:

        POST http://il-mark-lt/NC/AgentImplService/RepositoryDelta HTTP/1.1
        Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml, application/json
        User-Agent: RestSharp 101.3.0.0
        Content-Type: application/json
        Host: il-mark-lt
        Content-Length: 4013
        Expect: 100-continue
        Accept-Encoding: gzip, deflate

        {
          "Lib1.dll": "Ud/QDKYI8CPcWTxwBhfLjQ==",
          "Lib2.dll": "2yjGP63c4PcdtkK+MQS+5g==",
          "Lib3.dll": "1slfkwOzqfj/RDEYSKDE/Q==",
          "Lib4.dll": "l7Q8q4Tn2P6DSPkZS7GMQA==",
        }

        If nothing has changed, then the service response would be:

        HTTP/1.1 200 OK
        Content-Length: 5
        Content-Type: application/json
        Server: Microsoft-HTTPAPI/2.0
        Date: Thu, 07 Jul 2011 13:51:50 GMT

        {}

        Otherwise, if for instance, Lib2 was changed, Lib3 was deleted and Lib5 was added:

        HTTP/1.1 200 OK
        Content-Length: 591
        Content-Type: application/json
        Server: Microsoft-HTTPAPI/2.0
        Date: Thu, 07 Jul 2011 13:59:16 GMT

        {
          "Lib2.dll": "H4sIAAAA........."
          "Lib3.dll": null
          "Lib5.dll": "OPUguh89........."
        }

        (I have omitted the actual contents for the sake of clarity)

        There is a mechanism on the server side that prevents the service from serving requests while the local repository is being updated, so everything is fine on that end.

        Another note to make is that it is highly unlikely that anyone else, but our agents will ever use this service.

        What I am interested to know is whether it is feasible to implement it with a single GET request utilizing some clever conditional and/or partial GET semantics?

        Thank you very much.

      • Glenn Block
        1. I was thinking that a client receives an ETag that represents the current state of the files on the server. The first time the client connects it would not
        Message 3 of 3 , Jul 11 7:48 AM
        • 0 Attachment
          1. I was thinking that a client receives an ETag that represents the current state of the files on the server. The first time the client connects it would not send an ETag thus it would receive all files along with an ETag representing that snapshot. Once files change the etag would change. As the client sent an original tag as part of the if-none-match the server can use that to determine the delta of what the client has.

          2. Not sure the uri scheme matters there could be anything. The important thing is that the resource. The ETag does not specify the delta, it specifies what version of files the client has at any point in time.

          On Sun, Jul 10, 2011 at 9:21 PM, Mark Kharitonov <mark.kharitonov@...> wrote:
          Since I do not want to find myself dealing with agents getting partial upgrades the whole upgrade must be done in a single round trip. I.e. one conditional GET statement - one response.

          Now this brings the following questions:
          1. Do you mean to use multiples ETag values or encode the request file-hash mapping as a single ETag value which the server then unfolds into the original mapping?
          2. What is the name of the resource exposed by the server? Should it be something like "delta" or like "snapshot" ?

            For example, should the client GET
            http://bla-bla/agent-implementation/delta

            or
            http://bla-bla/agent-implementation/snapshot ?

            If it is delta, then the delta specification is given in the etag, which seems a bit strange to me (I refrain from utilizing the URI query component for fearing to overflow it). But if it is a well known technique - I am fine with it.

            If it is snapshot, then one has to be aware that it is only a partial snapshot, because not all the files in the repository are returned. But, then where is it expressed that the snapshot is partial? Should I employ the partial GET semantics as well?


          On 11/07/2011, at 05:45, Glenn Block wrote:

          What about using ETags? With ETags the client can send an if-none-match header to do a conditional GET it will receive 304 Not Modified status as long as the server files have not changed. If files have changed, the server issues an ETag header along with the response which the client holds.

          Ideally the server would then return the full file list in order to avoid server resources. An alternative would be to store the deltas for each etag. Then when a client request comes in you could determine based on the etah which files have change. As long as only deltas are stored the storage cost would be minimal.

          Sent from my Windows Phone

          From: mark69_fnd
          Sent: Sunday, July 10, 2011 6:51 PM
          To: rest-discuss@yahoogroups.com
          Subject: [rest-discuss] Implementing implementation upgrade service RESTfully.

           

          Dear ladies and sirs.

          Please, forgive me for this long post, but I have to present the full context, before actually asking my question. Whoever is going to read this post through - thank you for your patience.

          Our product consists of a server and multiple agents and we wish to implement an agent upgrade service, so that agents can upgrade their implementation silently without human intervention.

          I would also like to make the upgrade process in one round-trip to the service, otherwise I have to deal with issues like an agent getting partial implementation and I really really do not want to allow it.

          My current implementation uses overloaded POST method with the following semantics:

          • Agent periodically sends the server a mapping between the names of the files constituting the current implementation and their respective MD5 hash values. The mapping is represented in JSON.
          • The service compares the snapshot received from the agent with the local repository containing the most up-to-date agent implementation. The service then composes the mapping between the names of the modified/new files and their respective contents (zipped) and includes it in the response. Deleted files are mapped with no content. The response is represented in JSON again.

          For instance, here is a sample request from an agent:

          POST http://il-mark-lt/NC/AgentImplService/RepositoryDelta HTTP/1.1
          Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml, application/json
          User-Agent: RestSharp 101.3.0.0
          Content-Type: application/json
          Host: il-mark-lt
          Content-Length: 4013
          Expect: 100-continue
          Accept-Encoding: gzip, deflate

          {
            "Lib1.dll": "Ud/QDKYI8CPcWTxwBhfLjQ==",
            "Lib2.dll": "2yjGP63c4PcdtkK+MQS+5g==",
            "Lib3.dll": "1slfkwOzqfj/RDEYSKDE/Q==",
            "Lib4.dll": "l7Q8q4Tn2P6DSPkZS7GMQA==",
          }

          If nothing has changed, then the service response would be:

          HTTP/1.1 200 OK
          Content-Length: 5
          Content-Type: application/json
          Server: Microsoft-HTTPAPI/2.0
          Date: Thu, 07 Jul 2011 13:51:50 GMT

          {}

          Otherwise, if for instance, Lib2 was changed, Lib3 was deleted and Lib5 was added:

          HTTP/1.1 200 OK
          Content-Length: 591
          Content-Type: application/json
          Server: Microsoft-HTTPAPI/2.0
          Date: Thu, 07 Jul 2011 13:59:16 GMT

          {
            "Lib2.dll": "H4sIAAAA........."
            "Lib3.dll": null
            "Lib5.dll": "OPUguh89........."
          }

          (I have omitted the actual contents for the sake of clarity)

          There is a mechanism on the server side that prevents the service from serving requests while the local repository is being updated, so everything is fine on that end.

          Another note to make is that it is highly unlikely that anyone else, but our agents will ever use this service.

          What I am interested to know is whether it is feasible to implement it with a single GET request utilizing some clever conditional and/or partial GET semantics?

          Thank you very much.


          ==========================================================================
          There are two kinds of people. Those whose guns are loaded and those who dig.
          (The good, the bad and the ugly).
          So let us raise our cups for our guns always be loaded.



        Your message has been successfully submitted and would be delivered to recipients shortly.