International Federation of Digital Seismograph Networks

Thread: FDSN web service specification changes proposed by IRIS

None
Started: Sept. 18, 2013, 9:39 p.m.
Last activity: Oct. 18, 2013, 12:52 p.m.
Chad Trabant
Sept. 18, 2013, 9:39 p.m.

Hello WG III members,

As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

regards,
Chad


Proposed changes:

* Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

* Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

* Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

* Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

* Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

* Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

* Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.








  • Hello,

    I have a few comments after reading the updated specification:

    - The "Required" column in the specification is a little ambiguous. Either
    1) the parameter is required to be supported, which I suspect is the
    intent, or 2) the parameter is required for each request.

    - The event "text" format looks a lot like CSV, why not CSV?


    And a couple general comments, not specific to the latest changes:

    - The POST method circumvents any HTTP caching. I would suggest this be
    optional, because the requests could be broken into smaller GET requests
    that are cacheable.

    - HTTP status 413 (Request Entity Too Large) refers to the _request body_
    being too large, rather than the _response content_ being too large. It's
    possible an application firewall inserted into a network between the
    provider and client would generate this response even if the query would
    otherwise be acceptable by the provider; particularly with POST requests.

    One alternative would be 400 Bad Request when exceeding search limit, and
    specified "limit" and "count" methods so an agent could determine the
    result size was too large.


    Thanks,

    Jeremy



    On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu>wrote:


    Hello WG III members,

    As presented at the WG meeting in Gothenburg, Sweden, below is a list of
    FDSN web service additions proposed by the IRIS DMC. With the exception of
    the POST submission method to fdsnws-station, all of these changes are in
    production at the IRIS DMC.

    To provide further details and clarify how the changes would alter the
    specification, a draft version is attached with additions highlighted in
    red.

    We proposed that these additions to the 1.0 specification become version
    1.1 of the specification.

    regards,
    Chad


    Proposed changes:

    * Designate optional versus required parameters for each service
    interface. Effectively defining a minimum level of functionality that
    clients can expect. The WADL return from each service interface specifies
    which of the optional parameters are supported.

    * Allow fdsnws-station to accept arbitrary selections via the POST method.
    Support for this method is required for all 1.1 implementations of the
    service. The request is to be formatted similarly to the POST selection
    method supported by fdsnws-dataselect, where parameter key-value pairs are
    followed by a block of windowed channel requests.

    * Add optional 'format' parameter to all three interfaces. The default
    value for the fdsnws-station and fdsnws-event services is 'xml' (returning
    FDSN StationXML and QuakeML respectively), an additional value of 'text' is
    defined as returning results in a simple ASCII format (illustrated in the
    specification). The default value for fdsnws-dataselect is 'miniseed'.
    Future expansion of the accepted values for this parameter are anticipated
    to return the results in alternate formats.

    * Add optional 'nodata' parameter to all three interfaces accepting values
    of '204' and '404' to control the HTTP status code returned when a
    successful request matched no data. The default value is '204'.

    * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the
    results to matching data availability times.

    * Clarify definition of starttime and endtime parameters to explicitly
    include intersection with metadata epochs.

    * Clarify definition of minimum and maximum latitude and longitude to be
    inclusive by adding "or equal to".

    Except the POST method addition to fdsnws-station, all other additions are
    optional. Service implementations should return a WADL which appropriately
    advertises the optional features supported.









    _______________________________________________
    fdsn-wg3-products mailing list
    fdsn-wg3-products<at>iris.washington.edu
    http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products



    • Chad Trabant
      Sept. 19, 2013, 4:59 p.m.

      Hi Jeremy,

      Thanks for the feedback.

      1) Regarding ambiguity in the "Required" column of the parameter table. Good point, a sentence can be added after each service-specific table to clarify that Required refers to support and not each request. Alternatively, the column can be called "Support" and the entries can be "Required" and "Optional". Any suggestions or preferences? Earlier in the spec is language strongly imply that this designation refers to support, but it is not very close to the table.

      2) Regarding the text output in CSV. The 'text' output (event or station) can include commas in certain fields, comma-separated values are then mis-parsed. Schemes like quoted fields to contain or escape embedded commas can be a pain to parse, where as vertical bars are easily parsed in shell scripts, etc.

      The general comments:

      3) Regarding GET and HTTP caching. There are a number of use cases where a huge number of GET requests becomes a real limitation due to the overhead in the requests (thousands of requests) that the POST request capability resolves. In our experience with services so far there are very few requests that are exactly the same and therefore very limited use of HTTP caching at a level that is unaware of the data characteristics and therefore cannot subset data or recognize when to subset.

      4) Regarding HTTP status 413 usage. I agree that the HTTP spec states this is only for the request entity, for FDSN service it should technically only be used to communicate the passing of a limit on the size of a POST body that a service interface will accept.

      HTTP status 400 does not seem like a good fit either, it specifically means "The request could not be understood by the server due to malformed syntax.", which is not the case. At least a 413 has something to do with size, the message returned with the error can clarify that it refers to the size of the response.

      There seems to be no HTTP status for "Response entity is too large". Are there any other suggestions for how to communicate this condition?

      Chad

      On Sep 19, 2013, at 2:37 PM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

      Hello,

      I have a few comments after reading the updated specification:

      - The "Required" column in the specification is a little ambiguous. Either 1) the parameter is required to be supported, which I suspect is the intent, or 2) the parameter is required for each request.

      - The event "text" format looks a lot like CSV, why not CSV?


      And a couple general comments, not specific to the latest changes:

      - The POST method circumvents any HTTP caching. I would suggest this be optional, because the requests could be broken into smaller GET requests that are cacheable.

      - HTTP status 413 (Request Entity Too Large) refers to the _request body_ being too large, rather than the _response content_ being too large. It's possible an application firewall inserted into a network between the provider and client would generate this response even if the query would otherwise be acceptable by the provider; particularly with POST requests.

      One alternative would be 400 Bad Request when exceeding search limit, and specified "limit" and "count" methods so an agent could determine the result size was too large.


      Thanks,

      Jeremy



      On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu> wrote:

      Hello WG III members,

      As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

      To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

      We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

      regards,
      Chad


      Proposed changes:

      * Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

      * Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

      * Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

      * Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

      * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

      * Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

      * Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

      Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.









      _______________________________________________
      fdsn-wg3-products mailing list
      fdsn-wg3-products<at>iris.washington.edu
      http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products




      • Hi Chad,

        I'll reply inline:

        On Thu, Sep 19, 2013 at 5:59 PM, Chad Trabant <chad<at>iris.washington.edu>wrote:


        Hi Jeremy,

        Thanks for the feedback.

        1) Regarding ambiguity in the "Required" column of the parameter table.
        Good point, a sentence can be added after each service-specific table to
        clarify that Required refers to support and not each request.
        Alternatively, the column can be called "Support" and the entries can be
        "Required" and "Optional". Any suggestions or preferences? Earlier in the
        spec is language strongly imply that this designation refers to support,
        but it is not very close to the table.


        Support, with Required and Optional, would work, or 2 sections in each
        table to separate required supported parameters from optionally supported
        parameters.


        2) Regarding the text output in CSV. The 'text' output (event or station)
        can include commas in certain fields, comma-separated values are then
        mis-parsed. Schemes like quoted fields to contain or escape embedded
        commas can be a pain to parse, where as vertical bars are easily parsed in
        shell scripts, etc.


        I agree quoted CSV is tricky in shell scripts, although most languages have
        parsers available that handle this.

        It would also be useful for each of the columns in the format to be
        specified, for example what is the difference between eventid and
        contributorid?


        The general comments:

        3) Regarding GET and HTTP caching. There are a number of use cases where
        a huge number of GET requests becomes a real limitation due to the overhead
        in the requests (thousands of requests) that the POST request capability
        resolves. In our experience with services so far there are very few
        requests that are exactly the same and therefore very limited use of HTTP
        caching at a level that is unaware of the data characteristics and
        therefore cannot subset data or recognize when to subset.


        Agreed that many users will submit unique requests. However, web
        applications that consume a web service can make requests consistent across
        many users, allowing regular caching to be effective and keeping resources
        available for unique requests.

        I'm reluctant to see POST method support _required_ for something that
        would otherwise still be possible using GET.

        Maybe station web services are used differently from event web services.



        4) Regarding HTTP status 413 usage. I agree that the HTTP spec states
        this is only for the request entity, for FDSN service it should technically
        only be used to communicate the passing of a limit on the size of a POST
        body that a service interface will accept.

        HTTP status 400 does not seem like a good fit either, it specifically
        means "The request could not be understood by the server due to malformed
        syntax.", which is not the case. At least a 413 has something to do with
        size, the message returned with the error can clarify that it refers to the
        size of the response.

        There seems to be no HTTP status for "Response entity is too large". Are
        there any other suggestions for how to communicate this condition?


        After re-reading, I agree that 400 is more appropriate for syntax errors.

        Maybe 403 Forbidden: "the server understood the request, but is refusing to
        fulfill it." Text describing how it exceeds the server limit would explain
        why the request is forbidden.


        Thanks,

        Jeremy



        Chad

        On Sep 19, 2013, at 2:37 PM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

        Hello,

        I have a few comments after reading the updated specification:

        - The "Required" column in the specification is a little ambiguous.
        Either 1) the parameter is required to be supported, which I suspect is
        the intent, or 2) the parameter is required for each request.

        - The event "text" format looks a lot like CSV, why not CSV?


        And a couple general comments, not specific to the latest changes:

        - The POST method circumvents any HTTP caching. I would suggest this be
        optional, because the requests could be broken into smaller GET requests
        that are cacheable.

        - HTTP status 413 (Request Entity Too Large) refers to the _request body_
        being too large, rather than the _response content_ being too large. It's
        possible an application firewall inserted into a network between the
        provider and client would generate this response even if the query would
        otherwise be acceptable by the provider; particularly with POST requests.

        One alternative would be 400 Bad Request when exceeding search limit, and
        specified "limit" and "count" methods so an agent could determine the
        result size was too large.


        Thanks,

        Jeremy



        On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu>wrote:


        Hello WG III members,

        As presented at the WG meeting in Gothenburg, Sweden, below is a list of
        FDSN web service additions proposed by the IRIS DMC. With the exception of
        the POST submission method to fdsnws-station, all of these changes are in
        production at the IRIS DMC.

        To provide further details and clarify how the changes would alter the
        specification, a draft version is attached with additions highlighted in
        red.

        We proposed that these additions to the 1.0 specification become version
        1.1 of the specification.

        regards,
        Chad


        Proposed changes:

        * Designate optional versus required parameters for each service
        interface. Effectively defining a minimum level of functionality that
        clients can expect. The WADL return from each service interface specifies
        which of the optional parameters are supported.

        * Allow fdsnws-station to accept arbitrary selections via the POST
        method. Support for this method is required for all 1.1 implementations of
        the service. The request is to be formatted similarly to the POST
        selection method supported by fdsnws-dataselect, where parameter key-value
        pairs are followed by a block of windowed channel requests.

        * Add optional 'format' parameter to all three interfaces. The default
        value for the fdsnws-station and fdsnws-event services is 'xml' (returning
        FDSN StationXML and QuakeML respectively), an additional value of 'text' is
        defined as returning results in a simple ASCII format (illustrated in the
        specification). The default value for fdsnws-dataselect is 'miniseed'.
        Future expansion of the accepted values for this parameter are anticipated
        to return the results in alternate formats.

        * Add optional 'nodata' parameter to all three interfaces accepting
        values of '204' and '404' to control the HTTP status code returned when a
        successful request matched no data. The default value is '204'.

        * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the
        results to matching data availability times.

        * Clarify definition of starttime and endtime parameters to explicitly
        include intersection with metadata epochs.

        * Clarify definition of minimum and maximum latitude and longitude to be
        inclusive by adding "or equal to".

        Except the POST method addition to fdsnws-station, all other additions
        are optional. Service implementations should return a WADL which
        appropriately advertises the optional features supported.









        _______________________________________________
        fdsn-wg3-products mailing list
        fdsn-wg3-products<at>iris.washington.edu
        http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products






        • Hi Jeremy,

          My reply is inline below.


          Thanks for the feedback.

          1) Regarding ambiguity in the "Required" column of the parameter table. Good point, a sentence can be added after each service-specific table to clarify that Required refers to support and not each request. Alternatively, the column can be called "Support" and the entries can be "Required" and "Optional". Any suggestions or preferences? Earlier in the spec is language strongly imply that this designation refers to support, but it is not very close to the table.

          Support, with Required and Optional, would work, or 2 sections in each table to separate required supported parameters from optionally supported parameters.

          I'll work on adding the former.

          2) Regarding the text output in CSV. The 'text' output (event or station) can include commas in certain fields, comma-separated values are then mis-parsed. Schemes like quoted fields to contain or escape embedded commas can be a pain to parse, where as vertical bars are easily parsed in shell scripts, etc.

          I agree quoted CSV is tricky in shell scripts, although most languages have parsers available that handle this.

          It would also be useful for each of the columns in the format to be specified, for example what is the difference between eventid and contributorid?

          I will add brief descriptions of the columns for the text where they are ambiguous, many of them are obvious and map directly back to XML elements of the same data.

          The general comments:

          3) Regarding GET and HTTP caching. There are a number of use cases where a huge number of GET requests becomes a real limitation due to the overhead in the requests (thousands of requests) that the POST request capability resolves. In our experience with services so far there are very few requests that are exactly the same and therefore very limited use of HTTP caching at a level that is unaware of the data characteristics and therefore cannot subset data or recognize when to subset.

          Agreed that many users will submit unique requests. However, web applications that consume a web service can make requests consistent across many users, allowing regular caching to be effective and keeping resources available for unique requests.

          I'm reluctant to see POST method support _required_ for something that would otherwise still be possible using GET.

          Maybe station web services are used differently from event web services.

          Yes, much differently. The station service is, in many ways, the metadata equivalent of the dataselect service for time series where the POST method already exists and is required. This change brings these two, closely related, services more in line with each other.


          4) Regarding HTTP status 413 usage. I agree that the HTTP spec states this is only for the request entity, for FDSN service it should technically only be used to communicate the passing of a limit on the size of a POST body that a service interface will accept.

          HTTP status 400 does not seem like a good fit either, it specifically means "The request could not be understood by the server due to malformed syntax.", which is not the case. At least a 413 has something to do with size, the message returned with the error can clarify that it refers to the size of the response.

          There seems to be no HTTP status for "Response entity is too large". Are there any other suggestions for how to communicate this condition?

          After re-reading, I agree that 400 is more appropriate for syntax errors.

          Maybe 403 Forbidden: "the server understood the request, but is refusing to fulfill it." Text describing how it exceeds the server limit would explain why the request is forbidden.

          The dataselect service can support authenticated access to grant access to restricted data sets, in this scenario a 403 is commonly used to mean the credentials do no grant access to the selected data. Using a 403 for "too much data" would confuse the situation involving authenticated access.

          Unless there are others that object to using a 413 in this way and there is another code identified that does not conflict with other meanings I suggest we keep it the way it was in version 1.0.

          thanks,
          Chad


          Thanks,

          Jeremy



          Chad

          On Sep 19, 2013, at 2:37 PM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

          Hello,

          I have a few comments after reading the updated specification:

          - The "Required" column in the specification is a little ambiguous. Either 1) the parameter is required to be supported, which I suspect is the intent, or 2) the parameter is required for each request.

          - The event "text" format looks a lot like CSV, why not CSV?


          And a couple general comments, not specific to the latest changes:

          - The POST method circumvents any HTTP caching. I would suggest this be optional, because the requests could be broken into smaller GET requests that are cacheable.

          - HTTP status 413 (Request Entity Too Large) refers to the _request body_ being too large, rather than the _response content_ being too large. It's possible an application firewall inserted into a network between the provider and client would generate this response even if the query would otherwise be acceptable by the provider; particularly with POST requests.

          One alternative would be 400 Bad Request when exceeding search limit, and specified "limit" and "count" methods so an agent could determine the result size was too large.


          Thanks,

          Jeremy



          On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu> wrote:

          Hello WG III members,

          As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

          To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

          We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

          regards,
          Chad


          Proposed changes:

          * Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

          * Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

          * Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

          * Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

          * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

          * Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

          * Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

          Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.









          _______________________________________________
          fdsn-wg3-products mailing list
          fdsn-wg3-products<at>iris.washington.edu
          http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products





          • Hi Chad,

            One additional suggestion, about the "format" parameter. Could that be
            implemented via a pseudo file extension:

            /fdsnws/event/1/query.txt

            /fdsnws/event/1/query.xml


            It would address some issues users report when they try to save a file and
            browsers save it as "query".

            It is also suggested by these standards, which we try to adhere to for our
            own services:

            https://github.com/WhiteHouse/api-standards



            This is also somewhat supported by the "application.wadl" method. We added
            a "application.json" method for our own service:

            http://comcat.cr.usgs.gov/fdsnws/event/1/application.json


            We can just implement this as an extension for our own service, but thought
            you might want to consider it for the standard.


            Thanks,

            Jeremy




            On Thu, Oct 3, 2013 at 7:32 PM, Chad Trabant <chad<at>iris.washington.edu>wrote:


            Hi Jeremy,

            My reply is inline below.


            Thanks for the feedback.

            1) Regarding ambiguity in the "Required" column of the parameter table.
            Good point, a sentence can be added after each service-specific table to
            clarify that Required refers to support and not each request.
            Alternatively, the column can be called "Support" and the entries can be
            "Required" and "Optional". Any suggestions or preferences? Earlier in the
            spec is language strongly imply that this designation refers to support,
            but it is not very close to the table.


            Support, with Required and Optional, would work, or 2 sections in each
            table to separate required supported parameters from optionally supported
            parameters.


            I'll work on adding the former.

            2) Regarding the text output in CSV. The 'text' output (event or station)
            can include commas in certain fields, comma-separated values are then
            mis-parsed. Schemes like quoted fields to contain or escape embedded
            commas can be a pain to parse, where as vertical bars are easily parsed in
            shell scripts, etc.


            I agree quoted CSV is tricky in shell scripts, although most languages
            have parsers available that handle this.

            It would also be useful for each of the columns in the format to be
            specified, for example what is the difference between eventid and
            contributorid?


            I will add brief descriptions of the columns for the text where they are
            ambiguous, many of them are obvious and map directly back to XML elements
            of the same data.

            The general comments:

            3) Regarding GET and HTTP caching. There are a number of use cases where
            a huge number of GET requests becomes a real limitation due to the overhead
            in the requests (thousands of requests) that the POST request capability
            resolves. In our experience with services so far there are very few
            requests that are exactly the same and therefore very limited use of HTTP
            caching at a level that is unaware of the data characteristics and
            therefore cannot subset data or recognize when to subset.


            Agreed that many users will submit unique requests. However, web
            applications that consume a web service can make requests consistent across
            many users, allowing regular caching to be effective and keeping resources
            available for unique requests.

            I'm reluctant to see POST method support _required_ for something that
            would otherwise still be possible using GET.

            Maybe station web services are used differently from event web services.


            Yes, much differently. The station service is, in many ways, the metadata
            equivalent of the dataselect service for time series where the POST method
            already exists and is required. This change brings these two, closely
            related, services more in line with each other.



            4) Regarding HTTP status 413 usage. I agree that the HTTP spec states
            this is only for the request entity, for FDSN service it should technically
            only be used to communicate the passing of a limit on the size of a POST
            body that a service interface will accept.

            HTTP status 400 does not seem like a good fit either, it specifically
            means "The request could not be understood by the server due to malformed
            syntax.", which is not the case. At least a 413 has something to do with
            size, the message returned with the error can clarify that it refers to the
            size of the response.

            There seems to be no HTTP status for "Response entity is too large". Are
            there any other suggestions for how to communicate this condition?


            After re-reading, I agree that 400 is more appropriate for syntax errors.

            Maybe 403 Forbidden: "the server understood the request, but is refusing
            to fulfill it." Text describing how it exceeds the server limit would
            explain why the request is forbidden.


            The dataselect service can support authenticated access to grant access to
            restricted data sets, in this scenario a 403 is commonly used to mean the
            credentials do no grant access to the selected data. Using a 403 for "too
            much data" would confuse the situation involving authenticated access.

            Unless there are others that object to using a 413 in this way and there
            is another code identified that does not conflict with other meanings I
            suggest we keep it the way it was in version 1.0.

            thanks,
            Chad


            Thanks,

            Jeremy



            Chad

            On Sep 19, 2013, at 2:37 PM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

            Hello,

            I have a few comments after reading the updated specification:

            - The "Required" column in the specification is a little ambiguous.
            Either 1) the parameter is required to be supported, which I suspect is
            the intent, or 2) the parameter is required for each request.

            - The event "text" format looks a lot like CSV, why not CSV?


            And a couple general comments, not specific to the latest changes:

            - The POST method circumvents any HTTP caching. I would suggest this be
            optional, because the requests could be broken into smaller GET requests
            that are cacheable.

            - HTTP status 413 (Request Entity Too Large) refers to the _request body_
            being too large, rather than the _response content_ being too large. It's
            possible an application firewall inserted into a network between the
            provider and client would generate this response even if the query would
            otherwise be acceptable by the provider; particularly with POST requests.

            One alternative would be 400 Bad Request when exceeding search limit, and
            specified "limit" and "count" methods so an agent could determine the
            result size was too large.


            Thanks,

            Jeremy



            On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu>wrote:


            Hello WG III members,

            As presented at the WG meeting in Gothenburg, Sweden, below is a list of
            FDSN web service additions proposed by the IRIS DMC. With the exception of
            the POST submission method to fdsnws-station, all of these changes are in
            production at the IRIS DMC.

            To provide further details and clarify how the changes would alter the
            specification, a draft version is attached with additions highlighted in
            red.

            We proposed that these additions to the 1.0 specification become version
            1.1 of the specification.

            regards,
            Chad


            Proposed changes:

            * Designate optional versus required parameters for each service
            interface. Effectively defining a minimum level of functionality that
            clients can expect. The WADL return from each service interface specifies
            which of the optional parameters are supported.

            * Allow fdsnws-station to accept arbitrary selections via the POST
            method. Support for this method is required for all 1.1 implementations of
            the service. The request is to be formatted similarly to the POST
            selection method supported by fdsnws-dataselect, where parameter key-value
            pairs are followed by a block of windowed channel requests.

            * Add optional 'format' parameter to all three interfaces. The default
            value for the fdsnws-station and fdsnws-event services is 'xml' (returning
            FDSN StationXML and QuakeML respectively), an additional value of 'text' is
            defined as returning results in a simple ASCII format (illustrated in the
            specification). The default value for fdsnws-dataselect is 'miniseed'.
            Future expansion of the accepted values for this parameter are anticipated
            to return the results in alternate formats.

            * Add optional 'nodata' parameter to all three interfaces accepting
            values of '204' and '404' to control the HTTP status code returned when a
            successful request matched no data. The default value is '204'.

            * Add optional 'matchtimeseries' parameter to fdsnws-station to limit
            the results to matching data availability times.

            * Clarify definition of starttime and endtime parameters to explicitly
            include intersection with metadata epochs.

            * Clarify definition of minimum and maximum latitude and longitude to be
            inclusive by adding "or equal to".

            Except the POST method addition to fdsnws-station, all other additions
            are optional. Service implementations should return a WADL which
            appropriately advertises the optional features supported.









            _______________________________________________
            fdsn-wg3-products mailing list
            fdsn-wg3-products<at>iris.washington.edu
            http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products






            • Chad Trabant
              Oct. 18, 2013, 12:52 p.m.

              Hi Jeremy,

              That's an interesting idea. Another option would be to allow output format selection based on the HTTP Accept header included in the request.

              I suggest we consider these topics for the next version of the specification. In the mean time, we can learn from your usage of them.

              Chad

              On Oct 17, 2013, at 9:59 AM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

              Hi Chad,

              One additional suggestion, about the "format" parameter. Could that be implemented via a pseudo file extension:
              /fdsnws/event/1/query.txt
              /fdsnws/event/1/query.xml

              It would address some issues users report when they try to save a file and browsers save it as "query".

              It is also suggested by these standards, which we try to adhere to for our own services:
              https://github.com/WhiteHouse/api-standards


              This is also somewhat supported by the "application.wadl" method. We added a "application.json" method for our own service:
              http://comcat.cr.usgs.gov/fdsnws/event/1/application.json

              We can just implement this as an extension for our own service, but thought you might want to consider it for the standard.


              Thanks,

              Jeremy




              On Thu, Oct 3, 2013 at 7:32 PM, Chad Trabant <chad<at>iris.washington.edu> wrote:

              Hi Jeremy,

              My reply is inline below.


              Thanks for the feedback.

              1) Regarding ambiguity in the "Required" column of the parameter table. Good point, a sentence can be added after each service-specific table to clarify that Required refers to support and not each request. Alternatively, the column can be called "Support" and the entries can be "Required" and "Optional". Any suggestions or preferences? Earlier in the spec is language strongly imply that this designation refers to support, but it is not very close to the table.

              Support, with Required and Optional, would work, or 2 sections in each table to separate required supported parameters from optionally supported parameters.

              I'll work on adding the former.

              2) Regarding the text output in CSV. The 'text' output (event or station) can include commas in certain fields, comma-separated values are then mis-parsed. Schemes like quoted fields to contain or escape embedded commas can be a pain to parse, where as vertical bars are easily parsed in shell scripts, etc.

              I agree quoted CSV is tricky in shell scripts, although most languages have parsers available that handle this.

              It would also be useful for each of the columns in the format to be specified, for example what is the difference between eventid and contributorid?

              I will add brief descriptions of the columns for the text where they are ambiguous, many of them are obvious and map directly back to XML elements of the same data.

              The general comments:

              3) Regarding GET and HTTP caching. There are a number of use cases where a huge number of GET requests becomes a real limitation due to the overhead in the requests (thousands of requests) that the POST request capability resolves. In our experience with services so far there are very few requests that are exactly the same and therefore very limited use of HTTP caching at a level that is unaware of the data characteristics and therefore cannot subset data or recognize when to subset.

              Agreed that many users will submit unique requests. However, web applications that consume a web service can make requests consistent across many users, allowing regular caching to be effective and keeping resources available for unique requests.

              I'm reluctant to see POST method support _required_ for something that would otherwise still be possible using GET.

              Maybe station web services are used differently from event web services.

              Yes, much differently. The station service is, in many ways, the metadata equivalent of the dataselect service for time series where the POST method already exists and is required. This change brings these two, closely related, services more in line with each other.


              4) Regarding HTTP status 413 usage. I agree that the HTTP spec states this is only for the request entity, for FDSN service it should technically only be used to communicate the passing of a limit on the size of a POST body that a service interface will accept.

              HTTP status 400 does not seem like a good fit either, it specifically means "The request could not be understood by the server due to malformed syntax.", which is not the case. At least a 413 has something to do with size, the message returned with the error can clarify that it refers to the size of the response.

              There seems to be no HTTP status for "Response entity is too large". Are there any other suggestions for how to communicate this condition?

              After re-reading, I agree that 400 is more appropriate for syntax errors.

              Maybe 403 Forbidden: "the server understood the request, but is refusing to fulfill it." Text describing how it exceeds the server limit would explain why the request is forbidden.

              The dataselect service can support authenticated access to grant access to restricted data sets, in this scenario a 403 is commonly used to mean the credentials do no grant access to the selected data. Using a 403 for "too much data" would confuse the situation involving authenticated access.

              Unless there are others that object to using a 413 in this way and there is another code identified that does not conflict with other meanings I suggest we keep it the way it was in version 1.0.

              thanks,
              Chad


              Thanks,

              Jeremy



              Chad

              On Sep 19, 2013, at 2:37 PM, "Fee, Jeremy" <jmfee<at>usgs.gov> wrote:

              Hello,

              I have a few comments after reading the updated specification:

              - The "Required" column in the specification is a little ambiguous. Either 1) the parameter is required to be supported, which I suspect is the intent, or 2) the parameter is required for each request.

              - The event "text" format looks a lot like CSV, why not CSV?


              And a couple general comments, not specific to the latest changes:

              - The POST method circumvents any HTTP caching. I would suggest this be optional, because the requests could be broken into smaller GET requests that are cacheable.

              - HTTP status 413 (Request Entity Too Large) refers to the _request body_ being too large, rather than the _response content_ being too large. It's possible an application firewall inserted into a network between the provider and client would generate this response even if the query would otherwise be acceptable by the provider; particularly with POST requests.

              One alternative would be 400 Bad Request when exceeding search limit, and specified "limit" and "count" methods so an agent could determine the result size was too large.


              Thanks,

              Jeremy



              On Wed, Sep 18, 2013 at 10:39 PM, Chad Trabant <chad<at>iris.washington.edu> wrote:

              Hello WG III members,

              As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

              To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

              We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

              regards,
              Chad


              Proposed changes:

              * Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

              * Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

              * Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

              * Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

              * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

              * Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

              * Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

              Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.









              _______________________________________________
              fdsn-wg3-products mailing list
              fdsn-wg3-products<at>iris.washington.edu
              http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products







  • John Clinton
    Sept. 23, 2013, 2:19 p.m.
    Hi Chad,

    thanks for proposing these changes. We at the SED are happy to see these adopted in v1.1.

    John


    On 19 Sep 2013, at 06:39, Chad Trabant wrote:


    Hello WG III members,

    As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

    To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

    We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

    regards,
    Chad


    Proposed changes:

    * Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

    * Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

    * Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

    * Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

    * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

    * Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

    * Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

    Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.







    _______________________________________________
    fdsn-wg3-products mailing list
    fdsn-wg3-products<at>iris.washington.edu<fdsn-wg3-products<at>iris.washington.edu>
    http://www.iris.washington.edu/mailman/listinfo/fdsn-wg3-products
    <FDSN-WS-Specifications-1.1draft1.pdf>



    • Marcelo Belentani de Bianchi
      Sept. 23, 2013, 4:20 p.m.
      Dear Chad, All

      Thanks for the draft for the 1.1 version of the specification. Even that I
      was not at the meeting in Sweden I hope I can contribute with some
      comments for the document, more specifically on the ASCII format
      definitions proposed. Beyond that I think that in general they are nice
      additions to the specifications and my comments follows:

      1) The StartTime/EndTime fields changes meaning depending on the value of
      the level="value" parameter. We should add a notice for that behavior.
      When level=network they are network operation times. When the
      "level=station" is in effect station operation times.

      2) I miss the restricted flag on the ASCII format. Should we add a column
      on all levels to indicate that the network/station/channel is
      open/restricted/some like in the XML version?

      3) I could not understand what would be the contents for the "instrument"
      field before looking into one of the given examples (it is not straight
      forward from its name). I would suggest to rename it to
      InstrumentDescription (or just description) ? Moreover I don't oppose
      myself on having this field there, but I don't see any real usage, do you
      have already something in mind or it is just an informative field? I
      would still suggest that would be more useful to have a URI/URL where the
      user could fetch the response for the channel inside instrument field than
      the description itself. But for now, this is just one idea.

      With my best regards,

      Marcelo Bianchi

      Em Mon, 23 Sep 2013 11:19:49 -0300, Clinton John <jclinton<at>sed.ethz.ch>
      escreveu:

      Hi Chad,

      thanks for proposing these changes. We at the SED are happy to see these
      adopted in v1.1.

      John


      On 19 Sep 2013, at 06:39, Chad Trabant wrote:


      Hello WG III members,

      As presented at the WG meeting in Gothenburg, Sweden, below is a list of
      FDSN web service additions proposed by the IRIS DMC. With the exception
      of the POST submission method to fdsnws-station, all of these changes
      are in production at the IRIS DMC.

      To provide further details and clarify how the changes would alter the
      specification, a draft version is attached with additions highlighted in
      red.

      We proposed that these additions to the 1.0 specification become version
      1.1 of the specification.

      regards,
      Chad


      Proposed changes:

      * Designate optional versus required parameters for each service
      interface. Effectively defining a minimum level of functionality that
      clients can expect. The WADL return from each service interface
      specifies which of the optional parameters are supported.

      * Allow fdsnws-station to accept arbitrary selections via the POST
      method. Support for this method is required for all 1.1 implementations
      of the service. The request is to be formatted similarly to the POST
      selection method supported by fdsnws-dataselect, where parameter
      key-value pairs are followed by a block of windowed channel requests.

      * Add optional 'format' parameter to all three interfaces. The default
      value for the fdsnws-station and fdsnws-event services is 'xml'
      (returning FDSN StationXML and QuakeML respectively), an additional
      value of 'text' is defined as returning results in a simple ASCII format
      (illustrated in the specification). The default value for
      fdsnws-dataselect is 'miniseed'. Future expansion of the accepted
      values for this parameter are anticipated to return the results in
      alternate formats.

      * Add optional 'nodata' parameter to all three interfaces accepting
      values of '204' and '404' to control the HTTP status code returned when
      a successful request matched no data. The default value is '204'.

      * Add optional 'matchtimeseries' parameter to fdsnws-station to limit
      the results to matching data availability times.

      * Clarify definition of starttime and endtime parameters to explicitly
      include intersection with metadata epochs.

      * Clarify definition of minimum and maximum latitude and longitude to be
      inclusive by adding "or equal to".

      Except the POST method addition to fdsnws-station, all other additions
      are optional. Service implementations should return a WADL which
      appropriately advertises the optional features supported.

      --
      Dr. Marcelo Bianchi
      Centro de Sismologia da USP
      Instituto de Astronomia, Geofísica e Ciências Atmosféricas (IAG/USP)
      Rua do matão, 1226 São Paulo/SP 05508-090 Brasil
      Phone: +55 11 3091-4743 Cel: +55 15 99134-8532
      http://moho.iag.usp.br/ http://www.iag.usp.br/


      • Chad Trabant
        Sept. 25, 2013, 7:44 p.m.

        Dear Marcelo,

        Thank you for your comments.

        1) True, it is only implied that the time criteria are tested against different values depending on the level. This is not limited to the text output but also applies to the XML. I have no issue with adding a description of expected results. By the way, the IRIS DMC's implementation of the service always applies the time criteria to channel times to simplify the coding. The assumption is that channels "fit" within stations that "fit" within network time ranges. The consequence is that we cannot report networks or stations that do not have channels, so far this has not been a problem as we do not have any such metadata.

        2) We have to strike the right balance between too many fields in the text output versus keeping it small and useful. In my opinion it is OK to leave restricted status out of the text format. A caller can still limit the response to include restricted stations or not using the includrestricted parameter, this fulfills the main use case of restricted status.

        3) I have no problem changing the header name in the comment field. It is informative, but also designed to match the Instrument field of a SAC header. In the IRIS implementation this actually maps to the Network::Station::Channel::Sensor::Type field in StationXML which is the "instrument identifier" field of Blockette 52 in SEED. I propose we stick with something explicit such as "Sensor" or "InstrumentDescription" instead of anything generic, otherwise the field will not be very useful. I also do not think a URL is needed, a user already knows the URL to get more information, from the same service it requested the text from.

        At IRIS we have used this text format for a couple of years now and found it to be very useful, taking the place of the XML format for many needs. Regarding changes to this format I propose we first try this text format and discover where it is lacking, in the future we can define other text output formats are needed. There are many use cases and no one simple set of metadata will solve them all.

        best regards & obrigato,
        Chad

        On Sep 23, 2013, at 12:20 PM, Marcelo Bianchi <m.bianchi<at>iag.usp.br> wrote:

        Dear Chad, All

        Thanks for the draft for the 1.1 version of the specification. Even that I was not at the meeting in Sweden I hope I can contribute with some comments for the document, more specifically on the ASCII format definitions proposed. Beyond that I think that in general they are nice additions to the specifications and my comments follows:

        1) The StartTime/EndTime fields changes meaning depending on the value of the level="value" parameter. We should add a notice for that behavior. When level=network they are network operation times. When the "level=station" is in effect station operation times.

        2) I miss the restricted flag on the ASCII format. Should we add a column on all levels to indicate that the network/station/channel is open/restricted/some like in the XML version?

        3) I could not understand what would be the contents for the "instrument" field before looking into one of the given examples (it is not straight forward from its name). I would suggest to rename it to InstrumentDescription (or just description) ? Moreover I don't oppose myself on having this field there, but I don't see any real usage, do you have already something in mind or it is just an informative field? I would still suggest that would be more useful to have a URI/URL where the user could fetch the response for the channel inside instrument field than the description itself. But for now, this is just one idea.

        With my best regards,

        Marcelo Bianchi

        Em Mon, 23 Sep 2013 11:19:49 -0300, Clinton John <jclinton<at>sed.ethz.ch> escreveu:

        Hi Chad,

        thanks for proposing these changes. We at the SED are happy to see these adopted in v1.1.

        John


        On 19 Sep 2013, at 06:39, Chad Trabant wrote:


        Hello WG III members,

        As presented at the WG meeting in Gothenburg, Sweden, below is a list of FDSN web service additions proposed by the IRIS DMC. With the exception of the POST submission method to fdsnws-station, all of these changes are in production at the IRIS DMC.

        To provide further details and clarify how the changes would alter the specification, a draft version is attached with additions highlighted in red.

        We proposed that these additions to the 1.0 specification become version 1.1 of the specification.

        regards,
        Chad


        Proposed changes:

        * Designate optional versus required parameters for each service interface. Effectively defining a minimum level of functionality that clients can expect. The WADL return from each service interface specifies which of the optional parameters are supported.

        * Allow fdsnws-station to accept arbitrary selections via the POST method. Support for this method is required for all 1.1 implementations of the service. The request is to be formatted similarly to the POST selection method supported by fdsnws-dataselect, where parameter key-value pairs are followed by a block of windowed channel requests.

        * Add optional 'format' parameter to all three interfaces. The default value for the fdsnws-station and fdsnws-event services is 'xml' (returning FDSN StationXML and QuakeML respectively), an additional value of 'text' is defined as returning results in a simple ASCII format (illustrated in the specification). The default value for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted values for this parameter are anticipated to return the results in alternate formats.

        * Add optional 'nodata' parameter to all three interfaces accepting values of '204' and '404' to control the HTTP status code returned when a successful request matched no data. The default value is '204'.

        * Add optional 'matchtimeseries' parameter to fdsnws-station to limit the results to matching data availability times.

        * Clarify definition of starttime and endtime parameters to explicitly include intersection with metadata epochs.

        * Clarify definition of minimum and maximum latitude and longitude to be inclusive by adding "or equal to".

        Except the POST method addition to fdsnws-station, all other additions are optional. Service implementations should return a WADL which appropriately advertises the optional features supported.

        --
        Dr. Marcelo Bianchi
        Centro de Sismologia da USP
        Instituto de Astronomia, Geofísica e Ciências Atmosféricas (IAG/USP)
        Rua do matão, 1226 São Paulo/SP 05508-090 Brasil
        Phone: +55 11 3091-4743 Cel: +55 15 99134-8532
        http://moho.iag.usp.br/ http://www.iag.usp.br/



        • Marcelo Belentani de Bianchi
          Oct. 1, 2013, 2:29 p.m.
          Dear Chad,

          Em Wed, 25 Sep 2013 23:44:36 -0300, Chad Trabant
          <chad<at>iris.washington.edu> escreveu:
          Dear Marcelo,

          Thank you for your comments.

          1) True, it is only implied that the time criteria are tested against
          different values depending on the level. This is not limited to the
          text output but also applies to the XML. I have no issue with adding a
          description of expected results. By the way, the IRIS DMC's
          implementation of the service always applies the time criteria to
          channel times to simplify the coding. The assumption is that channels
          "fit" within stations that "fit" within network time ranges. The
          consequence is that we cannot report networks or stations that do not
          have channels, so far this has not been a problem as we do not have any
          such metadata.

          That would be very nice to see better described so people could implement a
          coherent service.

          2) We have to strike the right balance between too many fields in the
          text output versus keeping it small and useful. In my opinion it is OK
          to leave restricted status out of the text format. A caller can still
          limit the response to include restricted stations or not using the
          includrestricted parameter, this fulfills the main use case of
          restricted status.

          This I also agree, it is necessary to have this balance always. It is true
          that most of the use cases should be covered by the flags of the service
          itself
          independent of the output format.

          3) I have no problem changing the header name in the comment field. It
          is informative, but also designed to match the Instrument field of a SAC
          header. In the IRIS implementation this actually maps to the
          Network::Station::Channel::Sensor::Type field in StationXML which is the
          "instrument identifier" field of Blockette 52 in SEED. I propose we
          stick with something explicit such as "Sensor" or
          "InstrumentDescription" instead of anything generic, otherwise the field
          will not be very useful. I also do not think a URL is needed, a user
          already knows the URL to get more information, from the same service it
          requested the text from.

          At IRIS we have used this text format for a couple of years now and
          found it to be very useful, taking the place of the XML format for many
          needs. Regarding changes to this format I propose we first try this
          text format and discover where it is lacking, in the future we can
          define other text output formats are needed. There are many use cases
          and no one simple set of metadata will solve them all.

          Ok. I would just suggest that the description of the fields to be added
          or a just a note specifically for this column to avoid confusion
          similar to the startime, endtime issue raised at point 1.

          With my best regards,

          Abraços,

          Marcelo

          best regards & obrigato,
          Chad

          On Sep 23, 2013, at 12:20 PM, Marcelo Bianchi <m.bianchi<at>iag.usp.br>
          wrote:

          Dear Chad, All

          Thanks for the draft for the 1.1 version of the specification. Even
          that I was not at the meeting in Sweden I hope I can contribute with
          some comments for the document, more specifically on the ASCII format
          definitions proposed. Beyond that I think that in general they are nice
          additions to the specifications and my comments follows:

          1) The StartTime/EndTime fields changes meaning depending on the value
          of the level="value" parameter. We should add a notice for that
          behavior. When level=network they are network operation times. When the
          "level=station" is in effect station operation times.

          2) I miss the restricted flag on the ASCII format. Should we add a
          column on all levels to indicate that the network/station/channel is
          open/restricted/some like in the XML version?

          3) I could not understand what would be the contents for the
          "instrument" field before looking into one of the given examples (it is
          not straight forward from its name). I would suggest to rename it to
          InstrumentDescription (or just description) ? Moreover I don't oppose
          myself on having this field there, but I don't see any real usage, do
          you have already something in mind or it is just an informative field?
          I would still suggest that would be more useful to have a URI/URL where
          the user could fetch the response for the channel inside instrument
          field than the description itself. But for now, this is just one idea.

          With my best regards,

          Marcelo Bianchi

          Em Mon, 23 Sep 2013 11:19:49 -0300, Clinton John
          <jclinton<at>sed.ethz.ch> escreveu:

          Hi Chad,

          thanks for proposing these changes. We at the SED are happy to see
          these adopted in v1.1.

          John


          On 19 Sep 2013, at 06:39, Chad Trabant wrote:


          Hello WG III members,

          As presented at the WG meeting in Gothenburg, Sweden, below is a list
          of FDSN web service additions proposed by the IRIS DMC. With the
          exception of the POST submission method to fdsnws-station, all of
          these changes are in production at the IRIS DMC.

          To provide further details and clarify how the changes would alter the
          specification, a draft version is attached with additions highlighted
          in red.

          We proposed that these additions to the 1.0 specification become
          version 1.1 of the specification.

          regards,
          Chad


          Proposed changes:

          * Designate optional versus required parameters for each service
          interface. Effectively defining a minimum level of functionality that
          clients can expect. The WADL return from each service interface
          specifies which of the optional parameters are supported.

          * Allow fdsnws-station to accept arbitrary selections via the POST
          method. Support for this method is required for all 1.1
          implementations of the service. The request is to be formatted
          similarly to the POST selection method supported by fdsnws-dataselect,
          where parameter key-value pairs are followed by a block of windowed
          channel requests.

          * Add optional 'format' parameter to all three interfaces. The
          default value for the fdsnws-station and fdsnws-event services is
          'xml' (returning FDSN StationXML and QuakeML respectively), an
          additional value of 'text' is defined as returning results in a simple
          ASCII format (illustrated in the specification). The default value
          for fdsnws-dataselect is 'miniseed'. Future expansion of the accepted
          values for this parameter are anticipated to return the results in
          alternate formats.

          * Add optional 'nodata' parameter to all three interfaces accepting
          values of '204' and '404' to control the HTTP status code returned
          when a successful request matched no data. The default value is '204'.

          * Add optional 'matchtimeseries' parameter to fdsnws-station to limit
          the results to matching data availability times.

          * Clarify definition of starttime and endtime parameters to explicitly
          include intersection with metadata epochs.

          * Clarify definition of minimum and maximum latitude and longitude to
          be inclusive by adding "or equal to".

          Except the POST method addition to fdsnws-station, all other additions
          are optional. Service implementations should return a WADL which
          appropriately advertises the optional features supported.

          --
          Dr. Marcelo Bianchi
          Centro de Sismologia da USP
          Instituto de Astronomia, Geofísica e Ciências Atmosféricas (IAG/USP)
          Rua do matão, 1226 São Paulo/SP 05508-090 Brasil
          Phone: +55 11 3091-4743 Cel: +55 15 99134-8532
          http://moho.iag.usp.br/ http://www.iag.usp.br/



          --
          Dr. Marcelo Bianchi
          Centro de Sismologia da USP
          Instituto de Astronomia, Geofísica e Ciências Atmosféricas (IAG/USP)
          Rua do matão, 1226 São Paulo/SP 05508-090 Brasil
          Phone: +55 11 3091-4743 Cel: +55 15 99134-8532
          http://moho.iag.usp.br/ http://www.iag.usp.br/