International Federation of Digital Seismograph Networks

Thread: Force UTC times in the StationXML schema

None
Started: Oct. 26, 2016, 12:38 p.m.
Last activity: Oct. 27, 2016, 4:27 p.m.
Lion Krischer
Oct. 26, 2016, 12:38 p.m.
Dear all,


I propose a small change to be included in the next StationXML schema
version that would force all datetimes to be explicitly marked as being
in UTC.

Technical details can be found here:
https://github.com/FDSN/StationXML/pull/12

The proposed change forces all datetimes in a StationXML file to end
with `Z`. This explicitly marks them as being in UTC which (according to
the SEED standard) must be given in any case.

# Motivation

It is currently possible to have non-UTC datetimes in a StationXML file
(e.g. `2013-01-01T00:00:00+07:00`). While this is not allowed according
to the SEED standard the schema does not prevent it and any XML file
with this would still validate just fine.

The forced `Z` at the end explicitly marks a time as being UTC and as a
consequence also forbids setting any time zone.

While the schema alone will never be enough to define the semantics of
StationXML it should be sufficient to define its syntax. This change is
a small step in that direction.

StationXML would additionally gain a more consistent datetime
representation.

# Potential Impact

Little. Existing implementations would have to be changed to add a `Z`
at the end and remove time zones (if any). Considering implementations
would have to at least up the version number if they update to a new
StationXML version this is really no big problem.

# Other Possibilities

It would also be possibly (with a more complex regular expression) to
not have the `Z` at the end and still disallow timezones. I personally
prefer the current approach as it is more explicit.


All the best,

Lion

  • Fabian Euchner
    Oct. 26, 2016, 1:25 p.m.
    Hi all,

    I fully agree with Lion. Omitting the time zone information (although
    permitted in the xs:dateTime type) may lead XML processors to use the local
    time zone set in the locale of the processing machine. This has the potential
    to introduce subtle errors that are hard to detect.

    Regards,
    Fabian

    Dear all,


    I propose a small change to be included in the next StationXML schema
    version that would force all datetimes to be explicitly marked as being
    in UTC.

    Technical details can be found here:
    https://github.com/FDSN/StationXML/pull/12

    The proposed change forces all datetimes in a StationXML file to end
    with `Z`. This explicitly marks them as being in UTC which (according to
    the SEED standard) must be given in any case.

    # Motivation

    It is currently possible to have non-UTC datetimes in a StationXML file
    (e.g. `2013-01-01T00:00:00+07:00`). While this is not allowed according
    to the SEED standard the schema does not prevent it and any XML file
    with this would still validate just fine.

    The forced `Z` at the end explicitly marks a time as being UTC and as a
    consequence also forbids setting any time zone.

    While the schema alone will never be enough to define the semantics of
    StationXML it should be sufficient to define its syntax. This change is
    a small step in that direction.

    StationXML would additionally gain a more consistent datetime
    representation.

    # Potential Impact

    Little. Existing implementations would have to be changed to add a `Z`
    at the end and remove time zones (if any). Considering implementations
    would have to at least up the version number if they update to a new
    StationXML version this is really no big problem.

    # Other Possibilities

    It would also be possibly (with a more complex regular expression) to
    not have the `Z` at the end and still disallow timezones. I personally
    prefer the current approach as it is more explicit.


    All the best,

    Lion

    ----------------------
    FDSN Working Group II
    (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

    Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
    Update subscription preferences at http://www.fdsn.org/account/profile/

    --
    -----------------------------------------------------------------------------
    Fabian Euchner phone +41 44 633 7178
    Institute of Geophysics fax +41 44 633 1065
    ETH Zurich, NO F5 e-mail fabian<at>sed.ethz.ch
    Sonneggstrasse 5 orcid.org/0000-0001-6340-7439
    8092 Zurich (Switzerland)
    -----------------------------------------------------------------------------
    QuakeML http://quakeml.org QuakePy http://quakepy.org
    CSEP http://www.cseptesting.org/centers/eth
    -----------------------------------------------------------------------------

  • Joachim Saul
    Oct. 26, 2016, 4:15 p.m.
    Lion Krischer wrote on 10/26/2016 12:39 PM:
    I propose a small change to be included in the next StationXML schema
    version that would force all datetimes to be explicitly marked as being
    in UTC.

    Technical details can be found here:
    https://github.com/FDSN/StationXML/pull/12

    The proposed change forces all datetimes in a StationXML file to end
    with `Z`. This explicitly marks them as being in UTC which (according to
    the SEED standard) must be given in any case.

    Forcing the time to be UTC is a very good idea.

    My concern would be existing web fdsnws implementations as well as
    existing StationXML documents. Requiring a match with ".*Z" would render
    invalid most of what is now perfectly valid StationXML. Webservices
    could probably be updated relatively quickly to support a new time
    format. I am more worried about existing XML documents that suddenly
    would become invalid and which parsers might refuse to read. Not all
    StationXML is created by webservices on the fly.

    A pattern that also matches existing time strings *without* time zone
    would be better. This could be something like "[0-9-]+T[0-9.:Z]+" rather
    than just ".*Z". Note that this pattern does not enforce a time string
    ending with "Z" but only enforces a limited set of characters, which
    does not include "+", "-" or space after the "T".

    Assuming that time strings containing time zones are currently rarely
    used in StationXML anyway (if at all), the impact of this pattern will
    be minimal.

    The documentation would need a sentence clarifying that time strings
    *shall* (not *must*) end with a "Z".

    Cheers
    Joachim


    • Lion Krischer
      Oct. 26, 2016, 6:31 p.m.
      Hi Joachim,


      this is not an actual concern as the proposal would only affect the next
      StationXML version. The current version (1.0) will not change and files
      that are valid against it will naturally always remain valid.

      If and once a new StationXML + schema version is introduced all
      StationXML files will have to explicitly state that they adhere to the
      new schema by changing their stationxml namespace. As files and
      implementations have to be updated for this to happen in any case, the
      additional `Z` is only a minor change.

      Until that point the additional `Z` could be considered a recommendation
      and it is of course also valid against the current version of the schema.

      In general with data formats I think that it is always a good thing to
      limit choices and possibilities wherever possible.


      Cheers!

      Lion




      On 26/10/16 16:16, Joachim Saul wrote:
      Lion Krischer wrote on 10/26/2016 12:39 PM:
      I propose a small change to be included in the next StationXML schema
      version that would force all datetimes to be explicitly marked as being
      in UTC.

      Technical details can be found here:
      https://github.com/FDSN/StationXML/pull/12

      The proposed change forces all datetimes in a StationXML file to end
      with `Z`. This explicitly marks them as being in UTC which (according to
      the SEED standard) must be given in any case.

      Forcing the time to be UTC is a very good idea.

      My concern would be existing web fdsnws implementations as well as
      existing StationXML documents. Requiring a match with ".*Z" would render
      invalid most of what is now perfectly valid StationXML. Webservices
      could probably be updated relatively quickly to support a new time
      format. I am more worried about existing XML documents that suddenly
      would become invalid and which parsers might refuse to read. Not all
      StationXML is created by webservices on the fly.

      A pattern that also matches existing time strings *without* time zone
      would be better. This could be something like "[0-9-]+T[0-9.:Z]+" rather
      than just ".*Z". Note that this pattern does not enforce a time string
      ending with "Z" but only enforces a limited set of characters, which
      does not include "+", "-" or space after the "T".

      Assuming that time strings containing time zones are currently rarely
      used in StationXML anyway (if at all), the impact of this pattern will
      be minimal.

      The documentation would need a sentence clarifying that time strings
      *shall* (not *must*) end with a "Z".

      Cheers
      Joachim


      ----------------------
      FDSN Working Group II (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

      Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
      Update subscription preferences at http://www.fdsn.org/account/profile/


      • Chad Trabant
        Oct. 26, 2016, 9:53 a.m.

        Hi Lion and WG2,

        In principle I agree with the addition of a UTC time zone indicator.

        The practical issue when such a change is introduced. If you are proposing that we only include this at the next incompatible version, then it makes sense as is. Alternatively, if we would like to adopt this in the next backward-compatible version iteration, then the suggestion by Joachim to make the Z optional is the way to go. I think we should be prepared to do both. At this point it is unclear what kind of next version of StationXML is going to happen or even when it is going to happen.

        In addition to the Potential Impact stated in the proposal that producers would need a simple modifications and the impacts Joachim raised, I add that there are implications for web service clients and subsequent readers of StationXML. I think there are a lot more clients and readers than implementations/producers, which are, quite frankly, the easy part.

        Chad

        On Oct 26, 2016, at 9:32 AM, Lion Krischer <lion.krischer<at>gmail.com> wrote:

        Hi Joachim,


        this is not an actual concern as the proposal would only affect the next
        StationXML version. The current version (1.0) will not change and files
        that are valid against it will naturally always remain valid.

        If and once a new StationXML + schema version is introduced all
        StationXML files will have to explicitly state that they adhere to the
        new schema by changing their stationxml namespace. As files and
        implementations have to be updated for this to happen in any case, the
        additional `Z` is only a minor change.

        Until that point the additional `Z` could be considered a recommendation
        and it is of course also valid against the current version of the schema.

        In general with data formats I think that it is always a good thing to
        limit choices and possibilities wherever possible.


        Cheers!

        Lion

        On 26/10/16 16:16, Joachim Saul wrote:
        Lion Krischer wrote on 10/26/2016 12:39 PM:
        I propose a small change to be included in the next StationXML schema
        version that would force all datetimes to be explicitly marked as being
        in UTC.

        Technical details can be found here:
        https://github.com/FDSN/StationXML/pull/12

        The proposed change forces all datetimes in a StationXML file to end
        with `Z`. This explicitly marks them as being in UTC which (according to
        the SEED standard) must be given in any case.

        Forcing the time to be UTC is a very good idea.

        My concern would be existing web fdsnws implementations as well as
        existing StationXML documents. Requiring a match with ".*Z" would render
        invalid most of what is now perfectly valid StationXML. Webservices
        could probably be updated relatively quickly to support a new time
        format. I am more worried about existing XML documents that suddenly
        would become invalid and which parsers might refuse to read. Not all
        StationXML is created by webservices on the fly.

        A pattern that also matches existing time strings *without* time zone
        would be better. This could be something like "[0-9-]+T[0-9.:Z]+" rather
        than just ".*Z". Note that this pattern does not enforce a time string
        ending with "Z" but only enforces a limited set of characters, which
        does not include "+", "-" or space after the "T".

        Assuming that time strings containing time zones are currently rarely
        used in StationXML anyway (if at all), the impact of this pattern will
        be minimal.

        The documentation would need a sentence clarifying that time strings
        *shall* (not *must*) end with a "Z".

        Cheers
        Joachim


        ----------------------
        FDSN Working Group II (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

        Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
        Update subscription preferences at http://www.fdsn.org/account/profile/


        ----------------------
        FDSN Working Group II (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

        Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
        Update subscription preferences at http://www.fdsn.org/account/profile/


        • Lion Krischer
          Oct. 26, 2016, 8:49 p.m.
          Hi Chad and others,


          as soon as the addition of the UTC time zone indicator is forced one
          effectively creates a new version of StationXML and this really should
          be reflected in an incremented version number. Otherwise the schema
          version number becomes meaningless.

          Until that point it can happily be treated as a recommendation as the
          `Z` is also fully valid with the current schema. If it breaks readers it
          is a bug and should be fixed.

          But I guess this is pretty much what you are talking about just from a
          different perspective.

          In any case and independent of this particular proposal there has to be
          some path to update StationXML and move forward. Forcing the time zone
          indicator is about the smallest change I can think of so it might be a
          good test bed to see what happens when a new StationXML version is
          introduced.


          All the best,

          Lion




          On 26/10/16 18:54, Chad Trabant wrote:

          Hi Lion and WG2,

          In principle I agree with the addition of a UTC time zone indicator.

          The practical issue when such a change is introduced. If you are
          proposing that we only include this at the next incompatible version,
          then it makes sense as is. Alternatively, if we would like to adopt
          this in the next backward-compatible version iteration, then the
          suggestion by Joachim to make the Z optional is the way to go. I
          think we should be prepared to do both. At this point it is unclear
          what kind of next version of StationXML is going to happen or even
          when it is going to happen.

          In addition to the Potential Impact stated in the proposal that
          producers would need a simple modifications and the impacts Joachim
          raised, I add that there are implications for web service clients and
          subsequent readers of StationXML. I think there are a lot more
          clients and readers than implementations/producers, which are, quite
          frankly, the easy part.

          Chad

          On Oct 26, 2016, at 9:32 AM, Lion Krischer
          <lion.krischer<at>gmail.com> wrote:

          Hi Joachim,


          this is not an actual concern as the proposal would only affect the
          next StationXML version. The current version (1.0) will not change
          and files that are valid against it will naturally always remain
          valid.

          If and once a new StationXML + schema version is introduced all
          StationXML files will have to explicitly state that they adhere to
          the new schema by changing their stationxml namespace. As files
          and implementations have to be updated for this to happen in any
          case, the additional `Z` is only a minor change.

          Until that point the additional `Z` could be considered a
          recommendation and it is of course also valid against the current
          version of the schema.

          In general with data formats I think that it is always a good thing
          to limit choices and possibilities wherever possible.


          Cheers!

          Lion

          On 26/10/16 16:16, Joachim Saul wrote:
          Lion Krischer wrote on 10/26/2016 12:39 PM:
          I propose a small change to be included in the next StationXML
          schema version that would force all datetimes to be explicitly
          marked as being in UTC.

          Technical details can be found here:
          https://github.com/FDSN/StationXML/pull/12

          The proposed change forces all datetimes in a StationXML file
          to end with `Z`. This explicitly marks them as being in UTC
          which (according to the SEED standard) must be given in any
          case.

          Forcing the time to be UTC is a very good idea.

          My concern would be existing web fdsnws implementations as well
          as existing StationXML documents. Requiring a match with ".*Z"
          would render invalid most of what is now perfectly valid
          StationXML. Webservices could probably be updated relatively
          quickly to support a new time format. I am more worried about
          existing XML documents that suddenly would become invalid and
          which parsers might refuse to read. Not all StationXML is created
          by webservices on the fly.

          A pattern that also matches existing time strings *without* time
          zone would be better. This could be something like
          "[0-9-]+T[0-9.:Z]+" rather than just ".*Z". Note that this
          pattern does not enforce a time string ending with "Z" but only
          enforces a limited set of characters, which does not include "+",
          "-" or space after the "T".

          Assuming that time strings containing time zones are currently
          rarely used in StationXML anyway (if at all), the impact of this
          pattern will be minimal.

          The documentation would need a sentence clarifying that time
          strings *shall* (not *must*) end with a "Z".

          Cheers Joachim


          ---------------------- FDSN Working Group II
          (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

          Sent from the FDSN Message Center
          (http://www.fdsn.org/message-center/) Update subscription
          preferences at http://www.fdsn.org/account/profile/


          ---------------------- FDSN Working Group II
          (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

          Sent from the FDSN Message Center
          (http://www.fdsn.org/message-center/) Update subscription
          preferences at http://www.fdsn.org/account/profile/


          ---------------------- FDSN Working Group II
          (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

          Sent from the FDSN Message Center
          (http://www.fdsn.org/message-center/) Update subscription preferences
          at http://www.fdsn.org/account/profile/


          • Chad Trabant
            Oct. 26, 2016, 1:42 p.m.

            In any case and independent of this particular proposal there has to be
            some path to update StationXML and move forward. Forcing the time zone
            indicator is about the smallest change I can think of so it might be a
            good test bed to see what happens when a new StationXML version is
            introduced.

            Versioning to provide a path forward was built into the StationXML from the beginning. How the versions will progress and their meaning is documented in the comments at top of the schema, and now pasted here:

            Versioning for FDSN StationXML:

            The 'version' attribute of the schema definition identifies the version of the schema. This
            version is not enforced when validating documents.

            The required 'schemaVersion' attribute of the root element identifies the version of the schema
            that the document is compatible with. Validation only requires that a value is present but
            not that it matches the schema used for validation.

            The targetNamespace of the document identifies the major version of the schema and document,
            version 1.x of the schema uses a target namespace of "http://www.fdsn.org/xml/station/1".
            All minor versions of a will be backwards compatible with previous minor releases. For
            example, all 1.x schemas are backwards compatible with and will validate documents for 1.0.
            Major changes to the schema that would break backwards compabibility will increment the major
            version number, e.g. 2.0, and the namespace, e.g. "http://www.fdsn.org/xml/station/2".

            This combination of attributes and targetNamespaces allows the schema and documents to be
            versioned and allows the schema to be updated with backward compatible changes (e.g. 1.2)
            and still validate documents created for previous major versions of the schema (e.g. 1.0).



            On 26/10/16 18:54, Chad Trabant wrote:

            Hi Lion and WG2,

            In principle I agree with the addition of a UTC time zone indicator.

            The practical issue when such a change is introduced. If you are
            proposing that we only include this at the next incompatible version,
            then it makes sense as is. Alternatively, if we would like to adopt
            this in the next backward-compatible version iteration, then the
            suggestion by Joachim to make the Z optional is the way to go. I
            think we should be prepared to do both. At this point it is unclear
            what kind of next version of StationXML is going to happen or even
            when it is going to happen.

            In addition to the Potential Impact stated in the proposal that
            producers would need a simple modifications and the impacts Joachim
            raised, I add that there are implications for web service clients and
            subsequent readers of StationXML. I think there are a lot more
            clients and readers than implementations/producers, which are, quite
            frankly, the easy part.

            Chad

            On Oct 26, 2016, at 9:32 AM, Lion Krischer
            <lion.krischer<at>gmail.com> wrote:

            Hi Joachim,


            this is not an actual concern as the proposal would only affect the
            next StationXML version. The current version (1.0) will not change
            and files that are valid against it will naturally always remain
            valid.

            If and once a new StationXML + schema version is introduced all
            StationXML files will have to explicitly state that they adhere to
            the new schema by changing their stationxml namespace. As files
            and implementations have to be updated for this to happen in any
            case, the additional `Z` is only a minor change.

            Until that point the additional `Z` could be considered a
            recommendation and it is of course also valid against the current
            version of the schema.

            In general with data formats I think that it is always a good thing
            to limit choices and possibilities wherever possible.


            Cheers!

            Lion

            On 26/10/16 16:16, Joachim Saul wrote:
            Lion Krischer wrote on 10/26/2016 12:39 PM:
            I propose a small change to be included in the next StationXML
            schema version that would force all datetimes to be explicitly
            marked as being in UTC.

            Technical details can be found here:
            https://github.com/FDSN/StationXML/pull/12

            The proposed change forces all datetimes in a StationXML file
            to end with `Z`. This explicitly marks them as being in UTC
            which (according to the SEED standard) must be given in any
            case.

            Forcing the time to be UTC is a very good idea.

            My concern would be existing web fdsnws implementations as well
            as existing StationXML documents. Requiring a match with ".*Z"
            would render invalid most of what is now perfectly valid
            StationXML. Webservices could probably be updated relatively
            quickly to support a new time format. I am more worried about
            existing XML documents that suddenly would become invalid and
            which parsers might refuse to read. Not all StationXML is created
            by webservices on the fly.

            A pattern that also matches existing time strings *without* time
            zone would be better. This could be something like
            "[0-9-]+T[0-9.:Z]+" rather than just ".*Z". Note that this
            pattern does not enforce a time string ending with "Z" but only
            enforces a limited set of characters, which does not include "+",
            "-" or space after the "T".

            Assuming that time strings containing time zones are currently
            rarely used in StationXML anyway (if at all), the impact of this
            pattern will be minimal.

            The documentation would need a sentence clarifying that time
            strings *shall* (not *must*) end with a "Z".

            Cheers Joachim


            ---------------------- FDSN Working Group II
            (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

            Sent from the FDSN Message Center
            (http://www.fdsn.org/message-center/) Update subscription
            preferences at http://www.fdsn.org/account/profile/


            ---------------------- FDSN Working Group II
            (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

            Sent from the FDSN Message Center
            (http://www.fdsn.org/message-center/) Update subscription
            preferences at http://www.fdsn.org/account/profile/


            ---------------------- FDSN Working Group II
            (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

            Sent from the FDSN Message Center
            (http://www.fdsn.org/message-center/) Update subscription preferences
            at http://www.fdsn.org/account/profile/



            • Jeremy Fee
              Oct. 26, 2016, 2:59 p.m.
              Is the concern that the times aren't in UTC, or that times are being
              specified without _any_ timezone? When a non-UTC timezone is specified,
              it's at least an explicit time that can easily be converted to UTC.

              The Quakeml spec deals with similar issues by saying times without an
              explicit timezone should be interpreted as UTC (contrary to regular xml
              dateTime). This requires a little parser hoop-jumping but is maybe easier
              than introducing a new major version.


              Thanks,

              Jeremy


              On Wed, Oct 26, 2016 at 2:43 PM, Chad Trabant <chad<at>iris.washington.edu>
              wrote:


              In any case and independent of this particular proposal there has to be
              some path to update StationXML and move forward. Forcing the time zone
              indicator is about the smallest change I can think of so it might be a
              good test bed to see what happens when a new StationXML version is
              introduced.


              Versioning to provide a path forward was built into the StationXML from
              the beginning. How the versions will progress and their meaning is
              documented in the comments at top of the schema, and now pasted here:

              Versioning for FDSN StationXML:

              The 'version' attribute of the schema definition identifies the version of
              the schema. This
              version is not enforced when validating documents.

              The required 'schemaVersion' attribute of the root element identifies the
              version of the schema
              that the document is compatible with. Validation only requires that a
              value is present but
              not that it matches the schema used for validation.

              The targetNamespace of the document identifies the major version of the
              schema and document,
              version 1.x of the schema uses a target namespace of "
              http://www.fdsn.org/xml/station/1".
              All minor versions of a will be backwards compatible with previous minor
              releases. For
              example, all 1.x schemas are backwards compatible with and will validate
              documents for 1.0.
              Major changes to the schema that would break backwards compabibility will
              increment the major
              version number, e.g. 2.0, and the namespace, e.g. "
              http://www.fdsn.org/xml/station/2".

              This combination of attributes and targetNamespaces allows the schema and
              documents to be
              versioned and allows the schema to be updated with backward compatible
              changes (e.g. 1.2)
              and still validate documents created for previous major versions of the
              schema (e.g. 1.0).



              On 26/10/16 18:54, Chad Trabant wrote:


              Hi Lion and WG2,

              In principle I agree with the addition of a UTC time zone indicator.

              The practical issue when such a change is introduced. If you are
              proposing that we only include this at the next incompatible version,
              then it makes sense as is. Alternatively, if we would like to adopt
              this in the next backward-compatible version iteration, then the
              suggestion by Joachim to make the Z optional is the way to go. I
              think we should be prepared to do both. At this point it is unclear
              what kind of next version of StationXML is going to happen or even
              when it is going to happen.

              In addition to the Potential Impact stated in the proposal that
              producers would need a simple modifications and the impacts Joachim
              raised, I add that there are implications for web service clients and
              subsequent readers of StationXML. I think there are a lot more
              clients and readers than implementations/producers, which are, quite
              frankly, the easy part.

              Chad

              On Oct 26, 2016, at 9:32 AM, Lion Krischer
              <lion.krischer<at>gmail.com> wrote:

              Hi Joachim,


              this is not an actual concern as the proposal would only affect the
              next StationXML version. The current version (1.0) will not change
              and files that are valid against it will naturally always remain
              valid.

              If and once a new StationXML + schema version is introduced all
              StationXML files will have to explicitly state that they adhere to
              the new schema by changing their stationxml namespace. As files
              and implementations have to be updated for this to happen in any
              case, the additional `Z` is only a minor change.

              Until that point the additional `Z` could be considered a
              recommendation and it is of course also valid against the current
              version of the schema.

              In general with data formats I think that it is always a good thing
              to limit choices and possibilities wherever possible.


              Cheers!

              Lion

              On 26/10/16 16:16, Joachim Saul wrote:

              Lion Krischer wrote on 10/26/2016 12:39 PM:

              I propose a small change to be included in the next StationXML
              schema version that would force all datetimes to be explicitly
              marked as being in UTC.

              Technical details can be found here:
              https://github.com/FDSN/StationXML/pull/12

              The proposed change forces all datetimes in a StationXML file
              to end with `Z`. This explicitly marks them as being in UTC
              which (according to the SEED standard) must be given in any
              case.


              Forcing the time to be UTC is a very good idea.

              My concern would be existing web fdsnws implementations as well
              as existing StationXML documents. Requiring a match with ".*Z"
              would render invalid most of what is now perfectly valid
              StationXML. Webservices could probably be updated relatively
              quickly to support a new time format. I am more worried about
              existing XML documents that suddenly would become invalid and
              which parsers might refuse to read. Not all StationXML is created
              by webservices on the fly.

              A pattern that also matches existing time strings *without* time
              zone would be better. This could be something like
              "[0-9-]+T[0-9.:Z]+" rather than just ".*Z". Note that this
              pattern does not enforce a time string ending with "Z" but only
              enforces a limited set of characters, which does not include "+",
              "-" or space after the "T".

              Assuming that time strings containing time zones are currently
              rarely used in StationXML anyway (if at all), the impact of this
              pattern will be minimal.

              The documentation would need a sentence clarifying that time
              strings *shall* (not *must*) end with a "Z".

              Cheers Joachim


              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription
              preferences at http://www.fdsn.org/account/profile/


              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription
              preferences at http://www.fdsn.org/account/profile/



              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription preferences
              at http://www.fdsn.org/account/profile/




              ----------------------
              FDSN Working Group II (http://www.fdsn.org/message-
              center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
              Update subscription preferences at http://www.fdsn.org/account/profile/



            • Lion Krischer
              Oct. 27, 2016, 12:06 p.m.
              Hi,


              This combination of attributes and targetNamespaces allows the schema
              and documents to be
              versioned and allows the schema to be updated with backward compatible
              changes (e.g. 1.2)
              and still validate documents created for previous major versions of the
              schema (e.g. 1.0).


              Thanks for the clarification - I did not know this. If I read this
              correctly than this is essentially semantic versioning as is used for
              software. This does not fully carry over to data schemas as it only
              allows the addition of new and optional pieces of data or the weakening
              of some constraints for minor revisions; anything else would invalidate
              some existing data.

              Neither my proposed change nor Joachim's less restrictive version would
              be fully backwards compatible changes and thus would require a new major
              version.

              For such a minuscule change this is not worth it so I guess this should
              be postponed until enough changes have accumulated to justify a new
              major version. Until that point it could just be changed to a comment
              that recommends to add the Z time zone identifier to all datetimes.


              A side note and something to maybe discuss at a future meeting: These
              versioning semantics somewhat freeze StationXML and make it almost
              impossible to incrementally improve it. Here is a blog post illustrating
              an idea of semantic versioning for schemas:
              http://snowplowanalytics.com/blog/2014/05/13/introducing-schemaver-for-semantic-versioning-of-schemas/
              This would allow at least some flexibility but I'm not sure if this is
              desired by the community.


              Is the concern that the times aren't in UTC, or that times are being
              specified without _any_ timezone? When a non-UTC timezone is specified,
              it's at least an explicit time that can easily be converted to UTC.

              The Quakeml spec deals with similar issues by saying times without an
              explicit timezone should be interpreted as UTC (contrary to regular xml
              dateTime). This requires a little parser hoop-jumping but is maybe
              easier than introducing a new major version.


              A bit of both. The main motivation was that it would be a very simple
              and obvious change that makes an implicit assumption about the time zone
              explicit and potentially prevents some hard to discover bugs in software.


              Cheers!

              Lion







              On 26/10/16 18:54, Chad Trabant wrote:

              Hi Lion and WG2,

              In principle I agree with the addition of a UTC time zone indicator.

              The practical issue when such a change is introduced. If you are
              proposing that we only include this at the next incompatible version,
              then it makes sense as is. Alternatively, if we would like to adopt
              this in the next backward-compatible version iteration, then the
              suggestion by Joachim to make the Z optional is the way to go. I
              think we should be prepared to do both. At this point it is unclear
              what kind of next version of StationXML is going to happen or even
              when it is going to happen.

              In addition to the Potential Impact stated in the proposal that
              producers would need a simple modifications and the impacts Joachim
              raised, I add that there are implications for web service clients and
              subsequent readers of StationXML. I think there are a lot more
              clients and readers than implementations/producers, which are, quite
              frankly, the easy part.

              Chad

              On Oct 26, 2016, at 9:32 AM, Lion Krischer
              <lion.krischer<at>gmail.com <lion.krischer<at>gmail.com>> wrote:

              Hi Joachim,


              this is not an actual concern as the proposal would only affect the
              next StationXML version. The current version (1.0) will not change
              and files that are valid against it will naturally always remain
              valid.

              If and once a new StationXML + schema version is introduced all
              StationXML files will have to explicitly state that they adhere to
              the new schema by changing their stationxml namespace. As files
              and implementations have to be updated for this to happen in any
              case, the additional `Z` is only a minor change.

              Until that point the additional `Z` could be considered a
              recommendation and it is of course also valid against the current
              version of the schema.

              In general with data formats I think that it is always a good thing
              to limit choices and possibilities wherever possible.


              Cheers!

              Lion

              On 26/10/16 16:16, Joachim Saul wrote:
              Lion Krischer wrote on 10/26/2016 12:39 PM:
              I propose a small change to be included in the next StationXML
              schema version that would force all datetimes to be explicitly
              marked as being in UTC.

              Technical details can be found here:
              https://github.com/FDSN/StationXML/pull/12

              The proposed change forces all datetimes in a StationXML file
              to end with `Z`. This explicitly marks them as being in UTC
              which (according to the SEED standard) must be given in any
              case.

              Forcing the time to be UTC is a very good idea.

              My concern would be existing web fdsnws implementations as well
              as existing StationXML documents. Requiring a match with ".*Z"
              would render invalid most of what is now perfectly valid
              StationXML. Webservices could probably be updated relatively
              quickly to support a new time format. I am more worried about
              existing XML documents that suddenly would become invalid and
              which parsers might refuse to read. Not all StationXML is created
              by webservices on the fly.

              A pattern that also matches existing time strings *without* time
              zone would be better. This could be something like
              "[0-9-]+T[0-9.:Z]+" rather than just ".*Z". Note that this
              pattern does not enforce a time string ending with "Z" but only
              enforces a limited set of characters, which does not include "+",
              "-" or space after the "T".

              Assuming that time strings containing time zones are currently
              rarely used in StationXML anyway (if at all), the impact of this
              pattern will be minimal.

              The documentation would need a sentence clarifying that time
              strings *shall* (not *must*) end with a "Z".

              Cheers Joachim


              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription
              preferences at http://www.fdsn.org/account/profile/


              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription
              preferences at http://www.fdsn.org/account/profile/


              ---------------------- FDSN Working Group II
              (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center
              (http://www.fdsn.org/message-center/) Update subscription preferences
              at http://www.fdsn.org/account/profile/





              ----------------------
              FDSN Working Group II (http://www.fdsn.org/message-center/topic/fdsn-wg2-data/)

              Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
              Update subscription preferences at http://www.fdsn.org/account/profile/


              • Joachim Saul
                Oct. 27, 2016, 4:27 p.m.
                Hello Lion,

                Lion Krischer wrote on 10/27/2016 12:07 PM:
                Neither my proposed change nor Joachim's less restrictive version would
                be fully backwards compatible changes and thus would require a new major
                version.

                The question is how backwards compatible it really needs to be - not
                just theoretically but in practice.

                If a currently allowed time representation like
                "2009-10-11T12:13:14.1567+02:00" shall remain valid throughout
                StationXML 1.* then you are right and the proposed changes have to be
                postponed until a possible future version 2.

                But do we really expect that StationXML documents using non-UTC time
                strings exist somewhere? Are you aware of any? It will likely turn out
                that no currently existing, relevant software actually produces such
                strings. In that case I would still be in favor of introducing a non-UTC
                blocking pattern into the schema as soon as possible. I would think of
                it as a clarification rather than a new feature.

                For such a minuscule change this is not worth it so I guess this should
                be postponed until enough changes have accumulated to justify a new
                major version.

                When do you expect that new major version to be released?

                Another aspect to keep in mind is that the time string format is also
                relevant for several interfaces incl. all fdsnws's. If the *Z format is
                made mandatory in StationXML it would be inconsistent to not make it
                mandatory in the interfaces, too. I'd therefore prefer to be less
                restrictive and make the trailing "Z" optional in the schema (but
                recommended in the documentation). Instead define the time string format
                as precisely as it was done in the FDSN Web Service Specifications 1.1b
                page 6 *and* enforce this via a corresponding pattern in the xsd.

                Cheers
                Joachim