Loading ...
Sorry, an error occurred while loading the content.

Re: New SGF property: transposition / links

Expand Messages
  • David Bush
    ... Duplication by reflection and/or rotation is certainly important, but this could usually be settled after the first two moves of the game. Maybe a separate
    Message 1 of 12 , Jul 29 7:35 AM
    • 0 Attachment
      > Hello, I support this extension vry much, but I believe that the
      > restriction of the identical position is too restrictive, when I was
      > making my private sgf extension to solve this, I made the
      > Label/Pointer tags but the pointer tag also contained information
      > about transformation (axes, colors), so the dictionary joseki didn't
      > need to repeat any variations with same logic.

      Duplication by reflection and/or rotation is certainly important, but
      this could usually be settled after the first two moves of the game.
      Maybe a separate property should deal with this, which is restricted
      to the beginning of the tree, maybe even in the game info node. You
      might even use an existing property such as ON. Besides naming the
      opening, the value in ON could indicate how the actual game is
      oriented compared to the moves listed in the file. The
      TP property should be allowed throughout the file, subject to two
      restrictions listed below.

      > I've forgotten what the exact SGF terminology is, but could we
      > restrict the positions where this can occur so that the main line of
      > play has no TP in it? That would keep it easy for "simple" programs.
      > Since the full tree needs to appear somewhere, that doesn't seem
      > onerous.

      Since the first variation listed is the main variation, this request
      is covered by my two restrictions:

      1. The target node must be listed before the TP node in the file.

      2. The target node must not be an ancestor of the TP node.

      This implies that the main variation cannot contain a TP property,
      because all the nodes previous to a main node are ancestors of that node.

      Regarding the issue of whether TP functions as a link or as a macro
      call, I suppose that would be up to whoever writes the software that
      uses it. The above restrictions are intended to avoid any endless loop
      of references. That's not very likely in Go, but SGF is for many games.
    • William Shubert
      ... TP is fine with me, but the requirement that it s target N[] appear before the matching TP[] is strange. Why have it? That means that if somebody reorders
      Message 2 of 12 , Jul 29 8:05 AM
      • 0 Attachment
        On Tue, 2008-07-29 at 09:53 +0200, Arno Hollosi wrote:
        TP[ simplestring ] is a node annotation property. It means this node
        represents an identical position by TransPosition of moves to another
        node in the tree. The value is the same as the N property of a target
        node which must be listed earlier in the file, and must not be an
        ancestor of the TP node. This might be useful for an opening library,
        or anywhere that alternate move orders are plausible. A TP node should
        be a leaf node in the file.
        

        TP is fine with me, but the requirement that it's target N[] appear before the matching TP[] is strange. Why have it? That means that if somebody reorders the subtrees in an SGF file, the app will need to go through the subtrees, rearranging the N[]/TP[] groups to make sure that the N comes first. Let's face it, that won't happen, so you'll end up with a lot of invalid files floating around.

        On top of that, I don't see any reason why having the N[] first makes things easier for anybody. It's not like it's easier to search the start of the file than it is to search the whole file. Sure, if you resolve the TP links as you read the file then the N must come first, but just read in the file, then resolve the links, and the problem is solved.
      • David Bush
        ... You might have missed my post from about an hour ago. It might not have reached you before you posted. The purpose of the restrictions is to avoid forward
        Message 3 of 12 , Jul 29 8:47 AM
        • 0 Attachment
          > TP is fine with me, but the requirement that it's target N[] appear
          > before the matching TP[] is strange. Why have it? That means that if
          > somebody reorders the subtrees in an SGF file, the app will need to
          > go through the subtrees, rearranging the N[]/TP[] groups to make
          > sure that the N comes first. Let's face it, that won't happen, so
          > you'll end up with a lot of invalid files floating around.

          You might have missed my post from about an hour ago. It might not
          have reached you before you posted.

          The purpose of the restrictions is to avoid forward references, which
          could result in an endless loop bug, depending on the software which
          is using the file. Software generally reads files from beginning to
          end. At the moment it encounters a forward reference, it won't be able
          to resolve that reference until it reads the target later in the file,
          so it will have to be smart enough to remember where the TP call was,
          and go back and fill in the required information after it reads the
          target. I'm trying to make it easier for programmers to incorporate TP.

          If TP is treated as a macro call, then when the file is initially
          read, a duplicate subtree or "expanded macro" will replace each TP
          call. The user could work with this enlarged tree which will not have
          any TP calls in it. Manipulating the branches should not present any
          problem.

          > On top of that, I don't see any reason why having the N[] first
          > makes things easier for anybody. It's not like it's easier to
          > search the start of the file than it is to search the whole file.
          > Sure, if you resolve the TP links as you read the file then the N
          > must come first, but just read in the file, then resolve the links,
          > and the problem is solved.

          Besides helping programmers, the two restrictions "N first" and "N
          must not be an ancestor of the TP" can create a uniform standard which
          makes it more likely that files can be used in a variety of software,
          which is what SGF is all about.

          One user asked that TP should not appear in the main line. As my
          previous post explains, these restrictions guarantee that.
        • Arno Hollosi
          ... I would certainly add such an option to SGFC. ... I tend to agree with Bill. I think that most of us agree that we should keep the SGF game tree a
          Message 4 of 12 , Jul 29 2:46 PM
          • 0 Attachment
            Stuart asked:
            > Is the plan to extend sgfc to cover this and optionally expand trees?

            I would certainly add such an option to SGFC.

            Bill wrote:
            >> TP is fine with me, but the requirement that it's target N[] appear
            >> before the matching TP[] is strange. Why have it?

            I tend to agree with Bill. I think that most of us agree that we should
            keep the SGF game tree a directed, non-cylic graph. However, as Bill
            pointed out, as soon as someone reorders variations the TP[] and subtree
            denoted by N[] would have to change place as well. That is not exactly
            easy on programmers either. Thus I side with no restriction on N[]
            appearing before TP[] in the file.

            I am undecided about keeping TP[] out of the main branch. It makes some
            sense, as SGF's tree order allows programs to ignore variations and
            still read the main line.

            Michal wrote:
            > when I was making my private sgf extension to solve this, I made the
            > Label/Pointer tags but the pointer tag also contained information
            > about transformation (axes, colors), so the dictionary joseki didn't
            > need to repeat any variations with same logic.

            I have not thought about this before, but I have no objections (but, see
            below.)


            The issue I am currently thinking about is the exact semantics of TP[]
            and its implications. If we think of it as a macro that actually
            copies/expands to the subtree denoted by N[] we "mostly" know how to
            display the result.

            But can we really copy all properties as is? E.g. if transformations are
            allowed (axes, colors) what about comments like "in the upper left
            corner" or annotation properties like GB/GW (good for black/white), just
            to name a few issues.

            Also, how does navigation behave and look like? Is it really identical
            to a file that had its TP[]-macros expanded and replaced? Or does the
            standard have to mention some special issues? (Recall the mess about
            displaying variations as siblings or children.)

            Important issue: what about editing/writing such a file? Suppose I don't
            want to write the expanded tree, but the compressed macro-version. How
            does the user control whether changing a node affects both subtrees
            (N[]'s and TP[]'s) or only one of them (if the display looks exactly the
            same)? How do I tell the SGF program to create a new TP[] or
            remove/split/copy the tree denoted by N[]. I think the GUI issues (and
            therefore the usability of this extension) are not to be underestimated.


            Your opinions?


            Apart from TP[] I really think we should have something like a real
            link, where the semantics mean jumping to another node, not copying its
            subtree. The semantics for links, the navigation, display, etc. are much
            easier for links than for macro-like TP[].

            /Arno
          • Stuart A. Yeates
            ... I m also interested in the exact semantics of TP[] with respect to ko calculations. If two subtrees have different previous board positions (as opposed to
            Message 5 of 12 , Jul 29 3:22 PM
            • 0 Attachment
              On Wed, Jul 30, 2008 at 9:46 AM, Arno Hollosi <ahollosi@...> wrote:

              > The issue I am currently thinking about is the exact semantics of TP[] and
              > its implications. If we think of it as a macro that actually copies/expands
              > to the subtree denoted by N[] we "mostly" know how to display the result.

              I'm also interested in the exact semantics of TP[] with respect to ko
              calculations. If two subtrees have different previous board positions
              (as opposed to the same previous board positions in a different order)
              their children may have different legal moves. This is mainly an issue
              when considering the endgame rather then the opening book, of course.

              > Important issue: what about editing/writing such a file? Suppose I don't
              > want to write the expanded tree, but the compressed macro-version. How does
              > the user control whether changing a node affects both subtrees (N[]'s and
              > TP[]'s) or only one of them (if the display looks exactly the same)? How do
              > I tell the SGF program to create a new TP[] or remove/split/copy the tree
              > denoted by N[]. I think the GUI issues (and therefore the usability of this
              > extension) are not to be underestimated.

              The SGF file format has succeeded by being a file format. Specifying
              the way a program interacts with a user should be left up the program
              author. I hope/imagine that most programs will make the presence or
              absence of N[]'s and TP[]'s invisible to the user.

              cheers
              stuart
            • Arno Hollosi
              ... Sometimes the GUI or interaction with the user has to be addressed in the specification. That s the reason why we have a ST[] property. I am just trying to
              Message 6 of 12 , Jul 30 7:16 AM
              • 0 Attachment
                > The SGF file format has succeeded by being a file format. Specifying
                > the way a program interacts with a user should be left up the program
                > author.

                Sometimes the GUI or interaction with the user has to be addressed in
                the specification. That's the reason why we have a ST[] property. I am
                just trying to find out if someone thinks there may be issues.

                > I hope/imagine that most programs will make the presence or
                > absence of N[]'s and TP[]'s invisible to the user.

                I think that depends. When I am editing I'd like to know whether my
                change is copied multiple times within the SGF tree or is a local
                modification only. Furthermore, when I know that the subtree is copied
                (and maybe transformed) I know that I should avoid comments like "upper
                right corner" or "B5 should be at C17".

                /Arno
              • Lauri Paatero
                Hi, I think idea is good, but there are several problems with initial proposal that need some resolution: 1. N property Property N is inteded to be
                Message 7 of 12 , Jul 30 8:06 AM
                • 0 Attachment
                  Hi,

                  I think idea is good, but there are several problems with initial
                  proposal that need some resolution:

                  1. N property

                  Property N is inteded to be user-visible and editable name for node. So
                  duplicated exists, and preventing user from giving same node name twice
                  in file is bad solution.

                  I think it would be better to define some other property (for example
                  AC[ int ]), that could be used to reference node in all cases it is
                  machine referenced. (I have planned such id for other uses, but I would
                  prefer having single standard one.)

                  Proposal: define new invisible-to-user property AC[nro] for node, and
                  use that to reference to node.

                  2. Loops

                  I would like to say loops are not allowed, bu:

                  If we consider following chain of editing
                  1. create file which contain new properties;
                  2. edit file with editor that does not know new properties (they
                  continue to exists a long time).
                  3. Open file again with editor understanding new properties.

                  Now at step 3 we may have loops, as at step 2 there is no way new rules
                  are followed, even though editor was SGF4-compliant.
                  So I think we should not add new restrictions to what is valid file.
                  Loops will exists, and programs need to be able to deal with them.
                  Also nodes with TP might not be leaf nodes anymore.

                  So in essence new limitations would invalidate SGF4-compliance of
                  program, which is exactly what standards are supposed to prevent.

                  Proposal: No new restrictions to SGF4 file structure.

                  3. Link interpretation

                  I think we should have 2 types of links:
                  - Just jump to location, as user would have moved to location (so TP
                  does not need to be leaf)
                  - Continue with referenced subtree, as proposed here.

                  t.
                  Lauri Paatero


                  Arno Hollosi wrote:
                  >> The SGF file format has succeeded by being a file format. Specifying
                  >> the way a program interacts with a user should be left up the program
                  >> author.
                  >>
                  >
                  > Sometimes the GUI or interaction with the user has to be addressed in
                  > the specification. That's the reason why we have a ST[] property. I am
                  > just trying to find out if someone thinks there may be issues.
                  >
                  >
                  >> I hope/imagine that most programs will make the presence or
                  >> absence of N[]'s and TP[]'s invisible to the user.
                  >>
                  >
                  > I think that depends. When I am editing I'd like to know whether my
                  > change is copied multiple times within the SGF tree or is a local
                  > modification only. Furthermore, when I know that the subtree is copied
                  > (and maybe transformed) I know that I should avoid comments like "upper
                  > right corner" or "B5 should be at C17".
                  >
                  > /Arno
                  >
                  > ------------------------------------
                  >
                  > SGF spec: http://www.red-bean.com/sgf/
                  > Contact: Arno Hollosi <ahollosi@...>Yahoo! Groups Links
                  >
                  >
                  >
                  >
                  >
                • David Bush
                  ... Not only that, but if the target node is pointed to by more than one TP, you would have to make sure N comes before all of them. ... I guess it depends on
                  Message 8 of 12 , Jul 30 8:08 AM
                  • 0 Attachment
                    > Bill wrote:
                    > TP is fine with me, but the requirement that it's target N[] appear
                    > before the matching TP[] is strange. Why have it?
                    >
                    > I tend to agree with Bill. I think that most of us agree that we
                    > should keep the SGF game tree a directed, non-cylic graph. However,
                    > as Bill pointed out, as soon as someone reorders variations the
                    > TP[] and subtree denoted by N[] would have to change place as well.

                    Not only that, but if the target node is pointed to by more than one
                    TP, you would have to make sure N comes before all of them.

                    > That is not exactly easy on programmers either.

                    I guess it depends on what the program does. It wouldn't necessarily
                    be a problem if the program deals with an expanded tree instead of the
                    compressed file.

                    The purpose of TP is to make generation of a file manually with a text
                    editor a little less tedious. If you have software that can save a
                    file in SGF format from a game tree generated with a GUI, ***THERE
                    SHOULD BE NO NEED TO DEAL WITH TP.*** So as far as I can tell, the
                    problem of reordering a file is based on the assumption that this
                    reordering is done manually, with a text editor. Just out of
                    curiosity, what would be gained by doing this?

                    As was pointed out in another post, GUI issues and file loading and
                    saving issues should be regarded as independent of each other.

                    > Michal wrote:
                    > when I was making my private sgf extension to solve this, I made
                    > the Label/Pointer tags but the pointer tag also contained
                    > information about transformation (axes, colors), so the dictionary
                    > joseki didn't need to repeat any variations with same logic.

                    Sorry, I missed your point in my earlier post. But I'm not sure what
                    scope you want this extra information to apply to. Does a joseki
                    dictionary file concern itself entirely with one corner of a 19x19
                    board? Or are you talking about referring to another corner of the
                    board which was explored earlier in the same file? If the full-board
                    positions are not effectively identical by axis/color transformation,
                    there could be a problem when software tries to evaluate this
                    position. Joseki battles are of course not independent of each other.

                    Also, do you generate this dictionary with a GUI or with a text editor?

                    > ... If we think of it as a macro that actually
                    > copies/expands to the subtree denoted by N[] we "mostly" know how
                    > to display the result.
                    >
                    > But can we really copy all properties as is? E.g. if
                    > transformations are allowed (axes, colors) what about comments like
                    > "in the upper left corner" or annotation properties like GB/GW
                    > (good for black/white), just to name a few issues.

                    Whether TP is a macro call or a link, these same issues would crop up.
                    GB versus GW could be automatically taken care of. You could just
                    ignore all comments in an copied subtree, or not copy them in the
                    first place if there is any kind of axis/color change.

                    > Important issue: what about editing/writing such a file? Suppose I
                    > don't want to write the expanded tree, but the compressed macro-
                    > version. How does the user control whether changing a node affects
                    > both subtrees (N[]'s and TP[]'s) or only one of them (if the
                    > display looks exactly the same)?

                    Under what circumstances would you want to change one subtree but not
                    the other? I'm not asking about axis/color changes here. Why should
                    the user have this option to make the subtrees of the compressed file
                    different?

                    > How do I tell the SGF program to create a new TP[] or
                    > remove/split/copy the tree denoted by N[]. I think the GUI issues
                    > (and therefore the usability of this extension) are not to be
                    > underestimated.

                    Is this SGF program a text editor or a GUI? If it's a GUI which can
                    save files in SGF format, why would you want to deal with TP calls?

                    > The semantics for links, the navigation, display, etc. are much
                    > easier for links than for macro-like TP[].

                    I don't mean to require programmers to treat TP one way or the other.
                    The SGF definition should work for any way the programmer wants to use
                    it. I just used the "macro" analogy as a way of explaining what I was
                    talking about. Sorry if I wasn't clear about that.

                    syeates said:
                    > I'm also interested in the exact semantics of TP[] with respect to
                    > ko calculations. If two subtrees have different previous board
                    > positions (as opposed to the same previous board positions in a
                    > different order) their children may have different legal moves.
                    > This is mainly an issue when considering the endgame rather then
                    > the opening book, of course.

                    My proposal is that the positions need not be identical as long as
                    every move, position evaluation, and everything else that gets copied
                    or linked to is equally valid in both situations. If there are
                    "irreconcilable differences" then they should be different subtrees.
                    In a ko situation, and perhaps for the most part in Go, different
                    battles are arguably not truly independent of each other, because one
                    usually is more important than the others at any given point. So TP
                    calls might not be appropriate if the subtree in question is large.

                    Also I recommend you take a look at my LO[] proposal (in a separate
                    thread) for dealing with LOcal battles.
                  • Mark Lentczner
                    The big issue I see to resolve is the semantic of the link. I see several possibilities discussed in the thread, and I m going to try to restate the
                    Message 9 of 12 , Jul 30 9:18 PM
                    • 0 Attachment
                      The big issue I see to resolve is the semantic of the link. I see
                      several possibilities discussed in the thread, and I'm going to try to
                      restate the concisely:

                      1) Identity
                      The node referenced is the identical position as the node with the
                      link. Presumably they have the same position *after* each has applied
                      any moves.

                      2) Duplicate Sub-tree
                      The nodes that follow the referenced node are equally applicable to
                      the position at this node.

                      3) Jump
                      The referenced node should be offered to the user as a possible follow
                      on to this node. The position at the referenced node could be wholly
                      different.

                      Identity seems to have a number difficulties in definition and
                      resiliency:
                      - just how "identical" do the positions have to be?
                      - i.e.: same number of prisoners? or just same delta? passes?
                      - same comments? markup?
                      - what about differing in only a tenuki move?

                      Duplicate Sub-tree has the downside that now each node can no longer
                      be transformed into a single game position, since the state of the
                      game will depend on if you took a jump to get there.

                      Jump gets around these difficulties, but at the expense of being a
                      different kind of navigation that the existing options.

                      One way to approach the choice is to look at the use cases and see
                      which admits the most flexibility. The use cases I see are:

                      A) Joseki dictionary desire to not replicate whole sub-trees when a
                      position can be arrived at via two different move sequences. This use
                      case could be broadened to include color and spacial transposition, or
                      differing in tenuki.

                      B) Tsumego. Like joseki, a desire to not replicate whole sub-trees.
                      Here the game state identity requirement might be more rigorous.
                      GoProblems.com already does this automatically by identifying
                      identical positions in the tree: If one has follow on nodes and the
                      other doesn't -- they are "spliced".

                      C) General annotation need to have variant lines or main lines
                      reference other lines within the tree.

                      It seems to me that the Jump semantic supports all of these. If there
                      is a need for identical position at both ends, it is easy enough for
                      the creation tool to enforce that.

                      The only thing that then needs to be addressed is the intended user
                      navigation: Is it displayed as an option? Is it followed
                      automatically? One could support both by this rule: If the LI[]/TP[]
                      property appears as the only property in a node, then the link should
                      be immediately followed, replacing the game state with the state at
                      the target node. On the other hand, if there are any other
                      properties, including just a comment (even an empty comment!), then
                      the user is offered link as a sort of 'remote variation', and can
                      choose, as they would choose a variant, to continue in that part of
                      the tree.

                      - Mark
                    Your message has been successfully submitted and would be delivered to recipients shortly.