Re: ** SGF FF **
- Arno Hollosi wrote:
> the results of the CFV are: 7 votes for FF, 0 votes against.
> Therefore we start discussing FF. I'll post an announcement to
> rec.games.go as well. The following issues should be discussed
> (suggestions by voters):
It's exciting to start SGF discussion.
How about that we define SGF's preceding target
"Implementing the ability to record the whole of and
the all kind of, as far as known, Go game" ?
In another saying, a SGF will be a perfect document
of a played Go game.
Then our point of view will be more clear, I guess.
I'm new to this forum, and newcomers are supposed to be circumspect. But
in the hope that I can represent at least one section of ordinary users,
may I make the following comments.
1. Even though I favour standardisation, I am dismayed by what I have
seen so far. There seems to be a trend to make the spec fit the needs of
the big fish - the databases, IGS, or Nihon Ki-in. Ordinary users seem
to have different needs. Arno tells me that most SGF files he gets from
other people are incorrect. That tells me that most users want to
prepare their files in ways that the spec won't let them. Human nature
demands that the spec should change to suit them, rather than users be
dragooned into complying with an increasingly complex spec. Is the fact
that most features in FF4 are unused, and the fact that FF4 is going to
have a short life, not a sign that the standardisation is moving too
2. I am inclined to believe that, rather than make the spec cleverer and
cleverer, the database or other software should be made cleverer. For
example, splitting EV and RO might suit the needs of current software,
but I don't see why the search engine could not be programmed to scan a
single line of free text for whatever the database regards as key words.
3. While I am not up to Jan's level, with Mark Hall I have a database of
about 6,000 games. Most are in .GO format but I am rapidly converting to
FF3 (I rejected FF4 for reasons that may become apparent below.) I come
across so many things that the spec either doesn't allow or consider. In
general it seems to me that the heavy emphasis by some people on modern
games leads to overlooking many important things. For example, the old
(or not so old - it is still used in the Oteai) handicap system (B-B-W
etc) is VERY important because it affects the result of the game,
relative standing of the players, etc) yet there is nowhere to put it (I
was told to put it in Game Comment because HA won't allow text - I think
that is absurd).
4. I also find the DT date a major problem. Apart from the simple date,
there are lunar dates. I believe in the ISO standard, so I convert such
dates, but most people can't (or, worse, do what Invincible does and
pretend they are Gregorian). Often old dates are given as era years
(e.g. Kaei 1) but conversion would often require two western years
because of the lunar/solar overlap. How does one deal with the (rather
common) types exemplified by: "Early Qing era" or "Spring 1934" or "1894
or 1895" or "Published 1985-01-01~02-03" or or "Played before he became
a pro" or "Played 1845-03-03 and through the night till 9pm the
following day" [nb putting 1845-03-03,04 would omit the interesting
point that they played non-stop].
5. I find EV and RO a problem in various ways. First the split seems
artificial and secondly EV can be defined in different ways by different
people. I prefer both "Meijin Final" and "Meijin League" (and also
simply "Meijin") to be in EV, partly because these are seen as
significant separate items in biographies of players (e.g. the number of
times one has won a Final, or appeared in one, or appeared in a League,
are listed as discrete achievements).
6. An even bigger problem with EV is that tournament names have yet to
be standardised. At the risk of seeming rude, I would say that much of
the energy spent on rather esoteric parts of the spec could more
fruitfully be spent here. (A couple of examples: is it Kisung, Kiseong,
Kisong, Gisung, Korean Kisei, Kisei or what? Is it Women's Honinbo or
Ladies' Honinbo (or Ladies Honinbo, which may throw some search
engines]? Is it just Meijin or Old Meijin and New Meijin? Is it Hayago
Senshuken, Hayago Championship, Lightning/Speed/Fast Go Championship?
7. Players' names are a major problem, especially Korean. For many there
are 5 or 6 current variants. I can claim to have offered a standard for
these in my Names Dictionary (now out of print!) as virtually every name
that can appear in a significant game record is there, but a strategy
for making anyone follow the "standard" is beyond me.
8. I find RE a problem. Many old games are left unfinished. They are not
void. The reason they were left unfinished may be interesting (one
player was losing so broke off, it was a ceremonial game, it was just
meant to try a fuseki, etc etc). I have also recently encountered a
"both players lost" game - is that covered? To me, it is also logical to
include in the RE rubric information such as "No further moves known" or
"White connects the ko" [it is wrong to play these moves out as part of
the record - historical research often hinges on niceties such as these:
e.g. one record says W connects the ko, another says B.
9. TM: Don't forget that players may be given different times (as in
some pro-am games)
10. I have more points but I would be overegging the pudding to go on.
The conclusions I draw from my own experience are (1) that FF5 would be
an advance if it actually went backwards a little bit - in particular it
should allow free-form text where possible; (2) most urgent effort is
needed to standardise tournament names and player names; (3) the needs
of ordinary users - people who enter game records, not programmers or
anyone else with too much of a vested interest - should be given greater
weight. (It would be of interest to know what the commonest "mistakes"
of ordinary users are.)
- From: David Fotland <fotland@...>
> I'd like to make a very strong plea for backwards compatibility.I agree and favor compatibility as well. The question is if we can
> SGF was conceived from the start to be extensible in a way that
> new files can still be read by old software, and old files read
> by new software.
abandon some seldom used features in favor of better defined entities.
> Can we put a time limit on FF?I'd say a reasonable time-limit would be discussing proposals during the
next 6-8 weeks, and another 4-6 weeks to get the details fixed and the
spec written. FF could be ready by May.
From: Dave Dyer <ddyer@...>
> The core "shell" of SGF is a specification that will allow toolsI agree that the new spec shouldn't be game-specific. As far as I can see,
> to be written to read, write and manipulate collections of games
> in SGF format, with no knowlege at all about what game type is contained.
most changes will happen to the game-info properties, with some slight
modifications elsewhere. Plus restructuring of the spec according to the
property set concept. As most of as use SGF only for Go, I'd appreciate
if you'd point out things which are too Go specific.
- John Fairbairn <JF@...> wrote:
> There seems to be a trend to make the spec fit the needs ofCould you describe the needs of an ordinary user?
> the big fish - the databases, IGS, or Nihon Ki-in. Ordinary users seem
> to have different needs.
While creating FF I posted to rec.games.go that I like suggestions by
users about their needs. IIRC, I got about 5 replies, ALL of which were
about program issues and not file format issues. To me it seems that
there are only few programs which are truly user-friendly.
> Arno tells me that most SGF files he gets from other people are incorrect.Apart from the usual DT[Jan 4th 1991] or RE[Black wins] most errors are
made by the programs. Most of the time when using variations. Is
reading/writing variations really so tough? The range of errors that
I come across never ceases to amaze me.
Standard call: Use SGFC to check the output of your application !!!
> That tells me that most users want to prepare their files inI argue that it's because of sloppy programs and not because of SGF.
> ways that the spec won't let them.
> Human nature demands that the spec should change to suit them,I agree. But the spec itself should (almsot) be transparent to the user.
> rather than users be dragooned into complying with an increasingly
> complex spec.
But this is the task of applications. Of course the spec should make
it easy to create SGF compliant applications.
Coming back to the "big fishes": this is why I think "property sets"
are a good idea. "Little users" use the basic sets which suit there
needs. And the "big fishes" use extended sets to store all the
information they want to store.
> but I don't see why the search engine could not be programmed to scan aThis is true. However, if you want to have interchangeable SGF games
> single line of free text for whatever the database regards as key words.
you would need to specify those key words. Otherwise everyone would
have to write a conversion program for their database. And if we can
agree on key words, why not make them properties, such as EV & RO?
> ... the old handicap system (B-B-W) ...These issues will be addressed in FF.
> ... DT date a major problem ...
> ... EV and RO a problem in various ways ...
> ... find RE a problem ...
> ... TM: Don't forget that players may be given different times
Also, see Jan's suggestions for team players and other information.
> "Played 1845-03-03 and through the night till 9pm theI think that this information should be put into GC and that DT
> following day" [nb putting 1845-03-03,04 would omit the interesting
> point that they played non-stop].
should contain 1845-03-03,04.
> (1) that FF5 would be an advance if it actually went backwardsYes. Many of us agree on this.
> a little bit - in particular it should allow free-form text where possible
> (3) the needs of ordinary users - people who enter game records,I agree. As pointed out above, most users are concerned about the
> not programmers or anyone else with too much of a vested interest
program interface and the functions provided.
> (It would be of interest to know what the commonest "mistakes"Of users: formatting of game-info properties (as most programs don't
> of ordinary users are.)
provide a specific interface that does the formatting for them).
The most errornous property is TM (if entered manually - not on
sever games for apparent reasons), closly followed by DT and RE.
Other "errors" are e.g. making sibling variations in an application
which displays variations as childs. (Done by removing move with AE and
making new move within one node - illegal since FF).
Of programs: variations in all forms: empty variations "()", Ishi style
variations (sgf2misc), completely trashed variations (some versions of
WinMGT), basically everything you can make wrong with '(' and ')'.
Other errors include multiple properties (e.g. two C in one node),
wrong values (e.g. LB[1:bc] instead of LB[bc:1], or LB[bc:]), splitted
property ID's (e.g. W (linebreak) L - during mailing/posting --
which @#$% mail clients split words ??????).
AGAIN: Standard call: Use SGFC to check the output of your application !!!
> (2) most urgent effort isTrue. But this is beyond the scope of the spec, though we could recommend
> needed to standardise tournament names and player names;
using your dictionary. I can't think of a way to enforce this. If the
"big fish" adopt that recommendation then step by step those names will
be accepted. E.g. it's my experience that Jan's spelling is regarded
as a de-facto standard by many users.