Loading ...
Sorry, an error occurred while loading the content.
 

Low-traffic feeds

Expand Messages
  • Mark Nottingham
    I notice that a few feeds are rejected because, Never had more than 0 headlines per day. This is a shame, IMHO; to me, a big part of the value of RSS is that
    Message 1 of 16 , Nov 1, 2002
      I notice that a few feeds are rejected because,
      "Never had more than 0 headlines per day."

      This is a shame, IMHO; to me, a big part of the value of RSS is that
      you can subscribe to a source that doesn't change much and know that
      you'll see something in your aggregator when it changes, so that you
      don't have to go back and check all of the time.

      I'd go as far to say that RSS delivers *more* value in this case, in
      fact; I'm not able to check 100 occasionally-changing pages every day
      to see if something interesting is happening, but RSS enables this.

      Please reconsider rejecting feeds just because they don't change
      often; this doesn't mean they're low quality.
    • Bill Kearney
      If you can find the feed IDs for them it d help. While you may well be correct that some feeds do not have an abundance of fresh headlines, the polling code
      Message 2 of 16 , Nov 1, 2002
        If you can find the feed IDs for them it'd help. While you may well
        be correct that some feeds do not have an abundance of fresh
        headlines, the polling code does take the time to really check.
        There are many examples of feeds that start up with a burst and then
        never update themselves. These do get caught in "forever polling"
        condition. That's usually until they're noticed as having been long
        since dead and marked as such.

        As for it delivering more value that's questionable. That logic
        would require that reader program understand how to poll efficiently
        and to present such discoveries to you. Until very recently the
        checking for changed timestamps or eTags was entirely ignored in
        readers. And further more a great majority of feeds do not put forth
        the effort to indicate suitable polling frequency info. So the feeds
        end up relentlessly polled, pulling the entire file, even when
        they've obviously not had any fresh content. This is a greater
        shame, one of tremendous wastes of CPU and network resources.

        Feeds are (usually) only marked as rejected because there's something
        wrong with them. Usually it's reserved for test sites or ones known
        to be dead, never ever to return but still leaving content online.
        One could argue that these would serve better as being marked dead
        and that'd be a valid point.

        So dig up some examples of these feeds and let's see what can be done.

        -Bill Kearney


        > I notice that a few feeds are rejected because,
        > "Never had more than 0 headlines per day."
        >
        > This is a shame, IMHO; to me, a big part of the value of RSS is that
        > you can subscribe to a source that doesn't change much and know that
        > you'll see something in your aggregator when it changes, so that you
        > don't have to go back and check all of the time.
        >
        > I'd go as far to say that RSS delivers *more* value in this case, in
        > fact; I'm not able to check 100 occasionally-changing pages every
        day
        > to see if something interesting is happening, but RSS enables this.
        >
        > Please reconsider rejecting feeds just because they don't change
        > often; this doesn't mean they're low quality.
      • Mark Nottingham
        ... Er, that s what computers are for; making it so that humans don t have to go and check (yes, I know that most computer people consider that statement
        Message 3 of 16 , Nov 1, 2002
          --- In syndic8@y..., "Bill Kearney" <wkearney99@h...> wrote:

          > So the feeds
          > end up relentlessly polled, pulling the entire file, even when
          > they've obviously not had any fresh content. This is a greater
          > shame, one of tremendous wastes of CPU and network resources.

          Er, that's what computers are for; making it so that humans don't have
          to go and check (yes, I know that most computer people consider that
          statement unspeakable blasphemy).

          While resources shouldn't be wasted when we know about it, it doesn't
          do any good to over-optimise. I think it's perfectly legitimate for,
          say, a small business to make a feed of press releases available that
          only has five items a year come by. That sort of feed adds tremendous
          value if I want to monitor that company but don't want to go to their
          Web page every day (or week). That's not to say that such a feed
          should be polled every hour, but it adds little if it's polled five
          times a year, either (at least by an aggregator; syndic8 has a
          different purpose).

          > Feeds are (usually) only marked as rejected because there's
          something
          > wrong with them. Usually it's reserved for test sites or ones
          known
          > to be dead, never ever to return but still leaving content online.
          > One could argue that these would serve better as being marked dead
          > and that'd be a valid point.

          That makes sense.


          > So dig up some examples of these feeds and let's see what can be
          done.

          Well, the one that brought it to my attention was my own;
          16976 (also 19848)

          It doesn't have any indication of how often it's updated, so fair
          enough. However, there isn't a direct relationship between update
          interval and optimal polling frequency; polling frequency controls how
          quickly you notice updates. There are some heuristics you can use to
          reduce the waste in these situations; it's basically the same problem
          as heuristic cache freshness.

          It might be nice if there were some guidelines (are there already)?
          Indicating how long it is before a feed is considered dead.

          I'll state a longer update interval, both in the RSS and with the HTTP
          headers, and see how we go.
        • Bill Kearney
          ... Sure, except there s already cases where people running feeds have asked readers to stop polling them so frequently. ... I don t disagree with you.
          Message 4 of 16 , Nov 1, 2002
            > Er, that's what computers are for; making it so that humans don't have
            > to go and check (yes, I know that most computer people consider that
            > statement unspeakable blasphemy).

            Sure, except there's already cases where people running feeds have asked readers
            to stop polling them so frequently.

            > While resources shouldn't be wasted when we know about it, it doesn't
            > do any good to over-optimise. I think it's perfectly legitimate for,
            > say, a small business to make a feed of press releases available that
            > only has five items a year come by. That sort of feed adds tremendous
            > value if I want to monitor that company but don't want to go to their
            > Web page every day (or week). That's not to say that such a feed
            > should be polled every hour, but it adds little if it's polled five
            > times a year, either (at least by an aggregator; syndic8 has a
            > different purpose).

            I don't disagree with you. However, until the reader programs (all of them) get
            their act together and start polling more intelligently it's going to be a
            problem.

            > > One could argue that these would serve better as being marked dead
            > > and that'd be a valid point.
            >
            > That makes sense.
            >
            >
            > > So dig up some examples of these feeds and let's see what can be
            > done.
            >
            > Well, the one that brought it to my attention was my own;
            > 16976 (also 19848)

            16976 is not your feed. As indicated by the data URL. It's a scraping done by
            myRSS. There's already been a discussion with the myRSS developer, Ian Davis,
            about the unpredictable quality of the RSS provided. Ian himself suggested that
            we not list the feeds. Thus their marking as rejected.

            19848 is in fact your feed. The polling code hasn't seen new content in it.
            Put in new content and the polling code will present it to the approval queue.
            A summary-like feed such as this will get 'stuck' in polling if it never adds
            new content. It's possible to manually alter a feed's state. But the polling
            code is pretty good about detecting updated feeds. Make some changes and it
            should get presented for approval.

            > It doesn't have any indication of how often it's updated, so fair
            > enough. However, there isn't a direct relationship between update
            > interval and optimal polling frequency; polling frequency controls how
            > quickly you notice updates. There are some heuristics you can use to
            > reduce the waste in these situations; it's basically the same problem
            > as heuristic cache freshness.

            There are definite problems with RSS and polling frequency. There are any
            number of ways to attack the problem. We're working on several.

            > It might be nice if there were some guidelines (are there already)?
            > Indicating how long it is before a feed is considered dead.

            A feed is only marked dead by manual control, same thing with rejected or any
            other state changes. When an editor sees that a feed has gone dead they can
            manually change it to indicate it. Rejected usually gets applied when the feed
            is bad or the author specifically indicates it's not to be listed.

            > I'll state a longer update interval, both in the RSS and with the HTTP
            > headers, and see how we go.

            The first step to improving things is compliance with specs, such as they are.

            -Bill Kearney
          • Bill Kearney
            Mark, I should also point out that the bookmark feed has a bad site URL. http://www.syndic8.com/feedinfo.php?FeedID=19848 The text encoding is also not quite
            Message 5 of 16 , Nov 1, 2002
              Mark,

              I should also point out that the bookmark feed has a bad site URL.
              http://www.syndic8.com/feedinfo.php?FeedID=19848

              The text encoding is also not quite right. The errors also come up on the web
              page.
              http://www.mnot.net/bookmarks/Travel/Destinations/Europe/Scandanavia/index.html

              49 <item rdf:about="http://www.gotakanal.se/E_index.html">
              50 <title>GÃf¶ta Canal</title>
              51 <link>http://www.gotakanal.se/E_index.html</link>
              52 <dc:date>2002-08-15T04:15</dc:date>


              I don't know what's intended here so I really can't suggest a fix.

              I'm also curious as to what you're doing to 'decide' what gets included in the
              RSS from the bookmarks. It would appear this feed only contains some of the
              travel links. I recognize there's a whole 'can of worms' to be opened here;
              least of which indicating hierarchical structure within feed items. But at the
              very least one would expect the feed to at least closely match the channel's
              HTML page.

              -Bill Kearney

              ----- Original Message -----
              From: "Mark Nottingham" <mnot@...>
              To: <syndic8@yahoogroups.com>
              Sent: Friday, November 01, 2002 11:38 AM
              Subject: [syndic8] Re: Low-traffic feeds


              > Well, the one that brought it to my attention was my own;
              > 16976 (also 19848)
            • James Robertson
              using conditional-get instead of just pulling the feed every time solves this whole problem ... James Robertson
              Message 6 of 16 , Nov 1, 2002
                using conditional-get instead of just pulling the feed every time solves
                this whole problem

                At 04:38 PM 11/1/2002 +0000, you wrote:
                >--- In syndic8@y..., "Bill Kearney" <wkearney99@h...> wrote:
                >
                > > So the feeds
                > > end up relentlessly polled, pulling the entire file, even when
                > > they've obviously not had any fresh content. This is a greater
                > > shame, one of tremendous wastes of CPU and network resources.
                >
                >Er, that's what computers are for; making it so that humans don't have
                >to go and check (yes, I know that most computer people consider that
                >statement unspeakable blasphemy).
                >
                >While resources shouldn't be wasted when we know about it, it doesn't
                >do any good to over-optimise. I think it's perfectly legitimate for,
                >say, a small business to make a feed of press releases available that
                >only has five items a year come by. That sort of feed adds tremendous
                >value if I want to monitor that company but don't want to go to their
                >Web page every day (or week). That's not to say that such a feed
                >should be polled every hour, but it adds little if it's polled five
                >times a year, either (at least by an aggregator; syndic8 has a
                >different purpose).
                >
                > > Feeds are (usually) only marked as rejected because there's
                >something
                > > wrong with them. Usually it's reserved for test sites or ones
                >known
                > > to be dead, never ever to return but still leaving content online.
                > > One could argue that these would serve better as being marked dead
                > > and that'd be a valid point.
                >
                >That makes sense.
                >
                >
                > > So dig up some examples of these feeds and let's see what can be
                >done.
                >
                >Well, the one that brought it to my attention was my own;
                > 16976 (also 19848)
                >
                >It doesn't have any indication of how often it's updated, so fair
                >enough. However, there isn't a direct relationship between update
                >interval and optimal polling frequency; polling frequency controls how
                >quickly you notice updates. There are some heuristics you can use to
                >reduce the waste in these situations; it's basically the same problem
                >as heuristic cache freshness.
                >
                >It might be nice if there were some guidelines (are there already)?
                >Indicating how long it is before a feed is considered dead.
                >
                >I'll state a longer update interval, both in the RSS and with the HTTP
                >headers, and see how we go.
                >
                >
                >Yahoo! Groups Sponsor
                >ADVERTISEMENT
                ><http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706173512:HM/A=1226184/R=0/*http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.11;sz=300x250;adc=ZHS;ord=1036168682?>24dee95.jpg
                >24df0ab.jpg
                >
                >To find more info about Syndicated XML newsfeeds visit
                ><http://www.syndic8.com>http://www.syndic8.com
                >
                >Your use of Yahoo! Groups is subject to the
                ><http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.

                <Talk Small and Carry a Big Class Library>
                James Robertson
                Product Manager, Cincom Smalltalk
                jarober@...

                [Non-text portions of this message have been removed]
              • Bill Kearney
                Quite rightly. Now to just get the various other developers to get their collective acts together on it... One should also note that if the feed itself
                Message 7 of 16 , Nov 1, 2002
                  Quite rightly. Now to just get the various other developers to get their
                  collective acts together on it...

                  One should also note that if the feed itself contains frequency info that it
                  should be honored.

                  ----- Original Message -----
                  From: "James Robertson" <jarober@...>
                  To: <syndic8@yahoogroups.com>
                  Sent: Friday, November 01, 2002 12:05 PM
                  Subject: Re: [syndic8] Re: Low-traffic feeds


                  > using conditional-get instead of just pulling the feed every time solves
                  > this whole problem


                  > > > So the feeds
                  > > > end up relentlessly polled, pulling the entire file, even when
                  > > > they've obviously not had any fresh content. This is a greater
                  > > > shame, one of tremendous wastes of CPU and network resources.
                • James Robertson
                  The 2.1 version of BottomFeeder - http://www.cincomsmalltalk.com/BottomFeeder uses the ttl tag if it s in the feed. Whether it s present or not, all http
                  Message 8 of 16 , Nov 1, 2002
                    The 2.1 version of BottomFeeder - http://www.cincomsmalltalk.com/BottomFeeder

                    uses the ttl tag if it's in the feed. Whether it's present or not, all
                    http access are doen via conditional-get. So BottomFeeder ought to hit
                    feed syndicators a lot more lightly than many of the other tools out there

                    At 12:24 PM 11/1/2002 -0500, you wrote:

                    >Quite rightly. Now to just get the various other developers to get their
                    >collective acts together on it...
                    >
                    >One should also note that if the feed itself contains frequency info that it
                    >should be honored.
                    >
                    >----- Original Message -----
                    >From: "James Robertson" <jarober@...>
                    >To: <syndic8@yahoogroups.com>
                    >Sent: Friday, November 01, 2002 12:05 PM
                    >Subject: Re: [syndic8] Re: Low-traffic feeds
                    >
                    >
                    > > using conditional-get instead of just pulling the feed every time solves
                    > > this whole problem
                    >
                    >
                    > > > > So the feeds
                    > > > > end up relentlessly polled, pulling the entire file, even when
                    > > > > they've obviously not had any fresh content. This is a greater
                    > > > > shame, one of tremendous wastes of CPU and network resources.
                    >
                    >Yahoo! Groups Sponsor
                    >ADVERTISEMENT
                    ><http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706173512:HM/A=1226184/R=0/*http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.11;sz=300x250;adc=ZHS;ord=1036171511?>26626a7.jpg
                    >2662711.jpg
                    >
                    >To find more info about Syndicated XML newsfeeds visit
                    ><http://www.syndic8.com>http://www.syndic8.com
                    >
                    >Your use of Yahoo! Groups is subject to the
                    ><http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.

                    <Talk Small and Carry a Big Class Library>
                    James Robertson
                    Product Manager, Cincom Smalltalk
                    jarober@...

                    [Non-text portions of this message have been removed]
                  • Bill Kearney
                    What about the update interval elements; skipDays, skipHours? ... From: James Robertson To: Sent: Friday,
                    Message 9 of 16 , Nov 1, 2002
                      What about the update interval elements; skipDays, skipHours?

                      ----- Original Message -----
                      From: "James Robertson" <jarober@...>
                      To: <syndic8@yahoogroups.com>
                      Sent: Friday, November 01, 2002 12:31 PM
                      Subject: Re: [syndic8] Re: Low-traffic feeds


                      > The 2.1 version of BottomFeeder - http://www.cincomsmalltalk.com/BottomFeeder
                      >
                      > uses the ttl tag if it's in the feed. Whether it's present or not, all
                      > http access are doen via conditional-get. So BottomFeeder ought to hit
                      > feed syndicators a lot more lightly than many of the other tools out there
                      >
                      > At 12:24 PM 11/1/2002 -0500, you wrote:
                      >
                      > >Quite rightly. Now to just get the various other developers to get their
                      > >collective acts together on it...
                      > >
                      > >One should also note that if the feed itself contains frequency info that it
                      > >should be honored.
                      > >
                      > >----- Original Message -----
                      > >From: "James Robertson" <jarober@...>
                      > >To: <syndic8@yahoogroups.com>
                      > >Sent: Friday, November 01, 2002 12:05 PM
                      > >Subject: Re: [syndic8] Re: Low-traffic feeds
                      > >
                      > >
                      > > > using conditional-get instead of just pulling the feed every time solves
                      > > > this whole problem
                      > >
                      > >
                      > > > > > So the feeds
                      > > > > > end up relentlessly polled, pulling the entire file, even when
                      > > > > > they've obviously not had any fresh content. This is a greater
                      > > > > > shame, one of tremendous wastes of CPU and network resources.
                      > >
                      > >Yahoo! Groups Sponsor
                      > >ADVERTISEMENT
                      >
                      ><http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706173512:
                      HM/A=1226184/R=0/*http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.
                      11;sz=300x250;adc=ZHS;ord=1036171511?>26626a7.jpg
                      > >2662711.jpg
                      > >
                      > >To find more info about Syndicated XML newsfeeds visit
                      > ><http://www.syndic8.com>http://www.syndic8.com
                      > >
                      > >Your use of Yahoo! Groups is subject to the
                      > ><http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.
                      >
                      > <Talk Small and Carry a Big Class Library>
                      > James Robertson
                      > Product Manager, Cincom Smalltalk
                      > jarober@...
                      >
                      > [Non-text portions of this message have been removed]
                      >
                      >
                      > To find more info about Syndicated XML newsfeeds visit http://www.syndic8.com
                      >
                      > Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
                      >
                    • Leandro Mariano Lopez
                      ... I ve developed a PHP class that process conditional gets. If someone is interested I m gladly send you a copy. inkel
                      Message 10 of 16 , Nov 1, 2002
                        >>using conditional-get instead of just pulling the feed every time solves
                        >>this whole problem
                        >

                        I've developed a PHP class that process conditional gets. If someone is
                        interested I'm gladly send you a copy.

                        inkel
                      • James Robertson
                        I think we honor that, but I would have to ask Dave Murphy to be sure. Dave handles most of the feed level code. ...
                        Message 11 of 16 , Nov 1, 2002
                          I think we honor that, but I would have to ask Dave Murphy to be
                          sure. Dave handles most of the feed level code.

                          At 12:34 PM 11/1/2002 -0500, you wrote:
                          >What about the update interval elements; skipDays, skipHours?
                          >
                          >----- Original Message -----
                          >From: "James Robertson" <jarober@...>
                          >To: <syndic8@yahoogroups.com>
                          >Sent: Friday, November 01, 2002 12:31 PM
                          >Subject: Re: [syndic8] Re: Low-traffic feeds
                          >
                          >
                          > > The 2.1 version of BottomFeeder -
                          > <http://www.cincomsmalltalk.com/BottomFeeder>http://www.cincomsmalltalk.com/BottomFeeder
                          > >
                          > > uses the ttl tag if it's in the feed. Whether it's present or not, all
                          > > http access are doen via conditional-get. So BottomFeeder ought to hit
                          > > feed syndicators a lot more lightly than many of the other tools out there
                          > >
                          > > At 12:24 PM 11/1/2002 -0500, you wrote:
                          > >
                          > > >Quite rightly. Now to just get the various other developers to get their
                          > > >collective acts together on it...
                          > > >
                          > > >One should also note that if the feed itself contains frequency info
                          > that it
                          > > >should be honored.
                          > > >
                          > > >----- Original Message -----
                          > > >From: "James Robertson" <jarober@...>
                          > > >To: <syndic8@yahoogroups.com>
                          > > >Sent: Friday, November 01, 2002 12:05 PM
                          > > >Subject: Re: [syndic8] Re: Low-traffic feeds
                          > > >
                          > > >
                          > > > > using conditional-get instead of just pulling the feed every time
                          > solves
                          > > > > this whole problem
                          > > >
                          > > >
                          > > > > > > So the feeds
                          > > > > > > end up relentlessly polled, pulling the entire file, even when
                          > > > > > > they've obviously not had any fresh content. This is a greater
                          > > > > > > shame, one of tremendous wastes of CPU and network resources.
                          > > >
                          > > >Yahoo! Groups Sponsor
                          > > >ADVERTISEMENT
                          > >
                          > ><<http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706
                          > 173512:>http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706173512:
                          >HM/A=1226184/R=0/*<http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.>http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.
                          >11;sz=300x250;adc=ZHS;ord=1036171511?>26626a7.jpg
                          > > >2662711.jpg
                          > > >
                          > > >To find more info about Syndicated XML newsfeeds visit
                          > > ><<http://www.syndic8.com>http://www.syndic8.com>http://www.syndic8.com
                          > > >
                          > > >Your use of Yahoo! Groups is subject to the
                          > > ><<http://docs.yahoo.com/info/terms/>http://docs.yahoo.com/info/terms/>Y
                          > ahoo! Terms of Service.
                          > >
                          > > <Talk Small and Carry a Big Class Library>
                          > > James Robertson
                          > > Product Manager, Cincom Smalltalk
                          > > jarober@...
                          > >
                          > > [Non-text portions of this message have been removed]
                          > >
                          > >
                          > > To find more info about Syndicated XML newsfeeds visit
                          > <http://www.syndic8.com>http://www.syndic8.com
                          > >
                          > > Your use of Yahoo! Groups is subject to
                          > <http://docs.yahoo.com/info/terms/>http://docs.yahoo.com/info/terms/
                          > >
                          >
                          >Yahoo! Groups Sponsor
                          >ADVERTISEMENT
                          ><http://rd.yahoo.com/M=219695.2310151.3725769.1980433/D=egroupweb/S=1706173512:HM/A=1226184/R=0/*http://ad.doubleclick.net/jump/N879.ameritrade.yahoo/B1054521.11;sz=300x250;adc=ZHS;ord=1036172087?>270e5ec.jpg
                          >270edf4.jpg
                          >
                          >To find more info about Syndicated XML newsfeeds visit
                          ><http://www.syndic8.com>http://www.syndic8.com
                          >
                          >Your use of Yahoo! Groups is subject to the
                          ><http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.

                          <Talk Small and Carry a Big Class Library>
                          James Robertson
                          Product Manager, Cincom Smalltalk
                          jarober@...

                          [Non-text portions of this message have been removed]
                        • Mark Nottingham
                          ... Fixed; thanks. ... included in the ... some of the ... opened here; ... But at the ... channel s ... In this case, it s a chronological listing of
                          Message 12 of 16 , Nov 1, 2002
                            > I should also point out that the bookmark feed has a bad site URL.
                            > http://www.syndic8.com/feedinfo.php?FeedID=19848

                            Fixed; thanks.


                            > I'm also curious as to what you're doing to 'decide' what gets
                            included in the
                            > RSS from the bookmarks. It would appear this feed only contains
                            some of the
                            > travel links. I recognize there's a whole 'can of worms' to be
                            opened here;
                            > least of which indicating hierarchical structure within feed items.
                            But at the
                            > very least one would expect the feed to at least closely match the
                            channel's
                            > HTML page.

                            In this case, it's a chronological listing of bookmarks, so they can
                            be tracked as they're added. The top-level feed includes all
                            descendant bookmarks; the feeds for the individual categories are more
                            specialized (i.e., /bookmarks/feed has all bookmarks as they're added;
                            /bookmarks/Web/feed has just Web-related bookmarks, etc.) In other
                            words, the hierarchy isn't in the feed, it's in multiple feeds;
                            subscribe to the one you're interested in. (tho Syndic8 is probably
                            only interested in the top-level feed.)

                            I don't expect a feed to match the channel's HTML page in ordering or
                            presentation, only in topical content.
                          • Bill Kearney
                            ... That s what we re here for, glad to help improve the quality of feeds. ... Hrm, that s an interesting case. You re saying you ve got hierarchical
                            Message 13 of 16 , Nov 1, 2002
                              > Fixed; thanks.

                              That's what we're here for, glad to help improve the quality of feeds.

                              > In this case, it's a chronological listing of bookmarks, so they can
                              > be tracked as they're added. The top-level feed includes all
                              > descendant bookmarks; the feeds for the individual categories are more
                              > specialized (i.e., /bookmarks/feed has all bookmarks as they're added;
                              > /bookmarks/Web/feed has just Web-related bookmarks, etc.) In other
                              > words, the hierarchy isn't in the feed, it's in multiple feeds;
                              > subscribe to the one you're interested in. (tho Syndic8 is probably
                              > only interested in the top-level feed.)

                              Hrm, that's an interesting case. You're saying you've got hierarchical
                              bookmarks but the feed is only delivering them chronologically. That's an
                              interesting blend of the 'news' aspect of RSS with the 'summary' approach. The
                              staggering majority of feeds are 'news-like' with very few being summaries.
                              Here you've got quite the odd beast. I'm not saying it's wrong, no far from it,
                              but it's certainly an odd situation. I doubt many reader programs are going to
                              'build' a picture of your bookmarks from it so the summary aspect is sort of
                              'ruined'.

                              Hmmmm, I have some thoughts about embedding additional metadata that might give
                              it more contextual value.

                              > I don't expect a feed to match the channel's HTML page in ordering or
                              > presentation, only in topical content.

                              Sure. For news-like feeds it's generally a good sign to see them 'come close'
                              to matching up. For your 'running summary' I'm not sure what to say.

                              I'll have to think about this some.

                              -Bill Kearney
                            • Christian
                              Hello,... There are definite problems with RSS and polling frequency. There are any number of ways to attack the problem. We re working on several. ...
                              Message 14 of 16 , Nov 21, 2002
                                Hello,

                                >...
                                >There are definite problems with RSS and polling frequency. There are any
                                >number of ways to attack the problem. We're working on several.
                                >...

                                Is there something available, where I can read about your
                                approaches? I am currently working at a newsfeed-reader and this
                                is the problem which currently bothers me most.

                                I started to synchronize the polls with a file, which is
                                available from weblogs.com (http://www.weblogs.com/changes.xml).
                                I think it is a very good idea, what they do, but many feeds do
                                not ping them and therefore this is not a complete solution.

                                So I am very interested in any idea which could help to solve the
                                frequency problem.

                                Best Regards
                                Christian


                                __________________________________________________________________

                                Gesendet von Yahoo! Mail - http://mail.yahoo.de
                                Yahoo! pr�sentiert als offizieller Sponsor das Fu�ball-Highlight des
                                Jahres: - http://www.FIFAworldcup.com
                              • Bill Kearney
                                Yes, I m working with another developer to address that sort of issue. I m greatly dissatisfied with how Winer et al run the weblogs.com ping service. There s
                                Message 15 of 16 , Nov 21, 2002
                                  Yes, I'm working with another developer to address that sort of issue. I'm
                                  greatly dissatisfied with how Winer et al run the weblogs.com ping service.
                                  There's a behind the scenes amount of censorship that I find greatly
                                  distasteful. So I'm routing around it.

                                  There are several issues to consider. One is accuracy. Another is frequency of
                                  access. Bandwidth consumption comes racing to the forefront pretty quickly.
                                  The fundamental question is how often do you *really* need to know about
                                  changes?

                                  There's a segment of the community that appears to be utterly glued to their
                                  computers 24x7. They're driven by what seems to be some mad hunger for the idea
                                  you need to be informed IMMEDIATELY! I don't subscribe to that opinion but I
                                  see where they're coming from. I just think they're wrong and need to take a
                                  deep breath and step away from the keyboard... But that's just my opinion.

                                  This is why Syndic8 doesn't attempt to deliver up to the minute reports. The
                                  CPU time to generate and the bandwidth to deliver such things would crucify the
                                  already overloaded server. What we have been learning is how often real changes
                                  are detected in the XML. That, in conjunction with ping handling, will really
                                  help offer a clearer picture on how often you "need" to let your reader program
                                  poll for a feed. Tie this into the fact that you can tell Syndic8 what feeds
                                  your read by using a personal list and you've got something...

                                  But we're still in development so it's not like I've anything to offer right
                                  this minute.

                                  -Bill Kearney

                                  ----- Original Message -----
                                  From: "Christian" <wticker@...>
                                  To: <syndic8@yahoogroups.com>
                                  Sent: Thursday, November 21, 2002 8:20 PM
                                  Subject: Re: [syndic8] Re: Low-traffic feeds



                                  Is there something available, where I can read about your
                                  approaches? I am currently working at a newsfeed-reader and this
                                  is the problem which currently bothers me most.

                                  I started to synchronize the polls with a file, which is
                                  available from weblogs.com (http://www.weblogs.com/changes.xml).
                                  I think it is a very good idea, what they do, but many feeds do
                                  not ping them and therefore this is not a complete solution.

                                  So I am very interested in any idea which could help to solve the
                                  frequency problem.
                                • Christian
                                  Hello Bill,Yes, I m working with another developer to address that sort of issue. I m greatly dissatisfied with how Winer et al run the weblogs.com ping
                                  Message 16 of 16 , Nov 21, 2002
                                    Hello Bill,

                                    >Yes, I'm working with another developer to address that sort of issue. I'm
                                    >greatly dissatisfied with how Winer et al run the weblogs.com ping service.
                                    >There's a behind the scenes amount of censorship that I find greatly
                                    >distasteful. So I'm routing around it.

                                    I was not aware of this. I must say I am pretty sad about the 2
                                    hours of hard work, which just fell down to floor ...

                                    >There are several issues to consider. One is accuracy. Another is frequency of
                                    >access. Bandwidth consumption comes racing to the forefront pretty quickly.
                                    >The fundamental question is how often do you *really* need to know about
                                    >changes?
                                    >There's a segment of the community that appears to be utterly glued to their
                                    >computers 24x7. They're driven by what seems to be some mad hunger for the idea
                                    >you need to be informed IMMEDIATELY! I don't subscribe to that opinion but I
                                    >see where they're coming from. I just think they're wrong and need to take a
                                    >deep breath and step away from the keyboard... But that's just my opinion.

                                    My opinion as well. I see it from the administrator point of view
                                    only and would most like to give him the possibility to regulate
                                    the load which hits his server.

                                    Would it not be easy to have an extra tag in the feed-file which
                                    tells the reader the smallest allowed interval? The administrator
                                    could react quickly and easy on too big traffic this way. Well I
                                    assume you had this idea most likely before...

                                    >This is why Syndic8 doesn't attempt to deliver up to the minute reports. The
                                    >CPU time to generate and the bandwidth to deliver such things would crucify the
                                    >already overloaded server. What we have been learning is how often real changes
                                    >are detected in the XML.

                                    I thought about that as well and came to the conclusion that it
                                    will be possible to calculate a good interval for many feeds I
                                    have seen, but there will be always feeds where any calculated
                                    interval will fail badly. For this feeds indeed only a ping will
                                    be a low-bandwidth solution.

                                    >But we're still in development so it's not like I've anything to offer right
                                    >this minute.

                                    Thanks for answering.
                                    Christian

                                    __________________________________________________________________

                                    Gesendet von Yahoo! Mail - http://mail.yahoo.de
                                    Yahoo! pr�sentiert als offizieller Sponsor das Fu�ball-Highlight des
                                    Jahres: - http://www.FIFAworldcup.com
                                  Your message has been successfully submitted and would be delivered to recipients shortly.