Loading ...
Sorry, an error occurred while loading the content.

Re: [SimplePie] Re: Prevent this script from updating cache if Twitter is down

Expand Messages
  • Ryan Parman
    It looks at the HTTP header. If Twitter s fail-whale page is sending back an HTTP 200 status, then yes, that is a problem. Twitter *should not* be blindly
    Message 1 of 10 , Dec 9, 2009
    View Source
    • 0 Attachment
      It looks at the HTTP header. If Twitter's fail-whale page is sending back an HTTP 200 status, then yes, that is a problem.

      Twitter *should not* be blindly redirecting traffic to a page with an HTTP 200 status. If they are, however, I'm not sure of the best way to solve the problem:

      Do we:
      1. Maintain a list of sites that don't follow the rules?
      2. Say "too bad, so sad?"

      The first isn't realistically maintainable. The second sucks for developers and end users like you and me.

      Perhaps there can be a way to expose the URL or domain name where we've ended up, so that individual developers can write their own error-handling logic. This seems to be a better, future-compatible way to solve the problem.

      SimplePie is now on GitHub <http://github.com/rmccue/simplepie/>. If anybody reading this message wants to take a crack at implementing this logic, you can fork SimplePie and patch it. You'd probably want to take a look at the SimplePie_File class.

      --
      Ryan Parman
      <http://ryanparman.com>

      Please don't email/IM me directly. That's why we have a mailing list. ;)






      On Dec 9, 2009, at 6:59 AM, mark71441 wrote:

      > But when Twitter is down, the website still responds and takes you to a special page telling you Twitter is over capacity.
      >
      > Is SimpiePie configured to consider that particular page as meaning the service is down?
      >
      >
      > --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@...> wrote:
      >>
      >> SimplePie already does this, and has for years. That's why I'm wondering if it might be another issue.
      >>
      >> If the host is unreachable, SimplePie will keep the existing cache file instead of updating it with empty content.
      >>
      >> If you're sure this is happening, we need to be able to reproduce it to fix it. Can you provide specific step-by-step instructions for replicating the issue?
      >>
      >> --
      >> Ryan Parman
      >> <http://ryanparman.com>
      >>
      >> Please don't email/IM me directly. That's why we have a mailing list. ;)
      >>
      >>
      >>
      >>
      >>
      >>
      >> On Dec 7, 2009, at 7:43 AM, mark71441 wrote:
      >>
      >>> It is a cache issue... the cache is being updated even when the Twitter website is down thus causing SimpiePie to have an error or an empty feed display on the website.
      >>>
      >>> My Twitter feed with SimpiePie works absolutely fine apart from this, I've been using this setup for months but it would be nice to have this fixed. I've read your FAQ and commented out that code a long time ago, it's a different issue than that.
      >>>
      >>> I'm sorry if I'm not being very clear.... but please can you make it so SimpiePie does not try and update the cache when Twitter is down.
      >>>
      >>> --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@> wrote:
      >>>>
      >>>> It's probably not the cache. It's more likely this bug in Twitter's feeds.
      >>>>
      >>>> http://simplepie.org/wiki/faq/problematic_feeds#twitter
      >>>>
      >>>> --
      >>>> Ryan Parman
      >>>> <http://ryanparman.com>
      >>>>
      >>>> Please don't email/IM me directly. That's why we have a mailing list. ;)
      >>>>
      >>>>
      >>>>
      >>>>
      >>>>
      >>>>
      >>>> On Dec 6, 2009, at 5:23 PM, mark71441 wrote:
      >>>>
      >>>>> Hello, I have been using SimpiePie to update the latest Twitters on my home page. It caches fine thanks to tarzan but unfortunately Twitter is often down and the cache will try and still update therefore displaying nothing on my Twitter feed.
      >>>>>
      >>>>> Can someone please tell me the code to add so that SimpiePie/Tarzan do not try and store new data into the cache if Twitter is down?
      >>>>>
      >>>>> My current code:
      >>>>>
      >>>>> <? include("./php/twitter_time.php"); ?>
      >>>>> <?php
      >>>>> error_reporting(E_ALL);
      >>>>> require_once './php/simplepie.inc';
      >>>>> require_once './php/tarzan/_cachecore.class.php';
      >>>>> require_once './php/tarzan/_cachefile.class.php';
      >>>>> $url = 'http://twitter.com/statuses/user_timeline/14123683.rss';
      >>>>> $cache = new CacheFile(sha1($url), './php/cache', 60);
      >>>>> $body = $cache->response_manager('fetch_the_fresh_data', array($url));
      >>>>> function fetch_the_fresh_data($url)
      >>>>> {if ($file = new SimplePie_File($url))
      >>>>> {return preg_replace('/(andy_murray: )/', '', $file->body);}
      >>>>> return null;}
      >>>>> $feed = new SimplePie();
      >>>>> $feed->set_raw_data($body);
      >>>>> $feed->init();
      >>>>> $feed->handle_content_type(); ?>
      >>>>> <? foreach ($feed->get_items(0, 3) as $item): ?>
      >>>>> <div class="andysays">"<?php echo $item->get_title(); ?>"</div>
      >>>>> <span class="twitter_date"><?php echo doRelativeDate( $item->get_date( SIMPLEPIE_RELATIVE_DATE ) );?></span><br />
      >>>>> <hr />
      >>>>> <?php endforeach; ?>
      >>>>>
      >>>>>
      >>>>>
      >>>>>
      >>>>> ------------------------------------
      >>>>>
      >>>>> Yahoo! Groups Links
      >>>>>
      >>>>>
      >>>>>
      >>>>
      >>>
      >>>
      >>>
      >>>
      >>> ------------------------------------
      >>>
      >>> Yahoo! Groups Links
      >>>
      >>>
      >>>
      >>
      >
      >
      >
      >
      > ------------------------------------
      >
      > Yahoo! Groups Links
      >
      >
      >
    • mark71441
      If no one manages to code SimpiePie to work out when these pages are loaded then I would suggest make an exception with Twitter and add a special rule for it.
      Message 2 of 10 , Dec 10, 2009
      View Source
      • 0 Attachment
        If no one manages to code SimpiePie to work out when these pages are loaded then I would suggest make an exception with Twitter and add a special rule for it. It is after all, Twitter.

        I assume there's nothing I can practically do to mod my own SimpiePie while this is worked out? If not, what's the best way for me to keep up to date with developement on this issue?

        --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@...> wrote:
        >
        > It looks at the HTTP header. If Twitter's fail-whale page is sending back an HTTP 200 status, then yes, that is a problem.
        >
        > Twitter *should not* be blindly redirecting traffic to a page with an HTTP 200 status. If they are, however, I'm not sure of the best way to solve the problem:
        >
        > Do we:
        > 1. Maintain a list of sites that don't follow the rules?
        > 2. Say "too bad, so sad?"
        >
        > The first isn't realistically maintainable. The second sucks for developers and end users like you and me.
        >
        > Perhaps there can be a way to expose the URL or domain name where we've ended up, so that individual developers can write their own error-handling logic. This seems to be a better, future-compatible way to solve the problem.
        >
        > SimplePie is now on GitHub <http://github.com/rmccue/simplepie/>. If anybody reading this message wants to take a crack at implementing this logic, you can fork SimplePie and patch it. You'd probably want to take a look at the SimplePie_File class.
        >
        > --
        > Ryan Parman
        > <http://ryanparman.com>
        >
        > Please don't email/IM me directly. That's why we have a mailing list. ;)
        >
        >
        >
        >
        >
        >
        > On Dec 9, 2009, at 6:59 AM, mark71441 wrote:
        >
        > > But when Twitter is down, the website still responds and takes you to a special page telling you Twitter is over capacity.
        > >
        > > Is SimpiePie configured to consider that particular page as meaning the service is down?
        > >
        > >
        > > --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@> wrote:
        > >>
        > >> SimplePie already does this, and has for years. That's why I'm wondering if it might be another issue.
        > >>
        > >> If the host is unreachable, SimplePie will keep the existing cache file instead of updating it with empty content.
        > >>
        > >> If you're sure this is happening, we need to be able to reproduce it to fix it. Can you provide specific step-by-step instructions for replicating the issue?
        > >>
        > >> --
        > >> Ryan Parman
        > >> <http://ryanparman.com>
        > >>
        > >> Please don't email/IM me directly. That's why we have a mailing list. ;)
        > >>
        > >>
        > >>
        > >>
        > >>
        > >>
        > >> On Dec 7, 2009, at 7:43 AM, mark71441 wrote:
        > >>
        > >>> It is a cache issue... the cache is being updated even when the Twitter website is down thus causing SimpiePie to have an error or an empty feed display on the website.
        > >>>
        > >>> My Twitter feed with SimpiePie works absolutely fine apart from this, I've been using this setup for months but it would be nice to have this fixed. I've read your FAQ and commented out that code a long time ago, it's a different issue than that.
        > >>>
        > >>> I'm sorry if I'm not being very clear.... but please can you make it so SimpiePie does not try and update the cache when Twitter is down.
        > >>>
        > >>> --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@> wrote:
        > >>>>
        > >>>> It's probably not the cache. It's more likely this bug in Twitter's feeds.
        > >>>>
        > >>>> http://simplepie.org/wiki/faq/problematic_feeds#twitter
        > >>>>
        > >>>> --
        > >>>> Ryan Parman
        > >>>> <http://ryanparman.com>
        > >>>>
        > >>>> Please don't email/IM me directly. That's why we have a mailing list. ;)
        > >>>>
        > >>>>
        > >>>>
        > >>>>
        > >>>>
        > >>>>
        > >>>> On Dec 6, 2009, at 5:23 PM, mark71441 wrote:
        > >>>>
        > >>>>> Hello, I have been using SimpiePie to update the latest Twitters on my home page. It caches fine thanks to tarzan but unfortunately Twitter is often down and the cache will try and still update therefore displaying nothing on my Twitter feed.
        > >>>>>
        > >>>>> Can someone please tell me the code to add so that SimpiePie/Tarzan do not try and store new data into the cache if Twitter is down?
        > >>>>>
        > >>>>> My current code:
        > >>>>>
        > >>>>> <? include("./php/twitter_time.php"); ?>
        > >>>>> <?php
        > >>>>> error_reporting(E_ALL);
        > >>>>> require_once './php/simplepie.inc';
        > >>>>> require_once './php/tarzan/_cachecore.class.php';
        > >>>>> require_once './php/tarzan/_cachefile.class.php';
        > >>>>> $url = 'http://twitter.com/statuses/user_timeline/14123683.rss';
        > >>>>> $cache = new CacheFile(sha1($url), './php/cache', 60);
        > >>>>> $body = $cache->response_manager('fetch_the_fresh_data', array($url));
        > >>>>> function fetch_the_fresh_data($url)
        > >>>>> {if ($file = new SimplePie_File($url))
        > >>>>> {return preg_replace('/(andy_murray: )/', '', $file->body);}
        > >>>>> return null;}
        > >>>>> $feed = new SimplePie();
        > >>>>> $feed->set_raw_data($body);
        > >>>>> $feed->init();
        > >>>>> $feed->handle_content_type(); ?>
        > >>>>> <? foreach ($feed->get_items(0, 3) as $item): ?>
        > >>>>> <div class="andysays">"<?php echo $item->get_title(); ?>"</div>
        > >>>>> <span class="twitter_date"><?php echo doRelativeDate( $item->get_date( SIMPLEPIE_RELATIVE_DATE ) );?></span><br />
        > >>>>> <hr />
        > >>>>> <?php endforeach; ?>
        > >>>>>
        > >>>>>
        > >>>>>
        > >>>>>
        > >>>>> ------------------------------------
        > >>>>>
        > >>>>> Yahoo! Groups Links
        > >>>>>
        > >>>>>
        > >>>>>
        > >>>>
        > >>>
        > >>>
        > >>>
        > >>>
        > >>> ------------------------------------
        > >>>
        > >>> Yahoo! Groups Links
        > >>>
        > >>>
        > >>>
        > >>
        > >
        > >
        > >
        > >
        > > ------------------------------------
        > >
        > > Yahoo! Groups Links
        > >
        > >
        > >
        >
      • Ryan Parman
        File this as an issue: http://github.com/rmccue/simplepie/issues There s one part-time developer working on SimplePie. Unless we get some development help,
        Message 3 of 10 , Dec 10, 2009
        View Source
        • 0 Attachment
          File this as an issue: http://github.com/rmccue/simplepie/issues

          There's one part-time developer working on SimplePie. Unless we get some development help, this will be a low-priority fix. :(

          --
          Ryan Parman
          <http://ryanparman.com>

          Please don't email/IM me directly. That's why we have a mailing list. ;)






          On Dec 10, 2009, at 4:35 PM, mark71441 wrote:

          > If no one manages to code SimpiePie to work out when these pages are loaded then I would suggest make an exception with Twitter and add a special rule for it. It is after all, Twitter.
          >
          > I assume there's nothing I can practically do to mod my own SimpiePie while this is worked out? If not, what's the best way for me to keep up to date with developement on this issue?
          >
          > --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@...> wrote:
          >>
          >> It looks at the HTTP header. If Twitter's fail-whale page is sending back an HTTP 200 status, then yes, that is a problem.
          >>
          >> Twitter *should not* be blindly redirecting traffic to a page with an HTTP 200 status. If they are, however, I'm not sure of the best way to solve the problem:
          >>
          >> Do we:
          >> 1. Maintain a list of sites that don't follow the rules?
          >> 2. Say "too bad, so sad?"
          >>
          >> The first isn't realistically maintainable. The second sucks for developers and end users like you and me.
          >>
          >> Perhaps there can be a way to expose the URL or domain name where we've ended up, so that individual developers can write their own error-handling logic. This seems to be a better, future-compatible way to solve the problem.
          >>
          >> SimplePie is now on GitHub <http://github.com/rmccue/simplepie/>. If anybody reading this message wants to take a crack at implementing this logic, you can fork SimplePie and patch it. You'd probably want to take a look at the SimplePie_File class.
          >>
          >> --
          >> Ryan Parman
          >> <http://ryanparman.com>
          >>
          >> Please don't email/IM me directly. That's why we have a mailing list. ;)
          >>
          >>
          >>
          >>
          >>
          >>
          >> On Dec 9, 2009, at 6:59 AM, mark71441 wrote:
          >>
          >>> But when Twitter is down, the website still responds and takes you to a special page telling you Twitter is over capacity.
          >>>
          >>> Is SimpiePie configured to consider that particular page as meaning the service is down?
          >>>
          >>>
          >>> --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@> wrote:
          >>>>
          >>>> SimplePie already does this, and has for years. That's why I'm wondering if it might be another issue.
          >>>>
          >>>> If the host is unreachable, SimplePie will keep the existing cache file instead of updating it with empty content.
          >>>>
          >>>> If you're sure this is happening, we need to be able to reproduce it to fix it. Can you provide specific step-by-step instructions for replicating the issue?
          >>>>
          >>>> --
          >>>> Ryan Parman
          >>>> <http://ryanparman.com>
          >>>>
          >>>> Please don't email/IM me directly. That's why we have a mailing list. ;)
          >>>>
          >>>>
          >>>>
          >>>>
          >>>>
          >>>>
          >>>> On Dec 7, 2009, at 7:43 AM, mark71441 wrote:
          >>>>
          >>>>> It is a cache issue... the cache is being updated even when the Twitter website is down thus causing SimpiePie to have an error or an empty feed display on the website.
          >>>>>
          >>>>> My Twitter feed with SimpiePie works absolutely fine apart from this, I've been using this setup for months but it would be nice to have this fixed. I've read your FAQ and commented out that code a long time ago, it's a different issue than that.
          >>>>>
          >>>>> I'm sorry if I'm not being very clear.... but please can you make it so SimpiePie does not try and update the cache when Twitter is down.
          >>>>>
          >>>>> --- In simplepie-support@yahoogroups.com, Ryan Parman <simplepie.support@> wrote:
          >>>>>>
          >>>>>> It's probably not the cache. It's more likely this bug in Twitter's feeds.
          >>>>>>
          >>>>>> http://simplepie.org/wiki/faq/problematic_feeds#twitter
          >>>>>>
          >>>>>> --
          >>>>>> Ryan Parman
          >>>>>> <http://ryanparman.com>
          >>>>>>
          >>>>>> Please don't email/IM me directly. That's why we have a mailing list. ;)
          >>>>>>
          >>>>>>
          >>>>>>
          >>>>>>
          >>>>>>
          >>>>>>
          >>>>>> On Dec 6, 2009, at 5:23 PM, mark71441 wrote:
          >>>>>>
          >>>>>>> Hello, I have been using SimpiePie to update the latest Twitters on my home page. It caches fine thanks to tarzan but unfortunately Twitter is often down and the cache will try and still update therefore displaying nothing on my Twitter feed.
          >>>>>>>
          >>>>>>> Can someone please tell me the code to add so that SimpiePie/Tarzan do not try and store new data into the cache if Twitter is down?
          >>>>>>>
          >>>>>>> My current code:
          >>>>>>>
          >>>>>>> <? include("./php/twitter_time.php"); ?>
          >>>>>>> <?php
          >>>>>>> error_reporting(E_ALL);
          >>>>>>> require_once './php/simplepie.inc';
          >>>>>>> require_once './php/tarzan/_cachecore.class.php';
          >>>>>>> require_once './php/tarzan/_cachefile.class.php';
          >>>>>>> $url = 'http://twitter.com/statuses/user_timeline/14123683.rss';
          >>>>>>> $cache = new CacheFile(sha1($url), './php/cache', 60);
          >>>>>>> $body = $cache->response_manager('fetch_the_fresh_data', array($url));
          >>>>>>> function fetch_the_fresh_data($url)
          >>>>>>> {if ($file = new SimplePie_File($url))
          >>>>>>> {return preg_replace('/(andy_murray: )/', '', $file->body);}
          >>>>>>> return null;}
          >>>>>>> $feed = new SimplePie();
          >>>>>>> $feed->set_raw_data($body);
          >>>>>>> $feed->init();
          >>>>>>> $feed->handle_content_type(); ?>
          >>>>>>> <? foreach ($feed->get_items(0, 3) as $item): ?>
          >>>>>>> <div class="andysays">"<?php echo $item->get_title(); ?>"</div>
          >>>>>>> <span class="twitter_date"><?php echo doRelativeDate( $item->get_date( SIMPLEPIE_RELATIVE_DATE ) );?></span><br />
          >>>>>>> <hr />
          >>>>>>> <?php endforeach; ?>
          >>>>>>>
          >>>>>>>
          >>>>>>>
          >>>>>>>
          >>>>>>> ------------------------------------
          >>>>>>>
          >>>>>>> Yahoo! Groups Links
          >>>>>>>
          >>>>>>>
          >>>>>>>
          >>>>>>
          >>>>>
          >>>>>
          >>>>>
          >>>>>
          >>>>> ------------------------------------
          >>>>>
          >>>>> Yahoo! Groups Links
          >>>>>
          >>>>>
          >>>>>
          >>>>
          >>>
          >>>
          >>>
          >>>
          >>> ------------------------------------
          >>>
          >>> Yahoo! Groups Links
          >>>
          >>>
          >>>
          >>
          >
          >
          >
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          >
        • Ryan McCue
          ... If you submit a patch, or fork and fix it, I ll pull it. I /may/ have the time to do it, but I doubt it (and I don t make promises either). -- Ryan McCue
          Message 4 of 10 , Dec 11, 2009
          View Source
          • 0 Attachment
            Ryan Parman wrote:
            > There's one part-time developer working on SimplePie. Unless we get some development help, this will be a low-priority fix. :(

            If you submit a patch, or fork and fix it, I'll pull it. I /may/ have
            the time to do it, but I doubt it (and I don't make promises either).

            --
            Ryan McCue
            <http://ryanmccue.info/>
          • mark71441
            I ve posted it as an issue: http://github.com/rmccue/simplepie/issuesearch?state=open&q=twitter#issue/47
            Message 5 of 10 , Dec 13, 2009
            View Source
            • 0 Attachment
              I've posted it as an issue: http://github.com/rmccue/simplepie/issuesearch?state=open&q=twitter#issue/47

              --- In simplepie-support@yahoogroups.com, Ryan McCue <ryanmccue@...> wrote:
              >
              > Ryan Parman wrote:
              > > There's one part-time developer working on SimplePie. Unless we get some development help, this will be a low-priority fix. :(
              >
              > If you submit a patch, or fork and fix it, I'll pull it. I /may/ have
              > the time to do it, but I doubt it (and I don't make promises either).
              >
              > --
              > Ryan McCue
              > <http://ryanmccue.info/>
              >
            Your message has been successfully submitted and would be delivered to recipients shortly.