Loading ...
Sorry, an error occurred while loading the content.

ftpbench status

Expand Messages
  • Dan Kegel
    This is the first post to the new mailing list, welcome aboard. I notice a subscriber or two whose email address I don t recognize- should we do introductions?
    Message 1 of 11 , Dec 11, 1999
    • 0 Attachment
      This is the first post to the new mailing list, welcome aboard.

      I notice a subscriber or two whose email address I don't recognize-
      should we do introductions?

      http://www.kegel.com/ftpbench/ now includes a link
      to this mailing list, and a Status section with a
      to-do list. Throttling is going to be a bit tricky to add...
      maybe I'll tackle that next.

      I've updated the code to fix a few buglets, to
      print out overall bandwidth once per five seconds,
      and to terminate if the bandwidth goes to zero.
      index.html no longer mentions clftp.cc, as it's out of date.

      If anyone wants to try implementing something interesting, try doing
      ftp_client_proto::put() and ftp_client_pipe::put();
      that'll be handy for when we want to have bench upload
      its own test files to the server.
      - Dan

      --
      (The above is just my personal opinion; I don't speak for my employer,
      except on the occasional talk show.)
    • Dan Kegel
      I ve been playing with ftpbench and betaftpd on and off, and sent betaftpd s author a few bugfixes for his daemon. Haven t had much time to enhance ftpbench
      Message 2 of 11 , Feb 12, 2000
      • 0 Attachment
        I've been playing with ftpbench and betaftpd on and off,
        and sent betaftpd's author a few bugfixes for his daemon.
        Haven't had much time to enhance ftpbench lately; I'd like to
        get up to 1000 users with *some* daemon before I add too many
        more features.

        Anyone else playing with it?
        - Dan
      • Beau Kuiper
        hi, You may need more than one computer acting as the ftp client to pull off 1000 connections to a server (But I wouldn t really know :-) ). You will also need
        Message 3 of 11 , Feb 12, 2000
        • 0 Attachment
          hi,

          You may need more than one computer acting as the ftp client to pull off
          1000 connections to a server (But I wouldn't really know :-) ). You will
          also need a well speced machine to handle 1000 or more connections
          simultaniously. You may find you run into some of linux's scalability problems
          too, like lack of disk threading (only one disk working at a times), and lack
          of network threading Trying the ftp server with a 2.3.X kernel on the beefy
          machine may help.

          Please correct me if I am blatently wrong :-)

          My Imput
          Beau Kuiper
          ekuiperba@...

          On Sun, 13 Feb 2000, you wrote:
          > From: Dan Kegel <dank@...>
          >
          > I've been playing with ftpbench and betaftpd on and off,
          > and sent betaftpd's author a few bugfixes for his daemon.
          > Haven't had much time to enhance ftpbench lately; I'd like to
          > get up to 1000 users with *some* daemon before I add too many
          > more features.
          >
          > Anyone else playing with it?
          > - Dan
          >
          > --------------------------- ONElist Sponsor ----------------------------
          >
          > Get what you deserve with NextCard Visa. Rates as low as 2.9 percent
          > Intro or 9.9 percent Fixed APR, online balance transfers, Rewards
          > Points, no hidden fees, and much more. Get NextCard today and get the
          > credit you deserve. Apply now. Get your NextCard Visa at
          > <a href=" http://clickme.onelist.com/ad/NextcardCreative1CI ">Click Here</a>
          >
          > ------------------------------------------------------------------------
          >
          > Community email addresses:
          > Post message: ftpbench@onelist.com
          > Subscribe: ftpbench-subscribe@onelist.com
          > Unsubscribe: ftpbench-unsubscribe@onelist.com
          > List owner: ftpbench-owner@onelist.com
          >
          > Shortcut URL to this page:
          > http://www.onelist.com/community/ftpbench
          >
          > Code:
          > http://www.kegel.com/dkftpbench/
        • Dan Kegel
          Oh, you re right, no question. I m getting around the disk problem by using a document tree of a single file small enough to fit in RAM, so there are no disk
          Message 4 of 11 , Feb 12, 2000
          • 0 Attachment
            Oh, you're right, no question. I'm getting around the disk problem
            by using a document tree of a single file small enough to fit in RAM,
            so there are no disk accesses.
            I've gotten betaftpd up to 750 clients using dkftpbench, but the server
            daemon has always crapped out one way or another around there, so
            I think I have enough client horsepower. It helps that my client machine
            is five times faster than the server machine.

            I'd love to trade notes with anyone else trying to do similar stuff.
            - Dan

            Beau Kuiper wrote:
            > You may need more than one computer acting as the ftp client to pull off
            > 1000 connections to a server (But I wouldn't really know :-) ). You will
            > also need a well speced machine to handle 1000 or more connections
            > simultaniously. You may find you run into some of linux's scalability problems
            > too, like lack of disk threading (only one disk working at a times), and lack
            > of network threading Trying the ftp server with a 2.3.X kernel on the beefy
            > machine may help.
            >
            > Please correct me if I am blatently wrong :-)
            >
            > My Imput
            > Beau Kuiper
            > ekuiperba@...
            >
            > On Sun, 13 Feb 2000, you wrote:
            > > From: Dan Kegel <dank@...>
            > >
            > > I've been playing with ftpbench and betaftpd on and off,
            > > and sent betaftpd's author a few bugfixes for his daemon.
            > > Haven't had much time to enhance ftpbench lately; I'd like to
            > > get up to 1000 users with *some* daemon before I add too many
            > > more features.
            > >
            > > Anyone else playing with it?
          • Beau Kuiper
            Hi, Under heavy load, strange things occur. System calls that have never failed may fail. Often these are not checked correctly and cause the server to crash.
            Message 5 of 11 , Feb 12, 2000
            • 0 Attachment
              Hi,

              Under heavy load, strange things occur. System calls that have never failed may
              fail. Often these are not checked correctly and cause the server to crash.
              socket() is often a victim of this, as it ussually succeeds under
              non-excessive loads. This may be why betaftpd is crashing (if it is crashing).
              You may also need more RAM if the server is reporting buffer space problems.

              Anyway, have fun.
              Beau Kuiper
              ekuiperba@...

              On Sun, 13 Feb 2000, you wrote:
              > From: Dan Kegel <dank@...>
              >
              > Oh, you're right, no question. I'm getting around the disk problem
              > by using a document tree of a single file small enough to fit in RAM,
              > so there are no disk accesses.
              > I've gotten betaftpd up to 750 clients using dkftpbench, but the server
              > daemon has always crapped out one way or another around there, so
              > I think I have enough client horsepower. It helps that my client machine
              > is five times faster than the server machine.
              >
              > I'd love to trade notes with anyone else trying to do similar stuff.
              > - Dan
              >
              > Beau Kuiper wrote:
              > > You may need more than one computer acting as the ftp client to pull off
              > > 1000 connections to a server (But I wouldn't really know :-) ). You will
              > > also need a well speced machine to handle 1000 or more connections
              > > simultaniously. You may find you run into some of linux's scalability problems
              > > too, like lack of disk threading (only one disk working at a times), and lack
              > > of network threading Trying the ftp server with a 2.3.X kernel on the beefy
              > > machine may help.
              > >
              > > Please correct me if I am blatently wrong :-)
              > >
              > > My Imput
              > > Beau Kuiper
              > > ekuiperba@...
              > >
              > > On Sun, 13 Feb 2000, you wrote:
              > > > From: Dan Kegel <dank@...>
              > > >
              > > > I've been playing with ftpbench and betaftpd on and off,
              > > > and sent betaftpd's author a few bugfixes for his daemon.
              > > > Haven't had much time to enhance ftpbench lately; I'd like to
              > > > get up to 1000 users with *some* daemon before I add too many
              > > > more features.
              > > >
              > > > Anyone else playing with it?
              >
              > --------------------------- ONElist Sponsor ----------------------------
              >
              > Get what you deserve with NextCard Visa. ZERO. Rates as low as 0.0
              > percent Intro APR, online balance transfers, Rewards Points, no hidden
              > fees, and much more. Get NextCard today and get the credit you deserve.
              > Apply now! Get your NextCard Visa at
              > <a href=" http://clickme.onelist.com/ad/NextcardCreative6CI ">Click Here</a>
              >
              > ------------------------------------------------------------------------
              >
              > Community email addresses:
              > Post message: ftpbench@onelist.com
              > Subscribe: ftpbench-subscribe@onelist.com
              > Unsubscribe: ftpbench-unsubscribe@onelist.com
              > List owner: ftpbench-owner@onelist.com
              >
              > Shortcut URL to this page:
              > http://www.onelist.com/community/ftpbench
              >
              > Code:
              > http://www.kegel.com/dkftpbench/
            • Zach Brown
              ... lack of disk threading ? Oh, do tell :) ... this only becomes a problem with lots of cpus and lots of interfaces.. you should be able to fill a gigabit
              Message 6 of 11 , Feb 13, 2000
              • 0 Attachment
                > simultaniously. You may find you run into some of linux's scalability problems
                > too, like lack of disk threading (only one disk working at a times), and lack

                'lack of disk threading'? Oh, do tell :)

                > of network threading Trying the ftp server with a 2.3.X kernel on the beefy
                > machine may help.

                this only becomes a problem with lots of cpus and lots of interfaces.. you should
                be able to fill a gigabit pipe with a nice CPU and ftp server architecture (which
                poll() isn't, as Dan well knows :))

                --
                zach
              • Beau Kuiper
                ... Ops, I was wrong. 2.3.X improves SMP scalability of the disk subsystem. :-) Sorry for this mistake :-) ... Filling a gigabit pipe is not easy by any means,
                Message 7 of 11 , Feb 13, 2000
                • 0 Attachment
                  On Sun, 13 Feb 2000, you wrote:
                  > From: Zach Brown <zab@...>
                  >
                  > > simultaniously. You may find you run into some of linux's scalability problems
                  > > too, like lack of disk threading (only one disk working at a times), and lack
                  >
                  > 'lack of disk threading'? Oh, do tell :)

                  Ops, I was wrong. 2.3.X improves SMP scalability of the disk subsystem. :-)
                  Sorry for this mistake :-)

                  >
                  > > of network threading Trying the ftp server with a 2.3.X kernel on the beefy
                  > > machine may help.
                  >
                  > this only becomes a problem with lots of cpus and lots of interfaces.. you should
                  > be able to fill a gigabit pipe with a nice CPU and ftp server architecture (which
                  > poll() isn't, as Dan well knows :))

                  Filling a gigabit pipe is not easy by any means, especially if you are serving
                  lots of modems at modem rates. Multi-threaded servers suffer due to excessive
                  schedualling while poll based programs suffer due to disk waits (since linux
                  has no kernel support for user mode async-IO). Poll() would actually be good if
                  linux supported async-IO.

                  Sendfile under linux really sucks too :-(. It locks the entire kernel while
                  sending the file, and it may also make the machine unresponsive if you send a
                  large block over the network (esp to a slower 8-bit ethernet card).

                  Anyway, enough ranting
                  Beau Kuiper
                  ekuiperba@...
                • Dan Kegel
                  Yeah, at one point I pointed out to the author that he really did need to check every system call s return value for errors. If there are still crashes going
                  Message 8 of 11 , Feb 13, 2000
                  • 0 Attachment
                    Yeah, at one point I pointed out to the author that he really did need
                    to check every system call's return value for errors.

                    If there are still crashes going on, I suspect it's because of logic
                    errors. He says his server was adapted from an earlier program,
                    and not all the old code has been examined yet.
                    - Dan

                    Beau Kuiper wrote:
                    > Under heavy load, strange things occur. System calls that have never failed may
                    > fail. Often these are not checked correctly and cause the server to crash.
                    > socket() is often a victim of this, as it ussually succeeds under
                    > non-excessive loads. This may be why betaftpd is crashing (if it is crashing).
                    > You may also need more RAM if the server is reporting buffer space problems.
                    >
                    > Anyway, have fun.
                    > Beau Kuiper
                    > ekuiperba@...
                    >
                    > On Sun, 13 Feb 2000, you wrote:
                    > > From: Dan Kegel <dank@...>
                    > >
                    > > Oh, you're right, no question. I'm getting around the disk problem
                    > > by using a document tree of a single file small enough to fit in RAM,
                    > > so there are no disk accesses.
                    > > I've gotten betaftpd up to 750 clients using dkftpbench, but the server
                    > > daemon has always crapped out one way or another around there, so
                    > > I think I have enough client horsepower. It helps that my client machine
                    > > is five times faster than the server machine.
                    > >
                    > > I'd love to trade notes with anyone else trying to do similar stuff.
                    > > - Dan
                    > >
                    > > Beau Kuiper wrote:
                    > > > You may need more than one computer acting as the ftp client to pull off
                    > > > 1000 connections to a server (But I wouldn't really know :-) ). You will
                    > > > also need a well speced machine to handle 1000 or more connections
                    > > > simultaniously. You may find you run into some of linux's scalability problems
                    > > > too, like lack of disk threading (only one disk working at a times), and lack
                    > > > of network threading Trying the ftp server with a 2.3.X kernel on the beefy
                    > > > machine may help.
                    > > >
                    > > > Please correct me if I am blatently wrong :-)
                    > > >
                    > > > My Imput
                    > > > Beau Kuiper
                    > > > ekuiperba@...
                    > > >
                    > > > On Sun, 13 Feb 2000, you wrote:
                    > > > > From: Dan Kegel <dank@...>
                    > > > >
                    > > > > I've been playing with ftpbench and betaftpd on and off,
                    > > > > and sent betaftpd's author a few bugfixes for his daemon.
                    > > > > Haven't had much time to enhance ftpbench lately; I'd like to
                    > > > > get up to 1000 users with *some* daemon before I add too many
                    > > > > more features.
                    > > > >
                    > > > > Anyone else playing with it?
                  • Zach Brown
                    ... It also fixes the scalability in the network subsystem, _far_ more than in the disk. ... yes, tieing connection state to execution state is not the
                    Message 9 of 11 , Feb 13, 2000
                    • 0 Attachment
                      > Ops, I was wrong. 2.3.X improves SMP scalability of the disk subsystem. :-)
                      > Sorry for this mistake :-)

                      It also fixes the scalability in the network subsystem, _far_ more than in
                      the disk.

                      > Filling a gigabit pipe is not easy by any means, especially if you are serving
                      > lots of modems at modem rates. Multi-threaded servers suffer due to excessive
                      > schedualling

                      yes, tieing connection state to execution state is not the greatest idea :)

                      > while poll based programs suffer due to disk waits (since linux

                      no, poll based programs suffer due to the poor scaling of poll() as the
                      number of descriptors does through the roof.

                      disk waiting is another beast _entirely_. Its a problem regardless
                      of the event system you're using. (someone, somewhere has to wait
                      on the io. making kernel threads wait is just as unneceptable as having
                      userland threads wait.. this is the problem that we're working on
                      fixing in sgi's KAIO.. its about half completed. I just need more time, damn
                      it :))

                      > has no kernel support for user mode async-IO). Poll() would actually be good if
                      > linux supported async-IO.

                      no, poll() simply isn't very smart. real time queued signals are a far
                      better way to deal with this sort of thing, as long as you can keep
                      the queue down. (and you can with any modern cpu and a machine reasonably
                      matched with the load (read: memory, interfaces))

                      > Sendfile under linux really sucks too :-(. It locks the entire kernel while
                      > sending the file, and it may also make the machine unresponsive if you send a
                      > large block over the network (esp to a slower 8-bit ethernet card).

                      uhm, no.

                      --
                      zach
                    • Beau Kuiper
                      ... But it makes writing programs easier to be honest :-) And generally safer. ... Well not exactly. The number of descriptors will always be a F(N) to the
                      Message 10 of 11 , Feb 13, 2000
                      • 0 Attachment
                        On Mon, 14 Feb 2000, you wrote:
                        > From: Zach Brown <zab@...>
                        >
                        > > Ops, I was wrong. 2.3.X improves SMP scalability of the disk subsystem. :-)
                        > > Sorry for this mistake :-)
                        >
                        > It also fixes the scalability in the network subsystem, _far_ more than in
                        > the disk.
                        >
                        > > Filling a gigabit pipe is not easy by any means, especially if you are serving
                        > > lots of modems at modem rates. Multi-threaded servers suffer due to excessive
                        > > schedualling
                        >
                        > yes, tieing connection state to execution state is not the greatest idea :)

                        But it makes writing programs easier to be honest :-) And generally safer.

                        >
                        > > while poll based programs suffer due to disk waits (since linux
                        >
                        > no, poll based programs suffer due to the poor scaling of poll() as the
                        > number of descriptors does through the roof.

                        Well not exactly. The number of descriptors will always be a F(N) to the number
                        of connections. But I agree there are better ways to do it, eg the signals
                        mechinism you described below, or the ability to tell the kernel to send data
                        from one socket to another in the background

                        >
                        > disk waiting is another beast _entirely_. Its a problem regardless
                        > of the event system you're using. (someone, somewhere has to wait
                        > on the io. making kernel threads wait is just as unneceptable as having
                        > userland threads wait.. this is the problem that we're working on
                        > fixing in sgi's KAIO.. its about half completed. I just need more time, damn
                        > it :))

                        Sorry, my point wasn't very clear before, I know this is how the kernel works
                        (and why you need to tell the kernel how many knfsd threads to start). When I
                        want to emphisise is that with a poll() based solution, the disk becomes a real
                        bottleneck under heavy load because the single process is ALWAYS waiting on the
                        disk, whereas in a multi-process system, other threads may continue while
                        waiting on the disk.

                        >
                        > > has no kernel support for user mode async-IO). Poll() would actually be good if
                        > > linux supported async-IO.
                        >
                        > no, poll() simply isn't very smart. real time queued signals are a far
                        > better way to deal with this sort of thing, as long as you can keep
                        > the queue down. (and you can with any modern cpu and a machine reasonably
                        > matched with the load (read: memory, interfaces))

                        Yes, poll is very dumb. My idea for scalable static file serving goes like:

                        1) Have a kernel thread (but it needs Async-IO) designed for transfering data
                        socket to socket, including nicities like zero-copy and other optimizations.
                        2) When any sort of server, (FTP, HTTP, SMB, NNTP, SMTP) wishes to send
                        part of, or all of, a file to the network, it notifies the kernel of the fact.
                        3) The specialist kernel thread multiplexes these connections, delivering the
                        data the programs requested.
                        4) When the transfers are finished, or are interrupted, the kernel thread sends
                        a signal to the owner program, notifing them of this. The owner program may
                        also tell the kernel to stop transfering the file immediately.

                        There are propbably a ton of good reasons why "this will never work" :-)

                        >
                        > > Sendfile under linux really sucks too :-(. It locks the entire kernel while
                        > > sending the file, and it may also make the machine unresponsive if you send a
                        > > large block over the network (esp to a slower 8-bit ethernet card).
                        >
                        > uhm, no.

                        Having a deeper look at "The Code" (linux kernel), I will conclude that it was
                        the crappy 8-bit card I was using :-)

                        Beau Kuiper
                        ekuiperba@...
                      • Zach Brown
                        ... this only is meaningful if the amount of data returned by your average poll is significantly related to the amount of data you fed in, which in practice is
                        Message 11 of 11 , Feb 13, 2000
                        • 0 Attachment
                          > Well not exactly. The number of descriptors will always be a F(N) to the number

                          this only is meaningful if the amount of data returned by your average poll
                          is significantly related to the amount of data you fed in, which in practice
                          is very rarely the case. [yes, we've checked :)]

                          > Sorry, my point wasn't very clear before, I know this is how the kernel works
                          > (and why you need to tell the kernel how many knfsd threads to start). When I
                          > want to emphisise is that with a poll() based solution, the disk becomes a real
                          > bottleneck under heavy load because the single process is ALWAYS waiting on the
                          > disk, whereas in a multi-process system, other threads may continue while
                          > waiting on the disk.

                          Yes, its hard to write a poll() using threaded app where the threads
                          all share a descriptor poll and don't take ownership of the descriptors.

                          This is solved very nicely by a shared rt signal queue. (someone sent
                          a patch that mostly gets this right to l-k a few months ago)

                          > There are propbably a ton of good reasons why "this will never work" :-)

                          ?? I have about half the code written for async sendfile() :) And its
                          a weekend of mostly non-trivial hacking to half-assedly attach kiobufs
                          to skbs so that we get zero copy tx.. (we have zero copy tx going in
                          a custom kernel module with csum ranges, but its hardly generic and
                          the netapp guys have been doing it for years. Doing It Right will
                          take longer)

                          > Having a deeper look at "The Code" (linux kernel), I will conclude that it was
                          > the crappy 8-bit card I was using :-)

                          :)

                          you might have missed that going to sleep (being scheduled away)
                          transparently unlocks the kernel lock.

                          --
                          zach
                        Your message has been successfully submitted and would be delivered to recipients shortly.