Loading ...
Sorry, an error occurred while loading the content.

Introducing DTrace

Expand Messages
  • Alan DuBoff
    This message was posted on comp.unix.solaris several months ago, but DTrace is one of those tools that has not been talked about too much and it s one of the
    Message 1 of 12 , Feb 29, 2004
    • 0 Attachment
      This message was posted on comp.unix.solaris several months ago, but DTrace
      is one of those tools that has not been talked about too much and it's one of
      the very powerful tools in Solaris Express that developers should really take
      time to look at.

      As such, I'm posting Bryan Cantrill's message here so that others can learn
      and possibly be motivated to use DTrace. DTrace is one of the tools that
      truely seperates Solaris from other Optering Environments, IMO, giving the
      developer incredible power at thier fingertips.

      DTrace was introduced in Solaris Express, back at build 43 as I recall, and
      for those looking to play with it and learn more, grab the most recent
      Solaris Express, build 51.

      http://wwws.sun.com/software/solaris/solaris-express/get.html

      There's a very complete document on BigAdmin in PDF format, which is more
      than 300 pages, available here:

      http://www.sun.com/bigadmin/content/dtrace/d10_latest.pdf

      --

      Alan DuBoff
      Software Orchestration, Inc.
      GPG: 1024D/B7A9EBEE 5E00 57CD 5336 5E0B 288B 4126 0D49 0D99 B7A9 EBEE


      ---------------- Prior Message Posted by Bryan Cantrill -----------------

      From: Bryan Cantrill (bmc@...)
      Subject: Introducing DTrace
      View: Complete Thread (2 articles)
      Original Format
      Newsgroups: comp.unix.solaris
      Date: 2003-11-14 22:02:13 PST

      All,

      The DTrace cat has been pawing its way out of the metaphorical bag for quite
      some time now, with Gavin Maltby posting some especially enticing examples
      here on comp.unix.solaris. Now that Solaris Express 11/03 is publicly
      available for download, we on the DTrace engineering team wanted to take a
      moment to post some slightly more elaborate examples of this new technology.

      Before we get started, note that there is quite a bit of publicly available
      documentation on DTrace at BigAdmin:

      http://www.sun.com/bigadmin/content/dtrace

      Also, note that this post isn't meant to be a comprehensive tutorial -- it's
      really more of a quick demonstration of how DTrace might be used to solve
      problems that you may come across in your day-to-day workings with Solaris.

      On with the show: let's say you're looking at mpstat(1) output on your
      multiuser server. You might see something like this:

      CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
      12 1 27 3504 338 206 765 27 114 65 1 337 9 19 41 31
      13 1 19 5725 98 68 723 22 108 120 0 692 3 17 20 61
      14 0 57 3873 224 192 670 22 75 86 1 805 7 10 35 48
      15 36 7 1551 42 28 689 6 68 59 0 132 2 9 35 54
      16 14 7 7209 504 457 1031 37 125 244 0 459 6 30 4 60
      17 5 5 4960 150 108 817 37 98 154 0 375 6 26 6 62
      18 5 6 6085 1687 1661 741 60 76 248 0 434 3 33 0 64
      19 0 15 10037 72 41 876 23 100 291 1 454 2 19 9 71
      20 12 5 5890 746 711 992 32 122 216 2 960 10 33 4 53
      21 60 5 1567 729 713 467 15 80 59 0 376 2 35 10 53
      22 0 6 4378 315 291 751 17 84 142 1 312 3 16 1 80
      23 0 6 12119 33 3 874 20 82 384 1 513 4 24 11 62

      And well you may wonder (as perhaps you often have) -- what the hell is
      causing all of those cross calls, anyway? (Cross calls appear in the "xcal"
      column; see mpstat(1).)

      Using DTrace, investigating this is a snap:

      # dtrace -n xcalls'{@[execname] = count()}'
      dtrace: description 'xcalls' matched 4 probes
      [ letting this run for a few seconds ]
      ^C

      mozilla-bin 1
      lockd 1
      in.mpathd 2
      nsrmmd 5
      grep 6
      chmod 6
      cat 6
      nwadmin 13
      ls 24
      in.tftpd 28
      nsrindexd 34
      fsflush 38
      cut 42
      find 42
      mkdir 66
      rm 76
      ipop3d 78
      scp 79
      inetd 96
      dtrace 111
      nawk 118
      imapd-simmonmt 126
      rmdir 132
      sshd 138
      rpc.rstatd 159
      mv 398
      save 1292
      gzip 1315
      get_all2 1678
      sched 1712
      nfsd 3709
      tar 27054

      So the bulk of these are coming from tar(1) processes. To explore the cross
      calls coming from tar specifically, we could add a predicate to the above:

      # dtrace -n xcalls'/execname == "tar"/{@[pid] = count()}'
      dtrace: description 'xcalls' matched 4 probes
      [ again letting this run for a few seconds ]
      ^C

      938339 507
      938415 784
      938229 2438
      938390 3282
      938363 6464
      937067 13984

      But now we notice a strange thing: none of these processes seems to be
      around by the time we're back on the shell, e.g.:

      # pflags 937067
      pflags: cannot examine 937067: no such process or core file

      So is someone launching a bunch of short-lived tar processes? To explore
      this as a rough first cut, we could just print the pid whenever a tar
      process is launched:

      # dtrace -n syscall::exec\*:return'/execname == "tar"/{trace(pid)}'
      CPU ID FUNCTION:NAME
      18 112 exece:return 954538
      21 112 exece:return 954519
      17 112 exece:return 954584
      21 112 exece:return 954579
      21 112 exece:return 954607
      22 112 exece:return 954632
      21 112 exece:return 954658
      20 112 exece:return 954684
      20 112 exece:return 954705
      15 112 exece:return 954727
      14 112 exece:return 954763
      16 112 exece:return 954775
      12 112 exece:return 954798
      18 112 exece:return 954818
      22 112 exece:return 954873
      22 112 exece:return 954889
      23 112 exece:return 954893
      14 112 exece:return 954916
      14 112 exece:return 954971
      ...

      Seeing that this is happening with some frequency (the above has output
      every second or so), we want to explore more generally what is being
      exec(2)'d on the machine:

      # dtrace -n syscall::exec\*:entry'{@[copyinstr(arg0)] = count()}'
      [ letting this run for a bit ]
      ^C

      /usr/lib/mail.local 1
      /usr/bin/true 1
      /usr/bin/pgrep 1
      /usr/bin/wc 1
      /usr/bin/touch 1
      /usr/bin/sleep 1
      /usr/sbin/modinfo 4
      /bin/awk 4
      /bin/sh 4
      /usr/bin/chmod 14
      /usr/bin/cat 14
      /usr/bin/tar 14
      ./do1 14
      /usr/bin/cut 15
      /usr/bin/grep 15
      /usr/bin/find 15
      /usr/bin/ls 15
      /usr/bin/mv 15
      /usr/bin/rm 16
      /usr/bin/mkdir 17
      /usr/bin/gzip 29
      /usr/bin/nawk 44
      /usr/bin/rmdir 48
      /home/ellenp/rmdir 48
      /home1/netscape/rmdir 48
      /usr/dt/bin/rmdir 96
      /usr/openwin/bin/rmdir 96

      There's a bunch that's interesting here -- but for now we're interested
      in those tar processes. In particular, when someone is exec'ing a tar
      process, we want to know who exactly they are. To do this, we write
      the following DTrace script, and call it "whotar.d":

      syscall::exec*:entry
      /copyinstr(arg0) == "/usr/bin/tar"/
      {
      @[execname] = count();
      }

      Running this points us straight to the culprit:

      # dtrace -s whotar.d
      [ again letting this run for a bit ]
      ^C

      get_all2 22

      So get_all2 is the problem. We can now use traditional tools to hone in
      on this:

      # ptree `pgrep get_all2`
      100578 /usr/sbin/inetd -s
      900870 in.rlogind
      900872 -csh
      901382 /bin/sh get_all3
      901383 /bin/sh ./get_all2 80/80acc080 90/00fccc90
      959045 /bin/sh ./get_all2 80/80acc080 90/00fccc90
      959046 tar xvf -
      explorer.80c23d8e.ptgw02is-2003.10.27.09.35/README
      959047 gzip -dc /net/pamer.central/proactive/rawdata/8e/80c23d8e
      901384 /bin/sh ./get_all2 90/80bdbb90 a0/1274eda0
      959565 /bin/sh ./get_all2 90/80bdbb90 a0/1274eda0
      959566 tar xvf - explorer.8089ed9d.curly-2002.05.06.10.00/README
      exp
      959567 gzip -dc /net/pamer.central/proactive/rawdata/9d/8089ed9d
      901385 /bin/sh ./get_all2 a0/80aabda0 b0/1d6c05b0
      959619 mv disks/diskinfo disks/format.out disks/mount.out
      etc/release
      901386 /bin/sh ./get_all2 b0/80b264b0 c0/335419c0
      959612 /bin/sh ./get_all2 b0/80b264b0 c0/335419c0
      959613 tar xvf - explorer.82a86abd.seq22-2002.05.31.21.49/README
      exp
      959614 gzip -dc /net/pamer.central/proactive/rawdata/bd/82a86abd

      So that explains where the tar processes (and thus the cross calls) are coming
      from! And just how short-lived _are_ these tar processes? We can answer this
      question with DTrace, natch. To do this, we'll use an associative array and
      the "timestamp" variable. We are going to linearly quantize the result from 0
      seconds to 60 seconds. We'll also use a "tick-1sec" probe to automatically
      stop after a minute. The script (which we call "longtar.d"):

      syscall::exec*:return
      /execname == "tar"/
      {
      start[pid] = timestamp;
      }

      fbt::exit:entry
      /start[pid]/
      {
      @["tar time, in seconds"] =
      lquantize((timestamp - start[pid]) / 1000000000, 0, 60);
      start[pid] = 0;
      }

      tick-1sec
      /n++ >= 60/
      {
      exit(0);
      }

      Running this:

      # dtrace -s longtar.d
      dtrace: script 'longtar.d' matched 4 probes
      CPU ID FUNCTION:NAME
      23 38110 :tick-1sec

      tar time, in seconds
      value ------------- Distribution ------------- count
      < 0 | 0
      0 |@ 1
      1 |@@@@@ 4
      2 |@@@@@@@@@@@ 8
      3 |@@@@@@@ 5
      4 |@@@@@@@ 5
      5 |@@ 2
      6 | 0
      7 |@@@@ 3
      8 | 0

      This highlights just how difficult this problem would be to find using
      traditional (process-centric) tools: each tar process is only alive
      for (roughly) two seconds! And yet, taken together, the tar processes
      are accounting for the vast majority of cross calls on the system.

      This is a good example of using DTrace because the problem here is _not_ a
      single application -- it's an insurgent army of tar processes that are
      slipping away before they can be noticed. Indeed, running prstat(1) on this
      machine doesn't show tar -- or _any_ of the get_all2 progeny processes -- as
      top resource-consuming processes. Using DTrace, however, one can quickly
      root-cause aberrant systemic behavior of this nature.

      Of course, this is but a fraction of what DTrace can do. We encourage you to
      check out the AnswerBook at BigAdmin:

      http://www.sun.com/bigadmin/content/dtrace

      Or better yet, go to:

      http://wwws.sun.com/software/solaris/solaris-express/get.html

      There you can download Solaris Express 11/03 and try DTrace yourself!

      Enjoy,
      Bryan Cantrill, Mike Shapiro and Adam Leventhal
      DTrace Engineering Team

      ------------ End of Prior Message Posted by Bryan Cantrill -------------
    • John D Groenveld
      ... I wonder which of those characters gets to do the DTrace example on SUNWmoznav in Shanghai this June. [1] BTW The MadHatters have put SUNWmoz* 1.4 Beta for
      Message 2 of 12 , Mar 1, 2004
      • 0 Attachment
        > Bryan Cantrill, Mike Shapiro and Adam Leventhal

        I wonder which of those characters gets to do the DTrace example
        on SUNWmoznav in Shanghai this June. [1]

        BTW The MadHatters have put SUNWmoz* 1.4 Beta for x86 online.
        <URL:http://wwws.sun.com/software/solaris/browser/beta/index.html>

        1. Mark Tolliver has pushed back SunNetwork Asia to June 2-3.
        <URL:http://www.sun.com/sncasia2004/>

        John
        groenveld@...
      • Alan DuBoff
        ... I don t know, but all of them are exceptional developers. My vote would be for Bryan, not because he s any more exceptional than the others, but because
        Message 3 of 12 , Mar 1, 2004
        • 0 Attachment
          On Monday 01 March 2004 20:30, John D Groenveld wrote:
          > > Bryan Cantrill, Mike Shapiro and Adam Leventhal
          >
          > I wonder which of those characters gets to do the DTrace example
          > on SUNWmoznav in Shanghai this June. [1]

          I don't know, but all of them are exceptional developers.

          My vote would be for Bryan, not because he's any more exceptional than the
          others, but because he's very captivating when he speaks.

          I have on my agenda to try and get Bryan to do some video of using DTrace if
          possible, he really knows how to make his baby sing!

          --

          Alan DuBoff
          Software Orchestration, Inc.
          GPG: 1024D/B7A9EBEE 5E00 57CD 5336 5E0B 288B 4126 0D49 0D99 B7A9 EBEE
        • Laurent Blume
          ... Which reminds me: security-alert@sun answered me about SUNWmoz. They said the 1.4beta version does include the known vulnerabilities, but that the 1.4
          Message 4 of 12 , Mar 2, 2004
          • 0 Attachment
            Quoting John D Groenveld <jdg117@...>:

            > BTW The MadHatters have put SUNWmoz* 1.4 Beta for x86 online.
            > <URL:http://wwws.sun.com/software/solaris/browser/beta/index.html>

            Which reminds me:

            security-alert@sun answered me about SUNWmoz.

            They said the 1.4beta version does include the known vulnerabilities, but that
            the 1.4 final would not.

            I asked them if there would be a link with documentation on the topic, detailing
            the fixes included in Sun Moz and not in mozilla.org Moz, but didn't get an
            answer about that.

            Laurent
            --
            A hundred thousand lemmings can't be wrong!
          • Jörg Schilling
            ... DTrace ... it s one of ... really take ... can learn ... that Maybe this helps..... Let me explain why I did not yet test it: I am very intersted bu busy
            Message 5 of 12 , Mar 2, 2004
            • 0 Attachment
              --- In solarisx86@yahoogroups.com, Alan DuBoff <aland@s...> wrote:
              > This message was posted on comp.unix.solaris several months ago, but
              DTrace
              > is one of those tools that has not been talked about too much and
              it's one of
              > the very powerful tools in Solaris Express that developers should
              really take
              > time to look at.
              >
              > As such, I'm posting Bryan Cantrill's message here so that others
              can learn
              > and possibly be motivated to use DTrace. DTrace is one of the tools
              that

              Maybe this helps.....

              Let me explain why I did not yet test it:

              I am very intersted bu busy with hacking. I did not see any
              announcement for the slides from the Berlin talks (*), so I fetched
              the dtrace PDF and printed it. 300 pages that I could throw away
              as I found too late that they do not fit a A4 sheet but seem
              to be half of the size. Well, I am becoming an old man and
              I cannot read micro typograhpy anymore. I planed to print a resized
              version but it did remain a plan.

              The important problem for using dtrace is that it is complex,
              introduces a new programming language (d) and that the talk
              was nice and fast so you could get a wonderful overview but
              would need the slides to recall the talk later.

              Zones on the other side seem easy to set up so people use them
              immediately.



              *) As well as I never received one of the knapsacks from the
              conference although the hostesses told me on the 2nd day
              that they have none left but would send me one if they could
              scan my chip card.
            • palowoda
              ... I don t know if dtrace is any more complex than the Solaris kernel internals but would more cookbook examples help? I envision over time that dtrace will
              Message 6 of 12 , Mar 2, 2004
              • 0 Attachment
                --- In solarisx86@yahoogroups.com, Jörg Schilling <schily_x86@y...> wrote:

                > The important problem for using dtrace is that it is complex,
                > introduces a new programming language (d) and that the talk
                > was nice and fast so you could get a wonderful overview but
                > would need the slides to recall the talk later.
                >

                I don't know if dtrace is any more complex than the Solaris
                kernel internals but would more cookbook examples help?

                I envision over time that dtrace will be enhanced to view
                problems visually with graphs but it's still an evolving
                interface.


                > Zones on the other side seem easy to set up so people use them
                > immediately.

                Maybe too useful, very similar to bsd jails. I'm waiting for
                some nutcase to create 1000 zones and complain about the
                performance on a two cpu system.


                > *) As well as I never received one of the knapsacks from the
                > conference although the hostesses told me on the 2nd day
                > that they have none left but would send me one if they could
                > scan my chip card.

                Besides the Solaris Startup CD that are in the pack which can
                contain some very useful informantion. I never understood
                why the sessions where not recorded and available for online
                viewing in mpg4 format. I mean Sun has it's own video streaming
                server package. Might be difficult to view without a standard
                client application though.

                ---Bob
              • Robert Milkowski
                ... Dtrace is not so complex as a tool. Of course the better you understand system the more helpfull Dtrace is. Its language is so similar to C that you can
                Message 7 of 12 , Mar 2, 2004
                • 0 Attachment
                  On Tue, 2 Mar 2004, [iso-8859-1] Jörg Schilling wrote:
                  > The important problem for using dtrace is that it is complex,
                  > introduces a new programming language (d) and that the talk
                  > was nice and fast so you could get a wonderful overview but
                  > would need the slides to recall the talk later.


                  Dtrace is not so complex as a tool. Of course the better you understand
                  system the more helpfull Dtrace is. Its language is so similar to C that
                  you can hardly call it difficult to learn.

                  My advise is to try with it even without reading documentation (however
                  you should later) - with some simple examples and try to modify them.
                  It's really not so hard.

                  The hard part is how to get some information or I should say from where :)
                  The problem is not with the tool but with knowing/understanding the
                  system. I'm sure that book like Solaris Internals would be helpfull if
                  there were new edition for Solaris 10 and even more detailed.
                  Maybe Sun will write one? (maybe PDF?)


                  --
                  Robert Milkowski
                  milek@...
                • John D Groenveld
                  ... I submitted feedback to mozillafeedback@sun.com suggesting that the bugzilla bugids for security problems in the community mozilla and fixed in SUNWmoz* be
                  Message 8 of 12 , Mar 2, 2004
                  • 0 Attachment
                    In message <1078218563.40444f43a611b@...>, Laurent Blume writes:
                    >I asked them if there would be a link with documentation on the topic, detaili

                    I submitted feedback to mozillafeedback@... suggesting that the bugzilla
                    bugids for security problems in the community mozilla and fixed in SUNWmoz*
                    be referenced in the release notes.
                    <URL:chrome://global/locale/mozilla_release_notes.html>

                    John
                    groenveld@...
                  • Alan DuBoff
                    ... Yes, to some extent DTrace is a complex tool. It allows for complex analysis of the system in ways that just were not possible in the past. It is really a
                    Message 9 of 12 , Mar 2, 2004
                    • 0 Attachment
                      On Tuesday 02 March 2004 02:31, Jörg Schilling wrote:
                      > The important problem for using dtrace is that it is complex,
                      > introduces a new programming language (d) and that the talk
                      > was nice and fast so you could get a wonderful overview but
                      > would need the slides to recall the talk later.

                      Yes, to some extent DTrace is a complex tool. It allows for complex analysis
                      of the system in ways that just were not possible in the past. It is really a
                      remarkable tool when you see what it can do, and how it can be used.

                      None the less, it is nothing short of remarkable, and my hat is off to the
                      developers of DTrace, because this is exactly the type of technology that
                      sets Solaris apart from the rest of the pack, IMO.

                      Traditionally, this type of feature is something that has been offerred by a
                      compiler vendor, but being a part of the system is a big plus.

                      Please don't let the complexity prevent you from looking at this technology,
                      because I think you of all people will see great benifits in using such a
                      tool, and will leave to love it like one of your own family members! (family
                      members being cdrecord, mkisofs, etc...;-)

                      > Zones on the other side seem easy to set up so people use them
                      > immediately.

                      Yes, Zones are easier to get up and going, no question. Don't let this
                      deceive you, because some very complex things can be done with them as well.
                      Of course the more complex operations will require more thought and planning
                      to setup correctly. I also tip my hat to the Zones team, because this is yet
                      again another example of function/feature that will set Solaris apart from
                      the rest of the crowd when the next release comes out. This is also a
                      function/feature that is probably intended for a different crowd of users,
                      than DTrace which is more developer oriented.

                      --

                      Alan DuBoff
                      Software Orchestration, Inc.
                      GPG: 1024D/B7A9EBEE 5E00 57CD 5336 5E0B 288B 4126 0D49 0D99 B7A9 EBEE
                    • Alan DuBoff
                      ... I don t think it s possible to provide too many examples. The more the better. Let s see if we can get Sun to provide more. ... Yes, that would be nice.
                      Message 10 of 12 , Mar 2, 2004
                      • 0 Attachment
                        On Tuesday 02 March 2004 04:11, palowoda wrote:
                        > I don't know if dtrace is any more complex than the Solaris
                        > kernel internals but would more cookbook examples help?

                        I don't think it's possible to provide too many examples. The more the
                        better. Let's see if we can get Sun to provide more.

                        > I envision over time that dtrace will be enhanced to view
                        > problems visually with graphs but it's still an evolving
                        > interface.

                        Yes, that would be nice. IBM used to include a performance analyzer that
                        would produce graphics, but it was quite more difficult to use and required
                        that you place function calls in the top of funtions which you needed
                        analysis done on. DTrace is much easier in the sense of run it on the system
                        at runtime, and poof, here's your data. Very clever indeed.

                        > > Zones on the other side seem easy to set up so people use them
                        > > immediately.
                        >
                        > Maybe too useful, very similar to bsd jails. I'm waiting for
                        > some nutcase to create 1000 zones and complain about the
                        > performance on a two cpu system.

                        OTOH, there doesn't appear to be much overhead with a zone, so it might be
                        possible that dormant zones don't eat up all that much processing power at
                        all, and that 1000 might be realistic on some machines. I'm sure there is a
                        limit before the system starts to bog down, but even to have 1000 users on a
                        2 processor machine would be pretty intensive, so I would imagine that 1000
                        zones would cause some type of load on similar hardware.

                        > Besides the Solaris Startup CD that are in the pack which can
                        > contain some very useful informantion. I never understood
                        > why the sessions where not recorded and available for online
                        > viewing in mpg4 format. I mean Sun has it's own video streaming
                        > server package. Might be difficult to view without a standard
                        > client application though.

                        The Cisco opensource client, MPEG4IP, works fine as a client to Sun's
                        Streaming Server.

                        --

                        Alan DuBoff
                        Software Orchestration, Inc.
                        GPG: 1024D/B7A9EBEE 5E00 57CD 5336 5E0B 288B 4126 0D49 0D99 B7A9 EBEE
                      • Jörg Schilling
                        ... analysis ... really a ... to the ... that ... Definietly true: this is exactly the sort of stuff you will proably never get on Linux (at least not as a
                        Message 11 of 12 , Mar 2, 2004
                        • 0 Attachment
                          --- In solarisx86@yahoogroups.com, Alan DuBoff <aland@s...> wrote:

                          > Yes, to some extent DTrace is a complex tool. It allows for complex
                          analysis
                          > of the system in ways that just were not possible in the past. It is
                          really a
                          > remarkable tool when you see what it can do, and how it can be used.
                          >
                          > None the less, it is nothing short of remarkable, and my hat is off
                          to the
                          > developers of DTrace, because this is exactly the type of technology
                          that
                          > sets Solaris apart from the rest of the pack, IMO.

                          Definietly true: this is exactly the sort of stuff you will
                          proably never get on Linux (at least not as a first timer,
                          maybe as a replique).

                          > Traditionally, this type of feature is something that has been
                          offerred by a
                          > compiler vendor, but being a part of the system is a big plus.
                          >
                          > Please don't let the complexity prevent you from looking at this
                          technology,
                          > because I think you of all people will see great benifits in using
                          such a
                          > tool, and will leave to love it like one of your own family members!
                          (family
                          > members being cdrecord, mkisofs, etc...;-)

                          Thank you!

                          I did create a profiling SunOS-4.0 kernel when I was working
                          on my master thesis in 1990. I am definitely interested in this
                          kind of innovation and I love the way it is implemented (from
                          a talk I had with it's creator at ICC in december).
                          It is nice to see that there is a kernel debugger/profiler
                          that doesn't reduce the performance if you don't collect data.

                          What we need is a one page starter desciption that helps
                          to bootstrap brain.
                        • Alan DuBoff
                          ... Exactly. It s not that the Linux developers are not capable of doing the same thing, it s just that the more complicated features are often neglected due
                          Message 12 of 12 , Mar 2, 2004
                          • 0 Attachment
                            On Tuesday 02 March 2004 16:51, Jörg Schilling wrote:
                            > Definietly true: this is exactly the sort of stuff you will
                            > proably never get on Linux (at least not as a first timer,
                            > maybe as a replique).

                            Exactly. It's not that the Linux developers are not capable of doing the same
                            thing, it's just that the more complicated features are often neglected due
                            to the amount of work involved. OTOH, the opensource community has the
                            advantage of being able to pool many developers time into a lump sum to get
                            some type of product out the door.

                            > Thank you!
                            >
                            > I did create a profiling SunOS-4.0 kernel when I was working
                            > on my master thesis in 1990. I am definitely interested in this
                            > kind of innovation and I love the way it is implemented (from
                            > a talk I had with it's creator at ICC in december).
                            > It is nice to see that there is a kernel debugger/profiler
                            > that doesn't reduce the performance if you don't collect data.

                            I don't know if I can encourage Bryan Cantrill to help answer some questions
                            and/or encourage other people to use DTrace, but I'll see if there is
                            something he might be interested in doing.

                            > What we need is a one page starter desciption that helps
                            > to bootstrap brain.

                            There are some documents on DTrace up on BigAdmin that could help, aside from
                            the 300 page document (BTW, I concur, my eyes aren't as good as they used to
                            be either;-).

                            The article that I reposted on this mailing list was really meant to give a
                            person a quick intoduction to DTrace. I've seen Bryan do some demos and
                            DTrace is one amazing tool. Worth investing time to learn how to use and you
                            would be amazed at how it can detail out time spent in operations.

                            Here's a link to the article I reposted.

                            http://groups.google.com/groups?dq=&start=50&hl=en&lr=&ie=UTF-8&group=comp.unix.solaris&selm=bp4fa1%24civ%241%40news1nwk.SFbay.Sun.COM

                            --

                            Alan DuBoff
                            Software Orchestration, Inc.
                            GPG: 1024D/B7A9EBEE 5E00 57CD 5336 5E0B 288B 4126 0D49 0D99 B7A9 EBEE
                          Your message has been successfully submitted and would be delivered to recipients shortly.