Loading ...
Sorry, an error occurred while loading the content.

Question about sparse option

Expand Messages
  • muzieklampje
    Dear Wolf. Would it be possible to add a saving as sparse (only for ntfs file system) option for the buffer file so the harddisk get less fragmented? With
    Message 1 of 4 , Feb 15, 2013
    • 0 Attachment
      Dear Wolf. Would it be possible to add a 'saving as sparse' (only for ntfs file system) option for the buffer file so the harddisk get less fragmented?
      With kind regards, Marc, The Netherlands.
    • wolf_dl4yhf
      Hello Marc, I have no idea how to instruct the file system to treat a file as sparse when creating it. From the few bits and pieces I read in wikipedia about
      Message 2 of 4 , Feb 15, 2013
      • 0 Attachment
        Hello Marc,

        I have no idea how to instruct the file system to treat a file as 'sparse' when creating it.
        From the few bits and pieces I read in wikipedia about sparse files, I also don't think its really appropriate here: The big advantage with sparse files is if a large portion of them contains zeroes, which are actually not written to the file. This is definitely not the file when storing FFTs (with floating point numbers) in spectrum labs buffer file. Sooner or later there will be no such gaps at all, and 'sparse' files (as I understand them to work) will actually cause *MORE* fragmentation than if the file was allocated as a single large block, physically.

        But again, I am not an expert in the NTFS file system so maybe someone around here has more details.

        All the best,
           Wolf .



        Am 15.02.2013 11:03, schrieb muzieklampje:
         

        Dear Wolf. Would it be possible to add a 'saving as sparse' (only for ntfs file system) option for the buffer file so the harddisk get less fragmented?
        With kind regards, Marc, The Netherlands.


      • Mark McCarron
        Wolf, Sparse files are a form of compression. Blocks of 0 are converted to small description stating how many 0 there are. It cuts down the amount of
        Message 3 of 4 , Feb 15, 2013
        • 0 Attachment
          Wolf,

          Sparse files are a form of compression.  Blocks of '0' are converted to small description stating how many '0' there are.  It cuts down the amount of I/O.    The problem is that it can become inefficent if the '0' are in small clusters.

          You can read more here:
          http://msdn.microsoft.com/en-gb/library/windows/desktop/aa365564%28v=vs.85%29.aspx

          and the implementation at code level here:
          http://www.codeproject.com/Articles/53000/Managing-Sparse-Files-on-Windows

          I would not recommend it for a near real-time system.  It can be useful for long term storage though.  Reading a file is transparent, but saving a file as sparse requires specific calls and needs to be enabled on the volume.  Unless you are prepared for the overhead of managing the analysis of a file, I would just forget it.  The current implementation is dumb as it would need to be completely transparent.  It can break other apps.

          Regards,

          Mark McCarron


          To: SpectrumLabUsers@yahoogroups.com
          From: dl4yhf@...
          Date: Fri, 15 Feb 2013 21:07:37 +0100
          Subject: Re: [SpectrumLabUsers] Question about sparse option

           

          Hello Marc,

          I have no idea how to instruct the file system to treat a file as 'sparse' when creating it.
          From the few bits and pieces I read in wikipedia about sparse files, I also don't think its really appropriate here: The big advantage with sparse files is if a large portion of them contains zeroes, which are actually not written to the file. This is definitely not the file when storing FFTs (with floating point numbers) in spectrum labs buffer file. Sooner or later there will be no such gaps at all, and 'sparse' files (as I understand them to work) will actually cause *MORE* fragmentation than if the file was allocated as a single large block, physically.

          But again, I am not an expert in the NTFS file system so maybe someone around here has more details.

          All the best,
             Wolf .



          Am 15.02.2013 11:03, schrieb muzieklampje:
           
          Dear Wolf. Would it be possible to add a 'saving as sparse' (only for ntfs file system) option for the buffer file so the harddisk get less fragmented?
          With kind regards, Marc, The Netherlands.




        • muzieklampje
          Thank you all for the input on this. I didn t realize that the fragmentationlevel can exeed when the buffer file is full in sparse mode. It s a feature in
          Message 4 of 4 , Feb 17, 2013
          • 0 Attachment
            Thank you all for the input on this. I didn't realize that the fragmentationlevel can exeed when the buffer file is full in 'sparse' mode. It's a feature in eMule here, but in that case when a file is completed, it's completed though. I noticed that the hdd of the measurement pc here got a fragmentationlevel of 40% over time, but I guess that the only solution for that is to do some more regular maintenance at this point.
            Kind regards, Marc

            --- In SpectrumLabUsers@yahoogroups.com, wolf_dl4yhf <dl4yhf@...> wrote:
            >
            > Hello Marc,
            >
            > I have no idea how to instruct the file system to treat a file as
            > 'sparse' when creating it.
            > From the few bits and pieces I read in wikipedia about sparse files, I
            > also don't think its really appropriate here: The big advantage with
            > sparse files is if a large portion of them contains zeroes, which are
            > actually not written to the file. This is definitely not the file when
            > storing FFTs (with floating point numbers) in spectrum labs buffer file.
            > Sooner or later there will be no such gaps at all, and 'sparse' files
            > (as I understand them to work) will actually cause *MORE* fragmentation
            > than if the file was allocated as a single large block, physically.
            >
            > But again, I am not an expert in the NTFS file system so maybe someone
            > around here has more details.
            >
            > All the best,
            > Wolf .
            >
            >
            >
            > Am 15.02.2013 11:03, schrieb muzieklampje:
            > >
            > > Dear Wolf. Would it be possible to add a 'saving as sparse' (only for
            > > ntfs file system) option for the buffer file so the harddisk get less
            > > fragmented?
            > > With kind regards, Marc, The Netherlands.
            > >
            > >
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.