Loading ...
Sorry, an error occurred while loading the content.
 

Re: [XSL-FO] FOP: Will not process large file

Expand Messages
  • Peter Wu
    ...yeah, with 256 MB physical memory, the Windows swap file would only be 256 MB by default giving you only 512 MB to work with. It may be necessary to up
    Message 1 of 13 , Nov 2 2:30 PM
      ...yeah, with 256 MB physical memory, the Windows swap
      file would only be 256 MB by default giving you only
      512 MB to work with. It may be necessary to up Windows
      vitual memory setting to get higher -Xmx values to
      work. I tried a number of other settings and
      combination of settings as suggested by people in
      various forums and none of them worked except for -Xmx
      when it came to getting Apache FO to complete large
      jobs. I assume that this your personal computing
      resource as opposed to shared resource and that you
      are willing to risk crashing it.

      I notice that you're using -mx instead of -Xmx. I
      don't know if there's any difference but -mx doesn't
      work for me. If -Xmx is the right switch, then you
      might want to retry -Xmx256m to see if the job
      completes and incrementally increase it from that
      point as needed.

      By the way, you might want to swap out Apache FO and
      replace it with http://www.lunasil.com/ as a long term
      solution. They claim to be more efficient.



      --- "J.Pietschmann" <j3322ptm@...> wrote:

      > Peter Wu wrote:
      > > try java -Xmx1000m -jar fop.jar -fo
      > "C:\770470789E.fo"
      > ^^^^
      > Do *not* do this on a machine with 256M physical
      > memory.
      > A common strategy is to set heap and stack size on
      > such a
      > machine each to 128m or 96m. Watch both the real
      > memory
      > teh JVM process is using and the swapping activity.
      > A
      > memory profiler may be used for gathering additional
      > hints
      > for tweaking the memory settings.
      > Using the -server switch may also help.
      >
      > J.Pietschmann
      >
    • jasmine2501@netzero.com
      Yes, I immediately realized the obvious problem with that. We found that my machine actually has 1GB real chips, and I set the JVM to use 500MB. It still
      Message 2 of 13 , Nov 2 5:47 PM
        Yes, I immediately realized the obvious problem with that. We found that my machine actually has 1GB real chips, and I set the JVM to use 500MB. It still doesn't actually use that much (it goes up and down, peaking at about 350MB for the JVM process), and it works ok, but it stalls a lot and takes a super long time. I have let it run overnight, and we'll see what happens. By the time I left the office, it had done 146 pages in about a half hour. It seemed to stall for a long time after that, and if it's not done in the morning, we'll try something else. In my initial java command, I had left out the 'X' so it was ignoring my request for more memory. Crossing my fingers and hoping this will work, and we only have to render this PDF for this client once every 6 months, so it's cool to do it by hand. All the other clients have much smaller PDFs and we generate those upon request, which seems to work fine using the FOP servlet and an HTTP request from my C# program. I have learned a lot of new things on this project and from the folks on this group. Thank you all very much for your helpful suggestions!

        Jasmine


        ----------------------------------------------------------------


        Peter Wu wrote:
        > try java -Xmx1000m -jar fop.jar -fo "C:\770470789E.fo"
        ^^^^
        Do *not* do this on a machine with 256M physical memory.
        A common strategy is to set heap and stack size on such a
        machine each to 128m or 96m. Watch both the real memory
        teh JVM process is using and the swapping activity. A
        memory profiler may be used for gathering additional hints
        for tweaking the memory settings.
        Using the -server switch may also help.

        J.Pietschmann
      • jasmine2501@netzero.com
        I checked out Lunasil, but to be honest, I would like to get away from using java-based stuff altogether. We are a Microsoft shop, with backend mainframes and
        Message 3 of 13 , Nov 2 5:58 PM
          I checked out Lunasil, but to be honest, I would like to get away from using java-based stuff altogether. We are a Microsoft shop, with backend mainframes and such. Integrating the COBOL files from the mainframes is enough work for me right now, and the .NET stuff seems to work pretty well. I have done a .NET object in C# which creates the 21MB FO file I spoke of, and it does the database call (9400 rows X about 1K each row), and the transformation, inserting the data into the FO template, in like 3 seconds. What I would really like is a .NET or COM compatible PDF creator, but I have been unable to find one except for Crystal Reports, which sucks. That's why we're using FOP in the first place. It is funky though because I have to do a call out to the FOP servlet which runs in Apache Tomcat. I would like to be able to just create an instance of an object and control it from my C# program, but have not been able to find anything capable of this. If anyone has any suggestions I would greatly appreciate it.

          Have a groovy day y'all,
          Jasmine
        • Colin Spear
          Hi Jasmine I had similar problems like this a few years back....I the only way we found to be able to get around it was to allocate more memory to the Java VM,
          Message 4 of 13 , Nov 2 10:27 PM
            Hi Jasmine

            I had similar problems like this a few years back....I the only way we
            found to be able to get around it was to allocate more memory to the
            Java VM, placing this line at the end of the fop.bat file:

            java -cp "%LOCALCLASSPATH%" -Xms512m -Xmx1024m org.apache.fop.apps.Fop
            %1 %2 %3 %4 %5 %6 %7 %8

            Most of the XML's we deal with are just over 1Mb in size, so nowhere
            near your monster, but who knows...it may work.

            -----Original Message-----
            From: XSL-FO@yahoogroups.com [mailto:XSL-FO@yahoogroups.com] On Behalf
            Of Jasmine
            Sent: Wednesday 02 November 2005 18:48
            To: XSL-FO@yahoogroups.com
            Subject: [XSL-FO] FOP: Will not process large file


            I have a really large file I want to process with FOP. The XSL-FO file
            is 21MB. It is as small as we can make it. We have considered breaking
            it up into smaller files, but for our client, this would kind of defeat
            the purpose of generating the PDF in the first place. They can look at
            smaller chunks on the web site, but the PDF report is supposed to be a
            complete listing. We only have one client out of 8500 which has this
            problem, but they are the biggest and most valuable client, and we would
            like to make this work for them. Am I running up against some kind of
            limit for FOP? I don't think I'm exceeding the limit for PDF, cuz I know
            I've seen PDF files much larger than this. This PDF, if processed
            correctly, would result in 547 pages. I can not send sample code because
            it contains confidential healthcare information.

            Thanks,
            Jasmine








            Yahoo! Groups Links







            This message and any attachments are confidential and intended solely for the addressee. If you have received this message in error, please notify Discovery immediately, telephone number +27 11 529 2888. Any unauthorised use; alteration or dissemination of the contents of this email is strictly prohibited. In no event will Discovery or the sender be liable in any manner whatsoever to any person for any loss or any direct, indirect, special or consequential damages arising from use of this email or any linked website, including, without limitation, from any lost profits, business interruption, loss of programmes or other data that may be stored on any information handling system or otherwise from any assurance that this email is virus free even if Discovery is expressly advised of the possibility of such damages. Discovery is an Authorised Financial Services Provider.
          • Peter B. West
            ... I ... Jasmine, Oh dear, getting away from Java! Have you looked at AltSoft s (commercial) offering? As for processing large files in FOP, check the size
            Message 5 of 13 , Nov 3 3:14 AM
              jasmine2501@... wrote:
              > I checked out Lunasil, but to be honest, I would like to get away from using java-based stuff altogether. We are a Microsoft shop, with backend mainframes and such. Integrating the COBOL files from the mainframes is enough work for me right now, and the .NET stuff seems to work pretty well. I have done a .NET object in C# which creates the 21MB FO file I spoke of, and it does the database call (9400 rows X about 1K each row), and the transformation, inserting the data into the FO template, in like 3 seconds. What I would really like is a .NET or COM compatible PDF creator, but I have been unable to find one except for Crystal Reports, which sucks. That's why we're using FOP in the first place. It is funky though because I have to do a call out to the FOP servlet which runs in Apache Tomcat. I would like to be able to just create an instance of an object and control it from my C# program, but have not been able to find anything capable of this. If anyone has any suggestions

              I
              > would greatly appreciate it.
              >
              > Have a groovy day y'all,
              > Jasmine

              Jasmine,

              Oh dear, getting away from Java! Have you looked at AltSoft's
              (commercial) offering?

              As for processing large files in FOP, check the size of your
              page-sequences. Do you have one large page-sequence? If so, see if
              generating the file as a series of small page-sequences will work. If
              not, I am out of suggestions.

              I'm currently working on a project <http://defoe.sourceforge.net/folio/>
              which will handle large files. When I have something to show, I'll post
              here.

              Peter

              --
              Peter B. West <http://cv.pbw.id.au/>
              Folio <http://defoe.sourceforge.net/folio/>


              [Non-text portions of this message have been removed]
            • Olivier Collioud
              http://www.princexml.com ? ... I checked out Lunasil, but to be honest, I would like to get away from using java-based stuff altogether. We are a Microsoft
              Message 6 of 13 , Nov 3 3:44 AM
                http://www.princexml.com ?

                >>> jasmine2501@... 3/11/05 2:58:12 AM >>>
                I checked out Lunasil, but to be honest, I would like to get away from
                using java-based stuff altogether. We are a Microsoft shop, with backend
                mainframes and such. Integrating the COBOL files from the mainframes is
                enough work for me right now, and the .NET stuff seems to work pretty
                well. I have done a .NET object in C# which creates the 21MB FO file I
                spoke of, and it does the database call (9400 rows X about 1K each row),
                and the transformation, inserting the data into the FO template, in like
                3 seconds. What I would really like is a .NET or COM compatible PDF
                creator, but I have been unable to find one except for Crystal Reports,
                which sucks. That's why we're using FOP in the first place. It is funky
                though because I have to do a call out to the FOP servlet which runs in
                Apache Tomcat. I would like to be able to just create an instance of an
                object and control it from my C# program, but have not been able to find
                anything capable of this. If anyone has any suggestions I!
                would greatly appreciate it.

                Have a groovy day y'all,
                Jasmine





                Yahoo! Groups Links








                ------
                World Intellectual Property Organization Disclaimer:

                This electronic message may contain privileged, confidential and
                copyright protected information. If you have received this e-mail
                by mistake, please immediately notify the sender and delete this
                e-mail and all its attachments. Please ensure all e-mail attachments
                are scanned for viruses prior to opening or using.
              • Werner Donné
                Jasmine, XEP from RenderX is also available for .NET. Regards, Werner. ... -- Werner Donné -- Re BVBA Engelbeekstraat 8 B-3300 Tienen tel: (+32) 486 425803
                Message 7 of 13 , Nov 3 4:03 AM
                  Jasmine,

                  XEP from RenderX is also available for .NET.

                  Regards,

                  Werner.

                  Olivier Collioud wrote:
                  > http://www.princexml.com ?
                  >
                  >
                  >>>>jasmine2501@... 3/11/05 2:58:12 AM >>>
                  >
                  > I checked out Lunasil, but to be honest, I would like to get away from
                  > using java-based stuff altogether. We are a Microsoft shop, with backend
                  > mainframes and such. Integrating the COBOL files from the mainframes is
                  > enough work for me right now, and the .NET stuff seems to work pretty
                  > well. I have done a .NET object in C# which creates the 21MB FO file I
                  > spoke of, and it does the database call (9400 rows X about 1K each row),
                  > and the transformation, inserting the data into the FO template, in like
                  > 3 seconds. What I would really like is a .NET or COM compatible PDF
                  > creator, but I have been unable to find one except for Crystal Reports,
                  > which sucks. That's why we're using FOP in the first place. It is funky
                  > though because I have to do a call out to the FOP servlet which runs in
                  > Apache Tomcat. I would like to be able to just create an instance of an
                  > object and control it from my C# program, but have not been able to find
                  > anything capable of this. If anyone has any suggestions I!
                  > would greatly appreciate it.
                  >
                  > Have a groovy day y'all,
                  > Jasmine
                  >
                  >
                  >
                  >
                  >
                  > Yahoo! Groups Links
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  >
                  > ------
                  > World Intellectual Property Organization Disclaimer:
                  >
                  > This electronic message may contain privileged, confidential and
                  > copyright protected information. If you have received this e-mail
                  > by mistake, please immediately notify the sender and delete this
                  > e-mail and all its attachments. Please ensure all e-mail attachments
                  > are scanned for viruses prior to opening or using.
                  >
                  >
                  >
                  >
                  >
                  >
                  > Yahoo! Groups Links
                  >
                  >
                  >
                  >
                  >
                  >
                  >

                  --
                  Werner Donné -- Re BVBA
                  Engelbeekstraat 8
                  B-3300 Tienen
                  tel: (+32) 486 425803 e-mail: werner.donne@...
                • Victor Vishnyakov
                  I would recommend to take a look at Altsoft Xml2PDF (http://alt-soft.com/products_xml2pdf_download.jsp). Based on my experience it is one of the most compliant
                  Message 8 of 13 , Nov 3 5:10 AM
                    I would recommend to take a look at Altsoft Xml2PDF
                    (http://alt-soft.com/products_xml2pdf_download.jsp). Based on my experience
                    it is one of the most compliant .NET formatters for XSL-FO.


                    Best regards,
                    Victor Vishnyakov
                    http://xslfo-zone.com/



                    > -----Original Message-----
                    > From: XSL-FO@yahoogroups.com [mailto:XSL-FO@yahoogroups.com]
                    > On Behalf Of Olivier Collioud
                    > Sent: Thursday, November 03, 2005 1:45 PM
                    > To: XSL-FO@yahoogroups.com
                    > Subject: Re: [XSL-FO] FOP: Will not process large file
                    >
                    > http://www.princexml.com ?
                    >
                    > >>> jasmine2501@... 3/11/05 2:58:12 AM >>>
                    > I checked out Lunasil, but to be honest, I would like to get
                    > away from using java-based stuff altogether. We are a
                    > Microsoft shop, with backend mainframes and such. Integrating
                    > the COBOL files from the mainframes is enough work for me
                    > right now, and the .NET stuff seems to work pretty well. I
                    > have done a .NET object in C# which creates the 21MB FO file
                    > I spoke of, and it does the database call (9400 rows X about
                    > 1K each row), and the transformation, inserting the data into
                    > the FO template, in like
                    > 3 seconds. What I would really like is a .NET or COM
                    > compatible PDF creator, but I have been unable to find one
                    > except for Crystal Reports, which sucks. That's why we're
                    > using FOP in the first place. It is funky though because I
                    > have to do a call out to the FOP servlet which runs in Apache
                    > Tomcat. I would like to be able to just create an instance of
                    > an object and control it from my C# program, but have not
                    > been able to find anything capable of this. If anyone has any
                    > suggestions I!
                    > would greatly appreciate it.
                    >
                    > Have a groovy day y'all,
                    > Jasmine
                    >
                    >
                    >
                    >
                    >
                    > Yahoo! Groups Links
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    >
                    > ------
                    > World Intellectual Property Organization Disclaimer:
                    >
                    > This electronic message may contain privileged, confidential
                    > and copyright protected information. If you have received
                    > this e-mail by mistake, please immediately notify the sender
                    > and delete this e-mail and all its attachments. Please ensure
                    > all e-mail attachments are scanned for viruses prior to
                    > opening or using.
                    >
                    >
                    >
                    >
                    > ------------------------ Yahoo! Groups Sponsor
                    > --------------------~--> Get fast access to your favorite
                    > Yahoo! Groups. Make Yahoo! your home page
                    > http://us.click.yahoo.com/dpRU5A/wUILAA/yQLSAA/9rHolB/TM
                    > --------------------------------------------------------------
                    > ------~->
                    >
                    >
                    > Yahoo! Groups Links
                    >
                    >
                    >
                    >
                    >
                    >
                  Your message has been successfully submitted and would be delivered to recipients shortly.