Loading ...
Sorry, an error occurred while loading the content.
 

Re: "ocaml_beginners"::[] Any experience using oasis?

Expand Messages
  • Gabriel Scherer
    Makefiles, and more generally build and packaging tools of the Unix-like communities, tend to be designed to work well with the language C, and sometimes do
    Message 1 of 13 , Dec 1, 2011
      Makefiles, and more generally build and packaging tools of the
      Unix-like communities, tend to be designed to work well with the
      language C, and sometimes do not work so well with languages having
      different, stricter assumptions about how compilation should be done.

      Here are a few examples that may or may not be true, I'm no build system expert:
      1. C compilers are extremely lenient about compile order; basically
      you can give *.c files in any order you like, there is no cross-file
      dependency (but a common inclusion of .h files in the source) before
      linking; on the contrary OCaml is quite strict to get file paths in
      correct dependency order. Some build systems are impractical if you
      need strict file ordering, as they assume order doesn't matter so
      much.
      2. C files quite rarely need to be recompiled, as they deduce outside
      interfaces from their .h rather than depending on other modules, and
      have a quite lenient compatibility story; on the contrary, OCaml has
      an overly strict technique to guarantee type safety and representation
      coherence, it computes hashes of whole compilation units and require
      all files depending on them to be recompiled if their change

      Makefile caching mechanism will typically not be aware of point (2),
      which explains the not-infrequent "foo.cmi and bla.cmi makes
      inconsistent assumptions" error message. ocamlbuild tries to take care
      of this aspect. You have the same problem on the packaging/deployment
      front: usual Unix packaging systems do not expect to have to upgrade,
      say, the 'lwt' package whenever the 'ocaml' package gets an upgrade.
      GODI and, for example, the Debian ocaml packaging team, use special
      techniques to force recompilation / binary upgrade.

      On Thu, Dec 1, 2011 at 8:54 AM, <rixed@...> wrote:
      > If I can add a related question I've always been afraid to ask: what does ocaml
      > custom build systems bring over general-purpose and comunity-agnostic
      > Makefiles?  We already heard about portability to windows, is there anything
      > else?
      >
      >
      >
      > ------------------------------------
      >
      > Archives up to December 31, 2010 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners
      > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
      > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
      >
      >
      >
    • rixed@happyleptic.org
      -[ Thu, Dec 01, 2011 at 09:09:33AM +0100, Gabriel Scherer ]---- ... Either I don t see what you are referring to or your statement mix compilation with
      Message 2 of 13 , Dec 1, 2011
        -[ Thu, Dec 01, 2011 at 09:09:33AM +0100, Gabriel Scherer ]----
        > 1. C compilers are extremely lenient about compile order; basically
        > you can give *.c files in any order you like, there is no cross-file
        > dependency (but a common inclusion of .h files in the source) before
        > linking; on the contrary OCaml is quite strict to get file paths in
        > correct dependency order. Some build systems are impractical if you
        > need strict file ordering, as they assume order doesn't matter so
        > much.

        Either I don't see what you are referring to or your statement mix
        compilation with linking.

        AFAIK OCaml independent compilation mechanisms is copied from C, ie. you
        can compile any file independently of the others, in any order. I never
        encountered any difference between OCaml and C in this regard. So
        compilation is as easily handled by make in OCaml than in C (with the
        same drawback as in C: make is relying only on the file timestamp so
        changing a mere comment can yield a great amount of useless
        recompilation).

        Linking is another matter, since the unix system linker is not
        independent of the order of files on its command line. This is, again,
        the same with OCaml (although the order is reversed: while the
        traditional Unix linker will include symbols from a file if they were
        referenced in previous files, the OCaml linker requires that all
        mentioned modules in a file been already linked).

        All in all, I'm under the impression that OCaml build was inspired
        from C, perhaps because they regarded it as quite good (indeed, the make
        tool awarded an ACM prize to it's creator according to wikipedia :-)).
        This is one of OCaml aspect that makes this language attractive to C
        folks, I guess.

        > 2. C files quite rarely need to be recompiled, as they deduce outside
        > interfaces from their .h rather than depending on other modules, and
        > have a quite lenient compatibility story; on the contrary, OCaml has
        > an overly strict technique to guarantee type safety and representation
        > coherence, it computes hashes of whole compilation units and require
        > all files depending on them to be recompiled if their change
        >
        > Makefile caching mechanism will typically not be aware of point (2),
        > which explains the not-infrequent "foo.cmi and bla.cmi makes
        > inconsistent assumptions" error message.

        I don't know what caching mechanism you are referring to, but traditional
        make does not cache anything. The only occurrences of the not-infrequent
        error message about mismatch in module hashes I ever stumble upon
        involves cross-project builds (ie files A and B of project X uses
        external library Y, A and B are compiled alright then I recompile Y
        then go back to X and make a change in either A or B but not both ->
        boom!). I would says this relates to how dependencies are tracked, which
        is not make's business.

        > You have the same problem on the packaging/deployment
        > front: usual Unix packaging systems do not expect to have to upgrade,
        > say, the 'lwt' package whenever the 'ocaml' package gets an upgrade.
        > GODI and, for example, the Debian ocaml packaging team, use special
        > techniques to force recompilation / binary upgrade.

        Yes, Makefiles are not for packaging things, and I'm not questioning
        the use for a good packaging system. I'm only discussing the merit of
        the complexity added by custom _build_ systems over makefiles, not
        packaging systems.
        Anyway, GODI is build on top of makefiles so make can be part of the
        tool set for a good packaging system too :-)
      • Gabriel Scherer
        ... That s not exactly true as you need the .cmi of the files you depend on. echo let a = 1 a.ml echo let b = A.a b.ml File b.ml , line 1, characters
        Message 3 of 13 , Dec 1, 2011
          > AFAIK OCaml independent compilation mechanisms is copied from C, ie. you
          > can compile any file independently of the others, in any order.

          That's not exactly true as you need the .cmi of the files you depend on.

          echo "let a = 1" > a.ml
          echo "let b = A.a" > b.ml
          File "b.ml", line 1, characters 8-11:
          Error: Unbound module A
          ocamlc -c b.ml
          ocamlc -c a.ml
          ocamlc -c b.ml

          I believe this is a difference from C. But you're right that I was
          actually conflating compilation and linking: I was thinking of
          ordering of .cmo files during linking.

          Take this from a high-level point of view: for C, simple programs
          split in multiple files can be compiled with a simple "gcc *.c -o
          my_prog". On the contrary, such wildcards never work for OCaml. This
          makes writing Makefiles for OCaml a bit harder than for C -- even if
          make is very flexible and can absolutely be used in all kind of OCaml
          scenarii. I think one of ocamlbuild's main selling points is its
          automatic dependency discovery (there is also ocamldep, but I never
          found it as usable as ocamlbuild is).

          > I don't know what caching mechanism you are referring to, but traditional
          > make does not cache anything. The only occurrences of the not-infrequent
          > error message about mismatch in module hashes I ever stumble upon
          > involves cross-project builds

          I was awkwardly referring to the rebuild-on-timestamp-change policy.
          I'm thinking of this kind of scenario:
          - C depends on A and B, which also depends on A
          - you change a.ml and c.ml rebuild, and ask for recompilation of c; b
          hasn't changed so isn't recompiled, and you end up with inconsistent
          assumptions

          This problem is mostly solved by `ocamldep`; it's more an issue with
          simple, naive makefiles without enough dependency information. My
          point was that those simple makefiles work well for C in practice, and
          not for OCaml.

          > Anyway, GODI is build on top of makefiles so make can be part of the
          > tool set for a good packaging system too :-)

          Make is a very good tool and I don't wish to criticize it.

          But I think that, unless you are already very familiar with Make,
          ocamlbuild will probably be easier to use for a small ocaml project. I
          would advise beginners to use ocamlbuild ("just run `ocamlbuild
          foo.byte`") over Make ("well, you can write a make file, here is the
          documentation, you should probably use [OcamlMakefile] to make your
          life easier...").

          [OcamlMakefile]: http://hg.ocaml.info/release/ocaml-make

          I think that's the justification for OCamlbuild: a very simple build
          process in simple-to-moderately-complex cases.

          On Thu, Dec 1, 2011 at 9:50 AM, <rixed@...> wrote:
          > -[ Thu, Dec 01, 2011 at 09:09:33AM +0100, Gabriel Scherer ]----
          >> 1. C compilers are extremely lenient about compile order; basically
          >> you can give *.c files in any order you like, there is no cross-file
          >> dependency (but a common inclusion of .h files in the source) before
          >> linking; on the contrary OCaml is quite strict to get file paths in
          >> correct dependency order. Some build systems are impractical if you
          >> need strict file ordering, as they assume order doesn't matter so
          >> much.
          >
          > Either I don't see what you are referring to or your statement mix
          > compilation with linking.
          >
          > AFAIK OCaml independent compilation mechanisms is copied from C, ie. you
          > can compile any file independently of the others, in any order. I never
          > encountered any difference between OCaml and C in this regard. So
          > compilation is as easily handled by make in OCaml than in C (with the
          > same drawback as in C: make is relying only on the file timestamp so
          > changing a mere comment can yield a great amount of useless
          > recompilation).
          >
          > Linking is another matter, since the unix system linker is not
          > independent of the order of files on its command line. This is, again,
          > the same with OCaml (although the order is reversed: while the
          > traditional Unix linker will include symbols from a file if they were
          > referenced in previous files, the OCaml linker requires that all
          > mentioned modules in a file been already linked).
          >
          > All in all, I'm under the impression that OCaml build was inspired
          > from C, perhaps because they regarded it as quite good (indeed, the make
          > tool awarded an ACM prize to it's creator according to wikipedia :-)).
          > This is one of OCaml aspect that makes this language attractive to C
          > folks, I guess.
          >
          >> 2. C files quite rarely need to be recompiled, as they deduce outside
          >> interfaces from their .h rather than depending on other modules, and
          >> have a quite lenient compatibility story; on the contrary, OCaml has
          >> an overly strict technique to guarantee type safety and representation
          >> coherence, it computes hashes of whole compilation units and require
          >> all files depending on them to be recompiled if their change
          >>
          >> Makefile caching mechanism will typically not be aware of point (2),
          >> which explains the not-infrequent "foo.cmi and bla.cmi makes
          >> inconsistent assumptions" error message.
          >
          > I don't know what caching mechanism you are referring to, but traditional
          > make does not cache anything. The only occurrences of the not-infrequent
          > error message about mismatch in module hashes I ever stumble upon
          > involves cross-project builds (ie files A and B of project X uses
          > external library Y, A and B are compiled alright then I recompile Y
          > then go back to X and make a change in either A or B but not both ->
          > boom!). I would says this relates to how dependencies are tracked, which
          > is not make's business.
          >
          >> You have the same problem on the packaging/deployment
          >> front: usual Unix packaging systems do not expect to have to upgrade,
          >> say, the 'lwt' package whenever the 'ocaml' package gets an upgrade.
          >> GODI and, for example, the Debian ocaml packaging team, use special
          >> techniques to force recompilation / binary upgrade.
          >
          > Yes, Makefiles are not for packaging things, and I'm not questioning
          > the use for a good packaging system. I'm only discussing the merit of
          > the complexity added by custom _build_ systems over makefiles, not
          > packaging systems.
          > Anyway, GODI is build on top of makefiles so make can be part of the
          > tool set for a good packaging system too :-)
          >
          >
          >
          > ------------------------------------
          >
          > Archives up to December 31, 2010 are also downloadable at http://www.connettivo.net/cntprojects/ocaml_beginners
          > The archives of the very official ocaml list (the seniors' one) can be found at http://caml.inria.fr
          > Attachments are banned and you're asked to be polite, avoid flames etc.Yahoo! Groups Links
          >
          >
          >
        • Francois Berenger
          ... OK, I will give a try to Oasis then. I will try the binary installer or GODI if any problem. I don t know of odb.ml. Regards, F.
          Message 4 of 13 , Dec 1, 2011
            On 12/01/2011 04:43 PM, Sylvain Le Gall wrote:
            > Hello,
            >
            > On 01-12-2011, Francois Berenger <berenger@...
            > <mailto:berenger%40riken.jp>> wrote:
            > > --AbzQrc2f8IuL32mOVeSOwLRqkhh68QU2vK8afMS
            > > Content-Type: text/plain; charset=ISO-8859-1
            > > Content-Transfer-Encoding: 7bit
            > >
            > > Hello,
            > >
            > > Is it easy to use?
            > >
            >
            > Should be, but I need more inputs to make things more clear. This is an
            > active project that needs user input to improve.
            >
            > > Is it easy to create a new project and compile it with Oasis?
            > >
            > > I developed an allergy against ocamlbuild recently, and I really
            > > _love_ plain makefiles to do the job without doing things in my back.
            > >
            > > Is oasis installable even on quite old Linux box running dinosaurs
            > > like some RHEL 5.7 or the corresponding CentOS?
            > >
            >
            > The most problematic C dependencies you'll have will be libpcre (smthg
            > like version 6).
            >
            > Have a look at http://oasis.ocamlcore.org/dev/view/oasis/latest for
            > other deps:
            > unix, pcre, odn (>= 0.0.3), ocamlgraph, oUnit (>= 1.1.0), findlib,
            > fileutils (>= 0.4.2), extlib, expect.pcre (>= 0.0.2)
            >
            > It means that you'll need all this OCaml packages as well to build.
            >
            > I recommend using odb.ml or GODI (or the binary installer that was made
            > on an old version of centos).

            OK, I will give a try to Oasis then.

            I will try the binary installer or GODI if any problem.
            I don't know of odb.ml.

            Regards,
            F.

            > Cheers
            > Sylvain Le Gall
            >
            >
          • Francois Berenger
            ... Hi, Don t bother. I just find ocamlbuild too complex and not adapted to do some things. Recently, I read this:
            Message 5 of 13 , Dec 1, 2011
              On 12/01/2011 03:09 PM, Adrien wrote:
              > Hi,
              >
              > It's fairly easy to use for simple programs although it doesn't shine
              > there. But as soon as you have a more complex project, it quickly gets
              > very useful. And you get a configure and build system that works on
              > windows. I've also used it for C bindings where plain ocamlbuild
              > didn't work out-of-the-box and required more configuration.
              >
              > As for doing things behind your back, it doesn't really do that. The
              > whole stuff is very straightforward and only generates a bunch of
              > files which are then used by a build system. Currently, the only
              > available build system is ocamlbuild but new ones can be created.
              >
              > That means the question is rather whether ocamlbuild does things
              > behind your back and it usually does. However, the ocamlbuild files
              > generated by oasis are very simple and somehow "coerce" ocamlbuild
              > into behaving a very specific (and unsurprising) way. The only thing
              > left to ocamlbuild is sorting dependencies when compiling (which
              > modules depends on which other module and has to be compiled first?).
              >
              > Maybe you can tell us what was your issue with ocamlbuild and we'll
              > see if it's something that can happen when it is used in conjunction
              > with oasis.

              Hi,

              Don't bother.
              I just find ocamlbuild too complex and not adapted to do some things.
              Recently, I read this:

              http://mancoosi.org/~abate/ocamlbuild-stubs-and-dynamic-libraries

              And after reading the comment from Sylvain Le Gall I started to think
              "Oasis looks cool".

              I really think I will give it a try soon.

              > Also, makefiles are perfectly fine for small codebases but they don't
              > really scale and they also have their quirks like builtin rules which
              > can try to build foo.out out of a.o and b.o because that's how C works
              > (on a source tree which has already been compiled once).

              My only concern with them is that they require some maintenance.

              Regards,
              F.

              > --
              > Adrien Nader
              >
              >
            • rixed@happyleptic.org
              -[ Thu, Dec 01, 2011 at 10:28:37AM +0100, Gabriel Scherer ]---- ... Yes you are right! I forget all this cmi thing. This indeed makes a great difference. In
              Message 6 of 13 , Dec 1, 2011
                -[ Thu, Dec 01, 2011 at 10:28:37AM +0100, Gabriel Scherer ]----
                > > AFAIK OCaml independent compilation mechanisms is copied from C, ie. you
                > > can compile any file independently of the others, in any order.
                >
                > That's not exactly true as you need the .cmi of the files you depend on.
                >
                > echo "let a = 1" > a.ml
                > echo "let b = A.a" > b.ml
                > File "b.ml", line 1, characters 8-11:
                > Error: Unbound module A
                > ocamlc -c b.ml
                > ocamlc -c a.ml
                > ocamlc -c b.ml
                >
                > I believe this is a difference from C.

                Yes you are right! I forget all this cmi thing.
                This indeed makes a great difference. In practice, though, your
                makefile will have theses dependencies tracked. Here is the output
                of ocamldep for this example:

                a.cmo:
                a.cmx:
                b.cmo: a.cmo
                b.cmx: a.cmx

                So that with the additional rules :

                .SUFFIXES: .ml .cmo
                .ml.cmo:
                ocamlc -c $<

                make b.cmo now proceeds as required.
                (little game for ocamlbuild users: spot 7 errors in the above makefiles
                :-))

                > I think one of ocamlbuild's main selling points is its
                > automatic dependency discovery (there is also ocamldep, but I never
                > found it as usable as ocamlbuild is).

                Thank you, this might well be the other advantage of custom build system
                that I was asking for.

                > This problem is mostly solved by `ocamldep`; it's more an issue with
                > simple, naive makefiles without enough dependency information. My
                > point was that those simple makefiles work well for C in practice, and
                > not for OCaml.

                Well, there are plenty of buggy makefiles for C as well, actually.

                > I think that's the justification for OCamlbuild: a very simple build
                > process in simple-to-moderately-complex cases.

                So, that's a third advantage for ocamlbuild, then.

                Still, make is the only tool able to bind together a multi-cultural
                project (although, to be fair, I find omake a better general purpose
                make than make :-))

                Sorry if it all sounds like I'm criticizing OCaml builders for no
                reason. At work I have to fight the tendency of many developers to bound
                their horizon to whatever tools is provided by their community - each
                language community reinventing a build system, a test system, a package
                system, a code hosting service, a continuous deployment system, an
                editor (for the most advanced), an http server, and so on, although
                there are better general purpose tools at hand. It's frightening how often
                this leads to ridiculously sub-optimal situations... And I can't even
                trace this behavior back to some vendor marketing strategy since it's
                observable even in small niche communities without any vendor at sight -
                I guess most programmers like to produce code more than they like to
                learn what's already there :-)
              • Francois Berenger
                ... The proof that Yes, Ocaml can be used to do a build tool too! . Like the boost installer all in C++ (my guess), the Haskell compiler that is also a build
                Message 7 of 13 , Dec 1, 2011
                  On 12/01/2011 04:54 PM, rixed@... wrote:
                  > If I can add a related question I've always been afraid to ask: what
                  > does ocaml
                  > custom build systems bring over general-purpose and comunity-agnostic
                  > Makefiles? We already heard about portability to windows, is there anything
                  > else?

                  The proof that "Yes, Ocaml can be used to do a build tool too!".

                  Like the boost installer all in C++ (my guess), the Haskell
                  compiler that is also a build tool all in Haskell,
                  scons in Python, I don't know what in Ruby, etc. ;)
                • Adrien
                  ... I maintain a binding generator. Making a full build system with dependency sorting was not an option since it was far too complicated. I ve tried using
                  Message 8 of 13 , Dec 1, 2011
                    On 01/12/2011, Francois Berenger <berenger@...> wrote:
                    > On 12/01/2011 03:09 PM, Adrien wrote:
                    >> Hi,
                    >>
                    >> It's fairly easy to use for simple programs although it doesn't shine
                    >> there. But as soon as you have a more complex project, it quickly gets
                    >> very useful. And you get a configure and build system that works on
                    >> windows. I've also used it for C bindings where plain ocamlbuild
                    >> didn't work out-of-the-box and required more configuration.
                    >>
                    >> As for doing things behind your back, it doesn't really do that. The
                    >> whole stuff is very straightforward and only generates a bunch of
                    >> files which are then used by a build system. Currently, the only
                    >> available build system is ocamlbuild but new ones can be created.
                    >>
                    >> That means the question is rather whether ocamlbuild does things
                    >> behind your back and it usually does. However, the ocamlbuild files
                    >> generated by oasis are very simple and somehow "coerce" ocamlbuild
                    >> into behaving a very specific (and unsurprising) way. The only thing
                    >> left to ocamlbuild is sorting dependencies when compiling (which
                    >> modules depends on which other module and has to be compiled first?).
                    >>
                    >> Maybe you can tell us what was your issue with ocamlbuild and we'll
                    >> see if it's something that can happen when it is used in conjunction
                    >> with oasis.
                    >
                    > Hi,
                    >
                    > Don't bother.
                    > I just find ocamlbuild too complex and not adapted to do some things.
                    > Recently, I read this:
                    >
                    > http://mancoosi.org/~abate/ocamlbuild-stubs-and-dynamic-libraries
                    >
                    > And after reading the comment from Sylvain Le Gall I started to think
                    > "Oasis looks cool".
                    >
                    > I really think I will give it a try soon.

                    I maintain a binding generator. Making a full build system with
                    dependency sorting was not an option since it was far too complicated.
                    I've tried using ocamlbuild but stumbled on some issues which couldn't
                    be solved easily. That made me use OCamlMakefile which works quite
                    well for C bindings but I was probably not using properly and I still
                    had to do the dependency sorting (which I was doing with
                    ocamldepsort). I finally removed the ocamlmakefile support since it
                    was broken and added support for oasis. It's been working pretty well
                    so far.

                    The current _oasis file for lablwebkit can be seen at:
                    http://notk.org/~adrien/lablwebkit_oasis . The @CCOPT@ and @CCLIB@ are
                    replaced by the postconf command script which calls pkg-config (not
                    yet supported by oasis; my solution is a kind of hack).

                    Regards,
                    Adrien Nader
                  Your message has been successfully submitted and would be delivered to recipients shortly.