Loading ...
Sorry, an error occurred while loading the content.

Problems building heritrix-1.14.1 and 2

Expand Messages
  • stevenkhanderson
    Folks, I m having trouble building heritrix from 1.14.1 and 2 sources. Part of the problem is my unfamiliarity with maven, jelly, qdox, etc. The problems are
    Message 1 of 4 , Jan 13, 2009
    View Source
    • 0 Attachment
      Folks,

      I'm having trouble building heritrix from 1.14.1 and 2 sources.
      Part of the problem is my unfamiliarity with maven,
      jelly, qdox, etc.
      The problems are around "qdox" (version "current" -- which
      actual version does this correspond to?) and vdoclet (1.2),
      specifically:

      /maven-vdoclet-plugin-1.2/plugin.jelly (line 38)
      Element... vdoclet:vdocletBean

      Perhaps I've tried forcing the wrong version of qdox
      (copied from i'm not sure where) via the override mechanism.
      If I don't do this, it doesn't seem to know where to get
      qdox-current.jar from (I think someone stopped distributing it).

      Can someone try building from source, and verify that the
      release should actually build and download what it needs,
      and if not provide the appropriate .jars in the release?

      The reason I'm doing this is the version doesn't seem to spider
      yahoo correctly -- and I was going to dive into the code and
      try to see why. But first I need to be able to build the system.

      Maybe eventually I could switch to the new version, but the older
      version seems better filled out with conveniences, like a better
      web-based administration system, probably because the new one is
      still under construction.

      Thanks for any and all help.

      -- Steve
    • Noah Levitt
      Hello Steve, When you say heritrix-1.14.1 and 2 , I m not sure if you mean heritrix-1.14.2 or heritrix 2.0, but anyway the latest version on the 1.x line is
      Message 2 of 4 , Jan 14, 2009
      View Source
      • 0 Attachment
        Hello Steve,

        When you say "heritrix-1.14.1 and 2", I'm not sure if you mean
        heritrix-1.14.2 or heritrix 2.0, but anyway the latest version on the
        1.x line is 1.14.2. I would stick with the 1.x line until 2.2 comes out.

        I think I ran into the same problem building heritrix 1.14 as you:

        BUILD FAILED
        File...... /Users/nlevitt/.maven/cache/maven-xdoc-plugin-1.8/plugin.jelly
        Element... attainGoal
        Line...... 687
        Column.... 48
        The build cannot continue because of the following unsatisfied dependency:
        qdox-current.jar

        I ran "maven -X dist" to get more info, and saw this:

        Attempting to download qdox-current.jar.
        Getting URL: http://repo1.maven.org/maven/vdoclet/jars/qdox-current.jar
        Received status code: 404
        File not found on one of the repos
        java.io.FileNotFoundException:
        http://repo1.maven.org/maven/vdoclet/jars/qdox-current.jar

        http://repo1.maven.org/maven/readme.html advised me that this url is
        virtual, and to look at the CentralM1 repository at
        http://repository.sonatype.org/index.html. I found qdox-current.jar this
        way at
        http://repository.sonatype.org/content/repositories/centralm1/vdoclet/jars/qdox-current.jar.
        It would appear that the rewriting system for
        http://repo1.maven.org/maven/* is at least partially broken.

        So I downloaded
        http://repository.sonatype.org/content/repositories/centralm1/vdoclet/jars/qdox-current.jar
        to ~/.maven/repository/vdoclet/jars. After that the build worked. Hope
        this helps!

        Also, please feel free to email the list about it and/or submit an issue
        to jira (http://webteam.archive.org/jira/browse/HER) about the
        difficulties you're having with crawling.

        Noah


        stevenkhanderson wrote:
        > Folks,
        >
        > I'm having trouble building heritrix from 1.14.1 and 2 sources.
        > Part of the problem is my unfamiliarity with maven,
        > jelly, qdox, etc.
        > The problems are around "qdox" (version "current" -- which
        > actual version does this correspond to?) and vdoclet (1.2),
        > specifically:
        >
        > /maven-vdoclet-plugin-1.2/plugin.jelly (line 38)
        > Element... vdoclet:vdocletBean
        >
        > Perhaps I've tried forcing the wrong version of qdox
        > (copied from i'm not sure where) via the override mechanism.
        > If I don't do this, it doesn't seem to know where to get
        > qdox-current.jar from (I think someone stopped distributing it).
        >
        > Can someone try building from source, and verify that the
        > release should actually build and download what it needs,
        > and if not provide the appropriate .jars in the release?
        >
        > The reason I'm doing this is the version doesn't seem to spider
        > yahoo correctly -- and I was going to dive into the code and
        > try to see why. But first I need to be able to build the system.
        >
        > Maybe eventually I could switch to the new version, but the older
        > version seems better filled out with conveniences, like a better
        > web-based administration system, probably because the new one is
        > still under construction.
        >
        > Thanks for any and all help.
        >
        > -- Steve
        >
        >
        >
        >
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >
        >
      • stevenkhanderson
        Noah, Thanks -- that worked. I think I had the wrong qdox-current.jar before. Since your message, I tried from scratch, and apparently the old maven doesn t
        Message 3 of 4 , Jan 15, 2009
        View Source
        • 0 Attachment
          Noah,

          Thanks -- that worked. I think I had the wrong qdox-current.jar before.

          Since your message, I tried from scratch, and apparently the old
          maven doesn't understand redirects (301 HTTP response),
          which some apache locations are now responding --
          so good thing I saved the old .maven directory!
          Maven's great when it works, but maybe there needs to be a way
          to keep a local repository that you can make available with your
          own product, so no one pulls the rug out from under everyone...

          -- Steve
        • Noah Levitt
          Steve, that s an interesting idea. I filed http://webteam.archive.org/jira/browse/HER-1591. Feel free to add yourself as a watcher on that issue if you like. A
          Message 4 of 4 , Jan 15, 2009
          View Source
          • 0 Attachment
            Steve, that's an interesting idea. I filed
            http://webteam.archive.org/jira/browse/HER-1591. Feel free to add
            yourself as a watcher on that issue if you like. A couple of notes I put
            in the jira issue:

            - One complication is that there's a whole tree of dependencies and it
            might be unreasonable to try to include every branch/leaf of the tree.
            - Currently we only have a maven2 repository, so it would be extra work
            to cover heritrix 1.x.

            Noah

            stevenkhanderson wrote:
            > Noah,
            >
            > Thanks -- that worked. I think I had the wrong qdox-current.jar before.
            >
            > Since your message, I tried from scratch, and apparently the old
            > maven doesn't understand redirects (301 HTTP response),
            > which some apache locations are now responding --
            > so good thing I saved the old .maven directory!
            > Maven's great when it works, but maybe there needs to be a way
            > to keep a local repository that you can make available with your
            > own product, so no one pulls the rug out from under everyone...
            >
            > -- Steve
            >
            >
            >
            > ------------------------------------
            >
            > Yahoo! Groups Links
            >
            >
            >
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.