Loading ...
Sorry, an error occurred while loading the content.

File::Find troubles

Expand Messages
  • Richard Crawford
    This script is supposed to open an FTP connection from a Unix server to a Windows server and pull files in to the Unix server. The directory structures on
    Message 1 of 7 , Jul 1, 2003
    • 0 Attachment
      This script is supposed to open an FTP connection from a Unix server
      to a Windows server and pull files in to the Unix server. The
      directory structures on both servers are slightly difference -- hence
      the storm of regexp's in the move subroutine.

      The problem is this: in each subdirectory on both servers there is a
      subdirectory called "messages"; but when the program is running,
      File::Find seems unable to descend to that directory. I can't find
      any errors, and the permissions on the directories seem to be appropriate.

      If anyone has suggestions or comments, I would greatly appreciate it.

      #!/usr/local/bin/perl -w

      # FILE: vton.pl
      # PURPOSE: Retrieve current discussion board messages
      # from Vader and put them on Neo
      # AUTHOR: Richard Crawford (rcrawford@...)
      # CREATED: 25 June 2003
      # EDIT HISTORY:
      # ===============================================================
      # DATE AUTHOR EDIT
      # 07/01/03 RC Updated script to retrieve multiple files
      # ===============================================================

      use strict;
      use File::Find;
      use Net::FTP;




      my $vaderIP = "xxx.xxx.xxx.xxx";

      my $start_dir = shift or die "Usage: $0 <start_dir>\n";

      unless (-d $start_dir) {
      die "Start files $start_dir is not a directory.\n";
      }

      # Connect to Vader
      my $VaderConnect = Net::FTP->new($vaderIP) or die ("VTON: Can't
      connect to Vader: $!\n");

      $VaderConnect->login('Richard','mypassword') or die ("VTON: Can't log
      in to Vader: $!\n");
      $VaderConnect->ascii();

      print "\nVTON: Successfully connected to Vader.\n\n";

      # traverse directory tree and move files
      find (\&move, $start_dir);

      # Disconnect from Vader
      $VaderConnect->quit();

      sub move { #subroutine to move files

      print "\n\nMOVE subr output:\n\n";

      my $neoIP='unexdlc.ucdavis.edu';

      # Find the current directory
      my $neoDir = $File::Find::dir;
      print "\$neoDir: $neoDir\n";

      # Move to the equivalent directory on Vader
      my $baseDir = $neoDir;
      $baseDir =~ s#/space/usr/jrun4/servers/default/cfusion/DLC/##g;
      print "\$baseDir: $baseDir\n";
      my $vaderDir = "/HTTPHome/$baseDir";
      $vaderDir =~ s#/extras##g;
      print "\$vaderDir: $vaderDir\n";

      # Switch to equivalent directory on Vader
      $VaderConnect->cwd($vaderDir) or die ("Directory $vaderDir does not
      exist! $!\n");

      # Get all the files on Vader in this directory

      my @VaderFiles = $VaderConnect->ls("*.*") or die ("Cannot get list
      of files: $!\n");

      # Now get each item from Vader and process them

      foreach my $currentFile (@VaderFiles)
      {

      if (($currentFile =~ m/[0-9]+.html$/) || ($currentFile =~
      m/data.txt$/) || ($currentFile =~ m/wwwboard.html$/) || ($currentFile
      =~ m/wwwboard.pl$/))
      {
      # $currentFile =~ s(^.*/)();

      print "Fixing $currentFile\n";

      # Get the file from Vader
      $VaderConnect->get("$currentFile");

      # Replace "action=wwwboard.pl" with direct path to wwwboard.pl
      # Create an absolute path to wwwboard.pl
      $neoDir =~ s#/space/usr/jrun4/servers/default/cfusion/##g;

      my $URL="http://$neoIP/DLC/$baseDir/wwwboard.pl";
      $URL =~ s#/messages##g;
      my $VaderURL="http://$vaderIP/$baseDir/wwwboard.pl";
      $VaderURL =~ s#/messages##g;
      unless (open IN, $currentFile) {
      die "VTON: Can't open file $currentFile for reading: $!\n";
      }
      unless (open OUT, "> fixedfile") {
      die "VTON: Can't open output file: $!\n";
      }
      while (<IN>) {
      # Fix form action tags.
      s/action="wwwboard\.pl"/action="$URL"/gi;
      s/action="$VaderURL"/action="$URL"/gi;
      # Fix style sheet call:
      s#href="../../../../Library/#href="../../../../../Library/#g;
      if ($currentFile =~ m#[0-9]+.html$#) {

      s#href="../../../../../Library/#href="../../../../../../Library/#g;
      }
      print(OUT $_) or die "VTON: Can't write to fixedfile: $!\n";
      }
      close (IN); close (OUT);
      rename("fixedfile","$currentFile");

      chmod 0777, $currentFile;

      print "VTON: Copied and fixed $File::Find::dir/$currentFile\n\n";
      } else {
      print "VTON: $currentFile not fixed.\n";
      }
      }
      }
    • Damien Carbery
      ... appropriate. ... I don t think File::Find will work on FTP sites. I had a script that downloaded a directory tree from an FTP site. I think I issued a ls
      Message 2 of 7 , Jul 2, 2003
      • 0 Attachment
        --- In perl-beginner@yahoogroups.com, "Richard Crawford"
        <rcrawford@u...> wrote:
        > This script is supposed to open an FTP connection from a Unix server
        > to a Windows server and pull files in to the Unix server. The
        > directory structures on both servers are slightly difference -- hence
        > the storm of regexp's in the move subroutine.
        >
        > The problem is this: in each subdirectory on both servers there is a
        > subdirectory called "messages"; but when the program is running,
        > File::Find seems unable to descend to that directory. I can't find
        > any errors, and the permissions on the directories seem to be
        appropriate.
        >
        > If anyone has suggestions or comments, I would greatly appreciate it.
        >
        > <snip>

        I don't think File::Find will work on FTP sites.
        I had a script that downloaded a directory tree from an FTP site. I
        think I issued a 'ls -R' command through Net::FTP and it returned a
        list with a file on each line for all the subdirectories.
        I then parse each item in the list and downloaded the files as necessary.
      • Richard Crawford
        ... Hm, I wasn t using File::Find to traverse the remote directory tree; I was traversing the local directory tree, then using $myFTP- cwd() to move to the
        Message 3 of 7 , Jul 2, 2003
        • 0 Attachment
          At 08:44 AM 7/2/2003 +0000, you wrote:
          >I don't think File::Find will work on FTP sites.
          >I had a script that downloaded a directory tree from an FTP site. I
          >think I issued a 'ls -R' command through Net::FTP and it returned a
          >list with a file on each line for all the subdirectories.
          >I then parse each item in the list and downloaded the files as necessary.

          Hm, I wasn't using File::Find to traverse the remote directory tree; I was
          traversing the local directory tree, then using $myFTP->cwd() to move to
          the equivalent directory on the remote server.

          What I'm finding, though, is that File::Find doesn't seem to descend into a
          directory that is empty. Is that true? If so, how do I fix that?
        • Damien Carbery
          ... necessary. ... I was ... move to ... into a ... It probably does descend into the directory but your wanted function is never called while it is in the
          Message 4 of 7 , Jul 2, 2003
          • 0 Attachment
            --- In perl-beginner@yahoogroups.com, Richard Crawford
            <rcrawford@u...> wrote:
            > At 08:44 AM 7/2/2003 +0000, you wrote:
            > >I don't think File::Find will work on FTP sites.
            > >I had a script that downloaded a directory tree from an FTP site. I
            > >think I issued a 'ls -R' command through Net::FTP and it returned a
            > >list with a file on each line for all the subdirectories.
            > >I then parse each item in the list and downloaded the files as
            necessary.
            >
            > Hm, I wasn't using File::Find to traverse the remote directory tree;
            I was
            > traversing the local directory tree, then using $myFTP->cwd() to
            move to
            > the equivalent directory on the remote server.
            >
            > What I'm finding, though, is that File::Find doesn't seem to descend
            into a
            > directory that is empty. Is that true? If so, how do I fix that?

            It probably does descend into the directory but your 'wanted' function
            is never called while it is in the directory (because there is nothing
            in the dir to report on).

            Why do you want to descend into the empty directory?
          • Richard Crawford
            ... Because there is an equivalent directory on the other server that has files in it. Actually, the problem is pretty complicated. Here s a rundown. We re
            Message 5 of 7 , Jul 2, 2003
            • 0 Attachment
              At 04:02 PM 7/2/2003 +0000, you wrote:
              >It probably does descend into the directory but your 'wanted' function
              >is never called while it is in the directory (because there is nothing
              >in the dir to report on).
              >
              >Why do you want to descend into the empty directory?

              Because there is an equivalent directory on the other server that has files
              in it.

              Actually, the problem is pretty complicated. Here's a rundown.

              We're upgrading our servers from Windows to Solaris. In the process, we've
              altered the directory structure a little bit so that Apache can work with
              Perl when running the wwwboard script while also running JRun and Cold
              Fusion (ack!). So, on Windows, our directory structure looks something
              like this:

              CLASS
              +->DISBOARD
              +->SECTION
              +->MESSAGES

              While on the Solaris server, the directory structure looks like this:

              CLASS
              +->EXTRAS
              +->DISBOARD
              +->SECTION
              +->MESSAGES

              So there is one extra subdirectory on Solaris.

              My task is to move the files from class/disboard/ and its subdirectories to
              class/extras/disboard/ and matching subdirectories. Because some sections
              were created on our Windows server since the snapshot we took of it to
              build our Solaris server, there messages on Windows that aren't on Solaris;
              so some of the messages/ directories on Solaris have messages in them and
              some don't. And, of course, each message has to be processed as it is
              copied to change how it references the wwwboard.pl script.

              The solution I have is to descend into class/extras/disboard/section and
              its subdirectories on Solaris, then copy the files from the equivalent
              class/disboard/section subdirectory on Windows. But if the messages/
              subdirectory on Solaris is empty, File::Find won't descend into it, and so
              I can't get the files from the equivalent subdirectory on Windows.

              My hack for now is to use the touch command to create a dummy file in the
              messages/ subdirectory on Solaris so that there's something there so
              File::Find can descend into it. It's inelegant, though, and I don't like it.
            • Damien Carbery
              ... process, we ve ... with ... Can you tar up the files from Windows and simply untar them in the class/extras directory? Or cp -R from one to the other? If
              Message 6 of 7 , Jul 3, 2003
              • 0 Attachment
                --- In perl-beginner@yahoogroups.com, Richard Crawford
                <rcrawford@u...> wrote:
                > We're upgrading our servers from Windows to Solaris. In the
                process, we've
                > altered the directory structure a little bit so that Apache can work
                with
                > Perl when running the wwwboard script while also running JRun and Cold
                > Fusion (ack!). So, on Windows, our directory structure looks something
                > like this:
                > <snip>

                Can you tar up the files from Windows and simply untar them in the
                class/extras directory?
                Or 'cp -R' from one to the other?
                If you don't have to do any extra logic doing it by hand or a small
                shell script might be quicker.
              • Richard Crawford
                ... Unfortunately, there is a lot of extra logic to do. We re changing our entire operating system and directory structure, so each file needs to have changes
                Message 7 of 7 , Jul 3, 2003
                • 0 Attachment
                  At 08:50 AM 7/3/2003 +0000, Damien Carbery wrote:

                  >Can you tar up the files from Windows and simply untar them in the
                  >class/extras directory?
                  >Or 'cp -R' from one to the other?
                  >If you don't have to do any extra logic doing it by hand or a small
                  >shell script might be quicker.

                  Unfortunately, there is a lot of extra logic to do. We're changing our
                  entire operating system and directory structure, so each file needs to have
                  changes made: how form actions are called, how style sheets are linked, and
                  so on.

                  I've solved the problem, though, by adding a quick subroutine that creates
                  a dummy file in the empty directories long enough for File::Find to descend
                  into them so that the program can do what it needs to do, then deletes it
                  when the program is finished.
                Your message has been successfully submitted and would be delivered to recipients shortly.