Try running

  ldd `which `ls`

When I use 2 or more input/output files, I get an error that my FTP information is incorrect.

In your NewsClipper.cfg, change:

'ftpFiles' => [
   {},
#  {'server'   => 'SERVER',
#   'username' => 'USER NAME',
#   'password' => 'PASSWORD',
#   'dir'      => 'DEST DIR'},
],

'ftpFiles' => [
#  {'server'   => 'SERVER',
#   'username' => 'USER NAME',
#   'password' => 'PASSWORD',
#   'dir'      => 'DEST DIR'},
],

My license isn't working.

During registration, you gave several items which are used to build your registraction key. Here are some common problems:

1. Make sure the email in your NewsClipper.cfg is exactly the same as the one you gave during registration.
2. Make sure you are running News Clipper on the operating system you specified during registration.
3. Make sure you typed your registration key exactly as it appeared in your confirmation email.

When I run News Clipper, I get an error:

NewsClipper.cfg did not return a true value at /path/to/NewsClipper/bin/NewsClipper.pl line 134.

The only known solution is to upgrade your version of Perl to 5.005_03 or better.

I'm using a version of News Clipper earlier than 1.18, and when I run it it complains about hparser_xs_state.

Some installations of Perl have a newer version of HTML::Parser that is slightly incompatible with News Clipper. See instructions for correcting it here and here. NOTE: Be sure you modify the Parser.pm file located in the NewsClipper subdirectory, and not the one located in the HTML subdirectory.

I added a filter command and now I don't get any output. Here's my News Clipper command:

  <!--newsclipper
    <input name=dailynews>
    <filter name=limit number=5>
  -->

If you have anything besides a single input command, you must also provide an output command:

  <!--newsclipper
    <input name=dailynews>
    <filter name=limit number=5>
    <output name=array>
  -->

News Clipper can't seem to find my configuration file.

This is a bug in Perl 5.004 (and possibly earlier). Either use the -c flag, or update your version of Perl.

The output is missing end tags like </p>.

Update your version of HTML::Parser and HTML-Tree from CPAN.

I can't get News Clipper to work with Red Hat 5.2. It complains about calling main::-1.

Red Hat 5.2 is using a unsupported, unreleased, development version of Perl. We recommend installing the updated version distributed by RedHat.

1. Download the updated version of Perl for RedHat 5.2 from the following URL

  MIRROR/updates/5.2/PLATFORM/perl-5.004m7-1.PLATFORM.rpm
where MIRROR is a mirror site listed at http://www.redhat.com/download/mirror.html, and PLATFORM is i386, alpha, or sparc.
2. Install the RPM using the following command

  rpm -U perl-5.004-6.PLATFORM.rpm

Something is wrong, and I have no idea what.

Try turning News Clipper on DEBUG mode. Basically run it from the command line, with the -d switch. News Clipper will then send all its output to the screen with debugging information interspersed throughout. (Note that an output file will not be generated.)

If you are using an impaired command shell that doesn't let you scroll back to see the output, you can redirect the output to a file using a command like NewsClipper.exe -d > somefile.txt

When I go to the web page, new information is available, but News Clipper's data is old even though I just ran the script.

It's possible that there are small delays (on the order of an hour) between updates. If the delays are very long, contact the handler author and ask them to tune the update times.

Hey, this handler stopped working. What do I do?

First check that a fixed version isn't already available by running News Clipper with the -n switch. If that doesn't work, try contacting the author of the handler, whom you should be able to find by looking at the handler file, which is in ~/.NewsClipper/NewsClipper/handler.

Sometimes the output file it truncated after News Clipper is finished, and News Clipper takes a long time.

Sometimes servers that have the information News Clipper needs are not available. News Clipper waits a certain amount of time before giving up. In addition, News Clipper will kill itself after a certain amount of time elapses, so if there are a lot of downed servers, News Clipper might not finish processing the input file. To fix this, shorten the socketTimeout in your configuration file, and/or lengthen the scriptTimeout.

How do I get things set up as a cron job?

Cron runs under a restricted environment, where it basically runs /bin/sh, and doesn't process any .profile files. This means that your environment variables will not be set, such as your PATH. What people usually do is put their commands in an sh script that "sources" their profile and then executes the command, like so:

. /etc/profile
. /home/username/.profile
/path/to/command/thecommand

You have to give complete paths to every input and output file in the configuration. Also put the full path to your perl binary in the first line of the NewsClipper.pl file. (Using the #!/your/path/to/perl notation.) Then create a .crontab file in your root directory, and add something like the following line:
0 7,10,13,16,19,22 * * * /users/dwc3q/scripts/NewsClipper/NewsClipper.pl
Finally, type "crontab .crontab" to set up the cron job with the daemon.

I'm using Redhat Package Manager, and running into problems installing URI and libwww.

The problem is that URI supercedes URI::URL, which is included in libwww (LWP). Use rpm's --force option when installing URI, and install it after you've installed libwww. Also, don't try to install something like perl-LWP-attic-1.00 and perl-LWPng-alpha-0.23 instead of libwww -- it doesn't work.

Wait... Doesn't News Clipper violate all kinds of copyrights?

See the Copyright section below.

How can I temporarily disable a News Clipper tag?

Change "<!--newsclipper" to "<!--Xnewsclipper", so that the parser won't recognize the tag.

Cgiwrap reports that there is a server error. What's wrong?

Since cgiwrap runs under a very restricted shell, the path environment variable might not contain perl. Add "#!/your-perl-location/perl" as the first line of NewsClipper.pl.

I get the following error:

Couldn't get data. Error on HTTP request: Can't locate
auto/URI/URL/http/path_query.al in @INC (@INC contains:
/usr/local/lib/perl5/aix/5.00404 /usr/local/lib/perl5
/usr/local/lib/perl5/site_perl/aix /usr/local/lib/perl5/site_perl .
/some_path/NewsClipper-4.5).

You have URI::URL installed, not URI. URI::URL is deprecated, so install URI.

The date and time tags work okay, but for everything else, I get "Couldn't get blah" messages.

Make sure you have the right versions of the modules. Here's what we have:

HTML::Parser 2.06
HTML::FormatText 1.14 (from HTML-Tree)
URI 1.00
LWP::UserAgent 1.46 (from LWP 5.15)

Other, older versions might work, but they have not been tested.

How do I get News Clipper to work through a SOCKS proxy?

Sorry, but we have not tested it on firewalled accounts. If you have tried this, please let us know.

I want to write a handler that doesn't get the data from a page, but rather from a page that is a link from another page. How do I do that?

First, you'd probably only want to do this if the URL for the page you want changes frequently. (Otherwise you could just use it's URL directly.) Something like the following in the "Get" part of the handler might work:

$data = GetLinks("main page url", $startstring, $endstring);

# Get the first link
$url = $$data[0]

#extract the url you want from the href link
$url =~ s/.*<.*href\s*=\s*"(.*?)".*/$1/s;

$realdata = GetLinks($url, $otherstartstring,$otherendstring);

I want to install Perl modules, but I don't have root access. What do I do?

Normally, the perl5 module installation procedure includes commands something like these:

  % perl5 Makefile.PL
  % make
  % make test
  % make install

The first command, perl5 Makefile.PL, directs perl5 to create a makefile for the new module you are installing. When installing a perl5 module locally you must designate on the command line the home directory of your perl5 installation. That information is used by perl5 to create the makefile. Substitute the following command for perl5 Makefile.PL:

  % perl5 Makefile.PL PREFIX=/path/to/your/home/directory/perl

This will place your private copies of modules and their documentation in subdirectories of /path/to/your/home/directory/perl. So the complete installation process is:

  % perl5 Makefile.PL PREFIX=/path/to/your/home/directory/perl
  % make
  % make test
  % make install

For older modules it may be necessary to designate several other variables on the command line during the module installation:

  % perl5 Makefile.PL PREFIX=/path/to/your/home/directory/perl \
     INSTALLPRIVLIB=/path/to/your/home/directory/perl/lib \
     INSTALLSCRIPT=/path/to/your/home/directory/perl/bin \
     INSTALLSITELIB=/path/to/your/home/directory/perl/lib/site_perl \
     INSTALLBIN=/path/to/your/home/directory/perl/bin \
     INSTALLMAN1DIR=/path/to/your/home/directory/perl/man/man1 \
     INSTALLMAN3DIR=/path/to/your/home/directory/perl/man/man3

To save yourself some typing you can create a file and put these variable assignments above in the file something like this:

  PREFIX=/path/to/your/home/directory/perl \
  INSTALLPRIVLIB=/path/to/your/home/directory/perl/lib \
  INSTALLSCRIPT=/path/to/your/home/directory/perl/bin \
  INSTALLSITELIB=/path/to/your/home/directory/perl/lib/site_perl \
  INSTALLBIN=/path/to/your/home/directory/perl/bin \
  INSTALLMAN1DIR=/path/to/your/home/directory/perl/man/man1 \
  INSTALLMAN3DIR=/path/to/your/home/directory/perl/man/man3

Then, each time you install a perl5 module you can use the following syntax:

  % perl5 Makefile.PL `cat <filename>`
  % make
  % make test
  % make install

(Note the backticks on `cat <filename>`

Once you've installed your own modules, you'll need some way of telling Perl where they are. The following command should be put in your .bashrc, .cshrc, .login, or .profile. This command will prepend your new module locations to and existing search paths.

  % setenv PERL5LIB /path/to/your/home/directory/perl:$PERL5LIB

Run

  % finger <yourlogin>

to find out your shell, and then check the man page for your shell to find out how to set environment variables.

Module installation using CPAN.pm

An alternative to manually installing perl5 modules is the CPAN.pm module (see www.perl.com/CPAN/) which automates module download and installation. If you have perl5.004 or higher installed you have it bundled with the distribution. If not, you can download it from CPAN.

When you initially run the

  % perl5 -MCPAN -e shell

command, it will ask you a few questions. You can use all the defaults, except for this one

  Parameters for the 'perl Makefile.PL' command? [] PREFIX=/path/to/your/home/directory/perl INSTALLPRIVLIB=/path/to/your/home/directory/perl/lib INSTALLSCRIPT=/path/to/your/home/directory/perl/bin INSTALLSITELIB=/path/to/your/home/directory/perl/lib/site_perl INSTALLBIN=/path/to/your/home/directory/perl/bin INSTALLMAN1DIR=/path/to/your/home/directory/perl/man/man1 INSTALLMAN3DIR=/path/to/your/home/directory/perl/man/man3

(That should all be on the same line.)

After configuration of the module is complete, you will see a > prompt. Then you can try installing modules. To install the CGI module, do this:

  > install CGI

It will fetch the latest CGI module, unpack it, make it, test it and install it into your local area or the directory you specified as the PREFIX directory. The command:

  > i /CGI/

will return the list of modules that match that pattern.  The CPAN.pm module has more functionality, like checking for the latest modules, for example. Just run "perldoc CPAN" to read the man page.