[ macperl-Bugs-494368 ] quotewords crash

Bugs item #494368, was opened at 2001-12-17 14:50
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=107940&aid=494368&group_id=7940

Category: MacPerl Application
Group: Development Branch
Status: Open
Resolution: None
Priority: 5
Submitted By: Mike Pasini (mrpasini)
Assigned to: Nobody/Anonymous (nobody)
Summary: quotewords crash

Initial Comment:
I'm having a problem parsing comma-delimited data feeds 
using quotewords in MacPerl 5.6.1b2. As one particular 
feed is parsed, the system consistently crashes but with 
various errors (MacsBug StdLog attached). PowerPC 
illegal instruction and bus error while reading long 
word from user data space among them.

This project was originally developed under MacPerl 
5.2.0r4. Running the current code under that MacPerl 
does not crash either system.

MacPerl 5.2 has a memory allocation of 10-MB and 5.6 has 
15-MB. Adding significantly more to 5.6 (up to 80-MB) 
hasn't resolved the issue.

I have reproduced the crash on two machines:

- PB 3400, OS 8.1, 48-MB RAM, RAM Doubled to 120-MB

- 7300, OS 9.1, 192-MB, RAM Doubled to 256-MB

But only with one of several data feeds. The particular 
feed that is causing trouble is 652-KB with 455 lines 
and 11,442 chars in the longest line (24 fields, one of 
which is rather unrestricted text). The feeds are CSV 
files exported from Excel spreadsheets (both Windows and 
Macintosh) in which the data has been manually entered. 
Unfortunately, I can't narrow this down to a part of the 
troublesome feed (which, since it isn't mine, I haven't 
enclosed).

Here's a standalone, short version of the code that 
causes the problem:

#!/usr/local/bin/perl

use Text::ParseWords;
use Mac::Memory;

$l = 0;
@field_names = ();
@records = ();

open(FILE, "< CSVtext.txt") or die;
chomp($first_line = <FILE>);
if ($first_line =~ /,/) {
	$delimiter = ",";
	print "Comma delimited data: ";
} elsif ($first_line =~ /\t/) {
	$delimiter = "\t";
	print "Tab delimited data: ";
} elsif ($first_line =~ /|/) {
	$delimiter = "|";
	print "Vertical bar delimited data: ";
} else {
	print "Sorry, unknown delimiters\n";
	exit(0);
}
@field_names = &quotewords($delimiter,0, $first_line);
print scalar(@field_names) . " fields detected\n---\n";
foreach $name (@field_names) {
	print "   \" . $name . "\\n";
}
die;
while (<FILE>) {
	chomp($line = $_);
	if (length($line) == 0) { next; }
	$count++;
	$line =~ s/'/\'/g;    # unescaped quotes
	$line =~ s/(\d)"([^,\t|"])/$1\"$2/g;    # 
unescaped quotes
	if (length($line) > $l) { $l = length($line); }
	push(@records,[&quotewords($delimiter,0, $line)]);
	printf STDERR "$count [%0.2f/%0.2f]\n", FreeMem()/
(1024**2), (MaxMem())[0]/(1024**2);;
}
print "---\n$count lines, longest was $l characters.\n\
n";



----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=107940&aid=494368&group_id=7940