Re: gift-write-feature-descs segfaults... (is the GIFT broken?)

[email protected]
Newsgroups gmane.comp.gnu.gift.general
Message-ID <[email protected]>
Replying to myself.. i'm bad at this.

Patch to repair the problem committed to the tree.

basically, I was just not testing the "second binary". that, and a mis-read.

My Fault, My Fix.

attaching fix here, and the script i now use to perform tests.

One Dead Bug.

Julia Longtin <[email protected]>


On Mon, Aug 28, 2006 at 01:41:03PM -0500, [email protected] wrote:
> David, Jonas:
> 
> I use a test script to verify my work before commit.
> I was just very singlemindedly testing the feature extractor only.
> 
> I've upgraded my test script to handle the InvertedFileFeatureDescription.db
> checks as well.
> 
> Jonas:
> that one function call isnt all of the problem. i've fixed it locally,
> and theres still corruption in the InvertedFile...
> 
> I'll be working on this for the next four or five hours, and will test
> the InvertedFile... before each patch.
> 
> Testing only helps if you actually test every part. :)
> 
> Doing what i can...
> 
> Julia Longtin <[email protected]>
> 
> 
> On Mon, Aug 28, 2006 at 11:31:48AM +0100, David Squire wrote:
> > Jonas Lindqvist wrote:
> > >
> > >David Squire wrote:
> > >
> > 
> > >A "test" makefile target would be great.
> > >
> > >>>For example, using the latest CVS version of the GIFT, I was able to 
> > >>>run gift-add-collection on a collection, and query it for a random 
> > >>>set of images. No problem. However, whenever asking for any similar 
> > >>>images, it failed.
> > >>Well of course I meant an actual similarity query. From my point of 
> > >>view getting random images is not really a query (though that is how 
> > >>it appears in MRML and the code). Originally (before MRML) I did this 
> > >>in the interface, not by querying.
> > >
> > >:-) Yes, of course... My example was a rather over-simplified way of 
> > >illustrating that some parts may work when others don't.
> > >Testing that we get *a* result doesn't really prove that the images in 
> > >the result are at all similar to the query image(s). But then again, 
> > >it's better than nothing.
> > >
> > >I'm not sure if it's doable in the GIFT, but it could be handy to be 
> > >able to break out parts of the code to be used in some sort of "unit 
> > >tests", to assure that image A is still quite similar to image B but 
> > >not to image C, etc. The sources could be distributed with a handful 
> > >of images that can be used for the unit-tests. With a controlled set 
> > >of images we could also test that the server produces a predictable 
> > >result over and over again. (Apart from the random stuff).
> > >But this is obvious of course...
> > >Unfortunately I don't know enough about the inner workings of the GIFT 
> > >to be of any help on this.
> > >(But I would love having tests like this, when playing with the code)
> > >
> > >
> > Yes, that is exactly the sort of thing I had in mind. We could bundle a 
> > tiny test collection (say 10 images) with the GIFT, and the test would 
> > add that collection, and then fire MRML queries at the server for each 
> > image, checking that the query result had the right images in the right 
> > order with the right similarity values. This is certainly possible using 
> > the Perl interface. The only part I would need to put more thought into 
> > right now would be how and where to run the various executables before 
> > "make install", and where the indexing files etc. would go.
> > 
> > Regards,
> > 
> > David
> > 
> > 
> > -- 
> > Dr David McG. Squire, Senior Lecturer, on sabbatical in 2006
> > Caulfield School of Information Technology, Monash University, Australia
> > CRICOS Provider No. 00008C       http://www.csse.monash.edu.au/~davids/
> > 
> > 
> > 
> > _______________________________________________
> > help-GIFT mailing list
> > [email protected]
> > http://lists.gnu.org/mailman/listinfo/help-gift
> 
> 
> _______________________________________________
> help-GIFT mailing list
> [email protected]
> http://lists.gnu.org/mailman/listinfo/help-gift

_______________________________________________
help-GIFT mailing list
[email protected]
http://lists.gnu.org/mailman/listinfo/help-gift
51-ChangeLog (text/plain, 563 B)
--- ../../anoncvs/gift/ChangeLog	2006-08-28 17:18:09.000000000 +0000
+++ ChangeLog	2006-08-28 21:06:40.000000000 +0000
@@ -1,3 +1,10 @@
+2006-08-28    <[email protected]>
+
+	* FeatureExtraction/extract_block_features.c
+	handle gabor_ranges() better, fix gabor_ranges to 100000 instead of 10000
+	* FeatureExtraction/write_feature_descs.c
+	update initialization function call, provide variables to be 'initialized'.
+
 2006-08-02    <[email protected]>
 	* FeatureExtraction/extract_block_features.c
 	define the kernals here, pass to gabor.c for init, use
51-FeatureExtraction_extract_block_features.c_fix.patch (text/plain, 1.4 KB)
--- ../../anoncvs/gift/FeatureExtraction/extract_block_features.c	2006-08-28 17:18:09.000000000 +0000
+++ FeatureExtraction/extract_block_features.c	2006-08-28 21:02:04.000000000 +0000
@@ -94,8 +94,8 @@
 double gabor_ranges_var[num_gabor_ranges] = {0.0625, 0.125, 0.25, 0.5, 1, 1.5, 2, 3, 4, 5, 6, 7, 8, 9, 10, 100000};
 #else
 #if num_gabor_ranges==10
-/* this is magic, as far as i'm concerned. [2..10,10000] */
-#define gabor_ranges(i) (i+2+9989*(((i>>3)&1)&(i&1)))
+/* this is magic, as far as i'm concerned. [2..10,100000] */
+#define gabor_ranges(i) (i+2+99989*(((i>>3)&1)&(i&1)))
 #else
 #error invalid setting for num_gabor_ranges
 #endif
@@ -649,7 +649,13 @@
 			/* blocks */
 			for (i = 0; i < square((image_size/gabor_block_size)); i++) {
 				for (j = 0; j < num_gabor_ranges; j++) {
+#if num_gabor_ranges==16 /* technically, we should be checking the type.. */
 					fprintf(out_file, "%d %d GABOR_POS block size = %dx%d position = %d SCALE, ORIENTATION, ENERGY = %d, %d, %f\n", feature_index, GABOR_POS, gabor_block_size, gabor_block_size, i, scale, orientation, gabor_ranges(j));
+#else
+#if num_gabor_ranges==10
+					fprintf(out_file, "%d %d GABOR_POS block size = %dx%d position = %d SCALE, ORIENTATION, ENERGY = %d, %d, %d.000000\n", feature_index, GABOR_POS, gabor_block_size, gabor_block_size, i, scale, orientation, gabor_ranges(j));
+#endif
+#endif
 				feature_index++;
 				}
 			}
51-FeatureExtraction_write_feature_descs.c_fix.patch (text/plain, 1.4 KB)
--- ../../dev3/gift/FeatureExtraction/write_feature_descs.c	2004-02-25 06:47:12.000000000 +0000
+++ FeatureExtraction/write_feature_descs.c	2006-08-28 20:11:19.000000000 +0000
@@ -8,6 +8,17 @@
 
 //#include "extract_features.proto"
 
+/* for c99 uint32_t */
+
+#include <stdint.h>
+
+/* this is common between extract_features and this file. -ENEEDHEADER? */
+
+/* the number of scales to recognise features at */
+#define num_gabor_scales 3
+/* the number of color blocks we're going to break the image into (overlapping) */
+#define num_total_colour_blocks (256+64+16+4)
+
 int main(int argc, char *argv[]) {
 
 	FILE *ppm_file;
@@ -16,6 +27,12 @@
 	enum file_types ppm_type;
 	enum ppm_error the_error;
 	int numH, numS, numV, numGrey;
+	/* the following variables are here just to be initialized. */
+	/* the colour count of each block. for quantizing the image */
+	uint32_t * col_counts[num_total_colour_blocks];
+	uint32_t ** gabor_histogram[num_gabor_scales];
+	uint32_t ** block_gabor_class[num_gabor_scales];
+
  
 	switch(argc) {
 	case 5:
@@ -60,7 +77,7 @@
 	}
 
 	/* initialise the variables required for the feature extraction */
-	init_feature_variables(colmap_size);
+        init_feature_variables(colmap_size, col_counts, block_gabor_class, gabor_histogram);
 
 	/* write them to the file */
 	if ((the_error = write_feature_descriptions(stdout, colmap, colmap_size)) != PPM_OK) {
run_test.sh (application/x-sh, 1.4 KB)
#!/bin/bash

blah=""
runs=25

target=tempsized

#trusted is concidered the reference copy
trusted_ext=gift-extract-features
trusted_wri=gift-write-feature-descs

#untrusted is the version we're testing
untrusted_ext=/disk1/gift/src/merge/dev/gift/FeatureExtraction/gift-extract-features
#untrusted_wri=gift-write-feature-descs
untrusted_wri=/disk1/gift/src/merge/dev/gift/FeatureExtraction/gift-write-feature-descs


#first, generate "origional trusted" versions of our two data files ($target.fts, InvertedFileFeatureDescription.db)
time $trusted_ext $target.ppm
mv $target.fts orig_$target.fts

$trusted_wri 18 3 3 4 < $target.ppm > orig_InvertedFileFeatureDescription.db

#now, timing test our feature extractor, doing a diff for comparison each iteration

for ((i=0;i<runs;i++)); do 
    {
	/usr/bin/time -o blah.out $untrusted_ext $target.ppm
	blah="$blah`cat blah.out | sed -n \"/^[0-9.]*user/p\" | sed \"s/^0*\([0-9.]*\).*/\1+/\" | tr -d \"\n\"`"
	diff -a $target.fts orig_$target.fts
	echo $blah
    } 
done

#report our result

echo -n $blah=" "
echo "scale=5; $blah" | sed "s/.$//" | bc	
bleh=`echo "scale=5; $blah" | sed "s/.$//" | bc`	
echo -n "magic number: "

echo "scale=5; $bleh / $runs " | bc

blah=""
bleh=""
runs=""

#next, test our write-feature-descs
$untrusted_wri 18 3 3 4 < $target.ppm > untrusted_InvertedFileFeatureDescription.db
diff -a untrusted_InvertedFileFeatureDescription.db orig_InvertedFileFeatureDescription.db
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.