patch for puri
Ryan Davis <[email protected]>
| Newsgroups | gmane.lisp.clsql.general |
|---|---|
| Message-ID | <[email protected]> |
I ran into some problems with puri, and have a patch with tests. The basic issue is I have some vendors feeding me URIs with escaped ASCII control characters in the query. Puri was decoding "?foo%0abar" as "?foo bar" (with a #\Newline character), and then when encoding the URI via render-uri, the #\Newline was not getting converted back to "%0a", and that crashed the rest of the pipeline. Looking through http://www.ietf.org/rfc/rfc2396.txt, section 2.4.3 states that US-ASCII coded characters between 00-1F and 7F should be excluded. I've attached a patch with 2 commits that comply with section 2.4.3. The first patch adds a unit test and puts #x00 thru #x1F in the *excluded-characters* list, which solved my "%0a" problem. The second patch does the same thing for #x7F. Passing "%7f" in the query was causing an index out of bounds error. I put that in a separate patch because in order to get that working I had to bump the bit vector length in #'reserved-char-vector to 128, and I wasn't sure how the 127 length was initially selected. This patch can be applied with "git am puri-control-characters.patches", and should be a fast-forward. Thanks, Ryan _______________________________________________ CLSQL mailing list [email protected] http://lists.b9.com/cgi-bin/mailman/listinfo/clsql
puri-control-characters.patches
(text/plain, 3.8 KB)
From 78d5cf23974df22f6960c847b45090483b4b1ca8 Mon Sep 17 00:00:00 2001 From: Ryan Davis <[email protected]> Date: Fri, 2 Apr 2010 16:41:19 -0400 Subject: [PATCH 1/2] Add US-ASCII unprintable control characters to the exlusion list per RFC2396 2.4.3 http://www.ietf.org/rfc/rfc2396.txt Adds to *excluded-characters*, with a test --- src.lisp | 8 ++++++-- tests.lisp | 10 ++++++++++ 2 files changed, 16 insertions(+), 2 deletions(-) diff --git a/src.lisp b/src.lisp index ab13bad..d39d32b 100644 --- a/src.lisp +++ b/src.lisp @@ -355,10 +355,14 @@ ;; Parsing (defparameter *excluded-characters* - '(;; `delims' (except #\%, because it's handled specially): + (append + (loop for i from 0 to #x1f + collect (code-char i)) + '(;; `delims' (except #\%, because it's handled specially): #\< #\> #\" #\space #\# + ;; `unwise': - #\{ #\} #\| #\\ #\^ #\[ #\] #\`)) + #\{ #\} #\| #\\ #\^ #\[ #\] #\`))) (defun reserved-char-vector (chars &key except) (do* ((a (make-array 127 :element-type 'bit :initial-element 0)) diff --git a/tests.lisp b/tests.lisp index b5cbe37..77d1961 100644 --- a/tests.lisp +++ b/tests.lisp @@ -408,6 +408,16 @@ :condition-type 'uri-parse-error) res) + + ;;an escaped newline isn't rendered properly + (push + `(let ((weird-uri "https://example.com/q?foo%0abar%20baz")) + (test + weird-uri + (puri:render-uri (puri:parse-uri weird-uri) nil) + :test #'string=) + ) res) + `(progn ,@(nreverse res)))) (defun do-tests () -- 1.6.3.3 From 1d30702f6c38582f7b3ddfc00845ab9449b4f9e9 Mon Sep 17 00:00:00 2001 From: Ryan Davis <[email protected]> Date: Fri, 2 Apr 2010 16:55:14 -0400 Subject: [PATCH 2/2] Completed support for http://www.ietf.org/rfc/rfc2396.txt 2.4.3 by adding special case for control code 7F. Had to increase the bit-vector length to 128, and added #\Rubout (#x7F) to *excluded-characters*. Added a docstring and test. --- src.lisp | 8 +++++--- tests.lisp | 20 +++++++++++--------- 2 files changed, 16 insertions(+), 12 deletions(-) diff --git a/src.lisp b/src.lisp index d39d32b..44ec5ea 100644 --- a/src.lisp +++ b/src.lisp @@ -356,16 +356,18 @@ (defparameter *excluded-characters* (append + ;; exclude control characters (loop for i from 0 to #x1f collect (code-char i)) '(;; `delims' (except #\%, because it's handled specially): #\< #\> #\" #\space #\# - + #\Rubout ;; (code-char #x7f) ;; `unwise': - #\{ #\} #\| #\\ #\^ #\[ #\] #\`))) + #\{ #\} #\| #\\ #\^ #\[ #\] #\`)) + "Excluded charcters from RFC2369 (http://www.ietf.org/rfc/rfc2396.txt 2.4.3)") (defun reserved-char-vector (chars &key except) - (do* ((a (make-array 127 :element-type 'bit :initial-element 0)) + (do* ((a (make-array 128 :element-type 'bit :initial-element 0)) (chars chars (cdr chars)) (c (car chars) (car chars))) ((null chars) a) diff --git a/tests.lisp b/tests.lisp index 77d1961..0344922 100644 --- a/tests.lisp +++ b/tests.lisp @@ -408,15 +408,17 @@ :condition-type 'uri-parse-error) res) - - ;;an escaped newline isn't rendered properly - (push - `(let ((weird-uri "https://example.com/q?foo%0abar%20baz")) - (test - weird-uri - (puri:render-uri (puri:parse-uri weird-uri) nil) - :test #'string=) - ) res) + ;;; tests for weird control characters + ;; http://www.ietf.org/rfc/rfc2396.txt 2.4.3 + (dolist (x '("https://example.com/q?foo%0abar%20baz" ;;an escaped newline + "https://example.com/q?%7f" ;; 7f, 127 + )) + (push + `(let ((weird-uri ,x)) + (test weird-uri + (puri:render-uri (puri:parse-uri weird-uri) nil) + :test #'string=) + ) res)) `(progn ,@(nreverse res)))) -- 1.6.3.3