Re: PyYAML multi-line strings

Joe,

PyYAML forces double-quote style on scalars with end-of-line whitespace. 
  I think the rationale was that end-of-line whitespace could not be 
detected visually in block style.

Here is the relevant code:
http://pyyaml.org/browser/pyyaml/trunk/lib/yaml/emitter.py#L761

Thanks,
Kirill

On 11/28/2012 04:26 PM, Joe Abbate wrote:
> Hello,
>
> We (Pyrseas project) are trying to deal with text returned by PostgreSQL
> representing either SQL views or functions (stored procedures).  Until
> now, the text was retrieved from the database into a Python string and
> then passed to yaml.dump.  The output usually ended up as quoted or
> unquoted strings, e.g.,
>
> # function
>      source: SELECT $1 + $1;
> # view
>      definition: SELECT DISTINCT product.product_id,
> product.product_code, product.product_description
>        FROM (warehouse.inventory JOIN product.product USING (product_id));
> # or
>      definition: " SELECT DISTINCT product.product_id,
> product.product_code, product.product_description\n\
>        \   FROM warehouse.inventory\n   JOIN product.product USING
> (product_id);"
>
> The first view was fetched in "raw" format, i.e., as a single line, and
> apparently the line break before the "FROM" is just a fortunate
> coincidence.  The second view was requested "prettified" so the actual
> value in Python looks like:
>
> def = """SELECT DISTINCT product.product_id, product.product_code,
> product.product_description
>     FROM warehouse.inventory
>     JOIN product.product USING (product_id);"""
>
> A contributor suggested using the following to get the output formatted
> properly:
>
> --- test.py ---
> import yaml
>
>
> class MultiLineStr(str):
>      """ Marker for multiline strings"""
>
>
> def MultiLineStr_presenter(dumper, data):
>      return dumper.represent_scalar('tag:yaml.org,2002:str', data, style='|')
> yaml.add_representer(MultiLineStr, MultiLineStr_presenter)
> --- (continues below) ---
>
> The view definitions and function source strings were then changed to be
> instances of MultiLineStr.  This resulted in the following output (using
> the prettified views):
>
> # function
>      source: |-
>        SELECT $1 * $1;
> # view
>      definition: |2-
>         SELECT DISTINCT product.product_id, product.product_code,
> product.product_description
>           FROM warehouse.inventory
>           JOIN product.product USING (product_id);
>
> However, in some cases, PyYAML still outputs the quoted string with
> embedded newlines.  I've created a simple test case, see below.  In the
> second definition, I simply copied the first but removed the embedded
> \n's.  That causes it to be shown with the block scalar indicator,
> albeit as a single line.  That made me believe that the newlines were
> causing the problem, but the third definition outputs correctly even
> though it has newlines.  And I also have a longer text definition (not
> shown) so the text length isn't the cause either.
>
> I would appreciate any help or guidance with the above.  Thanks in advance.
>
> Regards,
>
>
> Joe
>
>
> --- test.py (continued) ---
>
> views = [
>      {'definition': ' SELECT cu.customer_id AS id, (cu.first_name::text
> || \' \'::text) || cu.last_name::text AS name, a.address, a.postal_code
> AS "zip code", a.phone, city.city, country.country, \n        CASE\n
>          WHEN cu.activebool THEN \'active\'::text\n            ELSE
> \'\'::text\n        END AS notes, cu.store_id AS sid\n   FROM customer
> cu\n   JOIN address a ON cu.address_id = a.address_id\n   JOIN city ON
> a.city_id = city.city_id\n   JOIN country ON city.country_id =
> country.country_id;',
>       },
>      {'definition': ' SELECT cu.customer_id AS id, (cu.first_name::text
> || \' \'::text) || cu.last_name::text AS name, a.address, a.postal_code
> AS "zip code", a.phone, city.city, country.country,         CASE
>      WHEN cu.activebool THEN \'active\'::text            ELSE \'\'::text
>         END AS notes, cu.store_id AS sid   FROM customer cu   JOIN
> address a ON cu.address_id = a.address_id   JOIN city ON a.city_id =
> city.city_id   JOIN country ON city.country_id = country.country_id;',
>       },
>      {'definition': """ SELECT e.month, e.day, e.year, e.topic_id,
> t.topic_name
>      FROM event e
>      JOIN topic t USING (topic_id)
>     WHERE e.type = 123;"""
>       }]
>
> for view in views:
>      print(yaml.dump(view, default_flow_style=False))
> for view in views:
>      view['definition'] = MultiLineStr(view['definition'])
>      print(yaml.dump(view, default_flow_style=False))
> --- end ---
>
> ------------------------------------------------------------------------------
> Keep yourself connected to Go Parallel:
> INSIGHTS What's next for parallel hardware, programming and related areas?
> Interviews and blogs by thought leaders keep you ahead of the curve.
> http://goparallel.sourceforge.net
> _______________________________________________
> Yaml-core mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/yaml-core
>


------------------------------------------------------------------------------
Keep yourself connected to Go Parallel: 
VERIFY Test and improve your parallel project with help from experts 
and peers. http://goparallel.sourceforge.net