Re: Encoding in Mason components
James Orr <[email protected]> Fri, 15 Aug 2014 14:22:01 -0400
| Newsgroups | gmane.comp.web.mason.user |
|---|---|
| Message-ID | <CAFWVHUEu61a8TiDk+=7NxVXkCJQQWAvvvfbK4yCd7LFVrL2T3Q@mail.gmail.com> |
--===============5661674545957823869==
Content-Type: multipart/alternative; boundary=20cf301cbea2fae66a0500af1867
--20cf301cbea2fae66a0500af1867
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
To answer my own question ...
after 'process_output' =3D> sub {
my ($self, $outref) =3D @_;
$$outref =3D encode_utf8($$outref) unless $self->{'already_encoded'};
};
before 'send_json' =3D> sub {
my $self =3D shift;
$self->{'already_encoded'} =3D 1;
};
That way we're not doing the same work twice. Anyone see any problems with
this?
On Fri, Aug 15, 2014 at 2:13 PM, James Orr <[email protected]> wrote:
> Another problem I had is that the $m->send_json method calls
> JSON::XS::encode_json, which itself encodes the string as utf8, and this =
is
> then getting double encoded by the after process_output call in the plugi=
n.
>
> I got around it by over-riding send_json and decoding the json string ...
>
> around 'send_json' =3D> sub {
> my $orig =3D shift;
> my $self =3D shift;
> my $data =3D shift;
> $self->clear_buffer;
> my $result =3D JSON::XS::encode_json($data);
> $self->print(decode_utf8(JSON::XS::encode_json($data)));
> $self->res->content_type("application/json; charset=3Dutf-8");
> $self->abort();
> };
>
> Is there a better way to handle this?
>
>
> On Thu, Aug 14, 2014 at 10:29 AM, James Orr <[email protected]> wrote:
>
>> I had some utf8 issues as well, I used the plugin at the stackoverflow
>> link which worked for the most part, except any parameters passed in to =
a
>> page were getting double encoded.
>>
>> I made this change to the around 'run' in Mason::Plugin::UTF8::Request
>> which seems to fix the problem ...
>>
>> around 'run' =3D> sub {
>> my $orig =3D shift;
>> my $self =3D shift;
>>
>> my %params =3D @_;
>> while (my($key, $value) =3D each(%params)) {
>> while (my ($k, $v) =3D each(%$value)) {
>> $params{$key}->{$k} =3D decode_utf8($v);
>> }
>> }
>> $self->$orig(%params);
>> };
>>
>> On Wed, Aug 13, 2014 at 8:19 AM, Jonathan Swartz <[email protected]>
>> wrote:
>>
>>> Ladislav - you can find some starting points here =E2=80=94
>>>
>>> http://stackoverflow.com/questions/5858596/how-to-make-mason2-utf-8-cle=
an
>>>
>>> https://www.mail-archive.com/[email protected]/msg03450=
.html
>>>
>>> There ought to be a Mason::Plugin::Utf8 that handles this, but I never
>>> had the time and it appears no one else did either=E2=80=A6 :)
>>>
>>> Best
>>> Jon
>>>
>>> On Aug 13, 2014, at 5:13 AM, Adam Sj=C3=B8gren <[email protected]> wrot=
e:
>>>
>>> Ladislav Laska <[email protected]> writes:
>>>
>>> So my question to the community is: what is the right way to do it?
>>>
>>>
>>> In the old HTML::Mason (1) days, I would put 'use utf8;' in the preambl=
e
>>> parameter:
>>>
>>> *
>>> https://metacpan.org/pod/distribution/HTML-Mason/lib/HTML/Mason/Params.=
pod#preamble
>>>
>>>
>>> I don't know if Mason (2) has changed anything in this regard.
>>>
>>>
>>> Best regards,
>>>
>>> Adam
>>>
>>> --
>>> "Repetition is the death of magic." Adam Sj=C3=
=B8gren
>>>
>>> [email protected]
>>>
>>>
>>>
>>> -----------------------------------------------------------------------=
-------
>>> _______________________________________________
>>> Mason-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/mason-users
>>>
>>>
>>>
>>>
>>> -----------------------------------------------------------------------=
-------
>>>
>>> _______________________________________________
>>> Mason-users mailing list
>>> [email protected]
>>> https://lists.sourceforge.net/lists/listinfo/mason-users
>>>
>>>
>>
>
--20cf301cbea2fae66a0500af1867
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">To answer my own question ...<div><br></div><div><div>afte=
r 'process_output' =3D> sub {</div><div><span class=3D"" style=
=3D"white-space:pre"> </span>my ($self, $outref) =3D @_;</div><div><span cl=
ass=3D"" style=3D"white-space:pre"> </span>$$outref =3D encode_utf8($$outre=
f) unless $self->{'already_encoded'};</div>
<div>};</div><div><br></div><div>before 'send_json' =3D> sub {</=
div><div><span class=3D"" style=3D"white-space:pre"> </span>my $self =3D sh=
ift;</div><div><span class=3D"" style=3D"white-space:pre"> </span>$self->=
;{'already_encoded'} =3D 1;</div>
<div>};</div></div><div><br></div><div>That way we're not doing the sam=
e work twice. =C2=A0Anyone see any problems with this?</div></div><div clas=
s=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Fri, Aug 15, 2014 a=
t 2:13 PM, James Orr <span dir=3D"ltr"><<a href=3D"mailto:james.orr7@gma=
il.com" target=3D"_blank">[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Another problem I had is th=
at the $m->send_json method calls JSON::XS::encode_json, which itself en=
codes the string as utf8, and this is then getting double encoded by the af=
ter process_output call in the plugin.<div>
<br></div><div>I got around it by over-riding send_json and decoding the js=
on string ...</div><div><br></div><div><div>around 'send_json' =3D&=
gt; sub {</div><div class=3D""><div><span style=3D"white-space:pre-wrap"> <=
/span>my $orig =3D shift;</div>
<div><span style=3D"white-space:pre-wrap"> </span>my $self =3D shift;</div>=
</div><div><span style=3D"white-space:pre-wrap"> </span>my $data =3D shift;=
</div><div><span style=3D"white-space:pre-wrap"> </span>$self->clear_buf=
fer;</div>
<div><span style=3D"white-space:pre-wrap"> </span>my $result =3D JSON::XS::=
encode_json($data);</div><div><span style=3D"white-space:pre-wrap"> </span>=
$self->print(decode_utf8(JSON::XS::encode_json($data)));</div><div>
<span style=3D"white-space:pre-wrap"> </span>$self->res->content_type=
("application/json; charset=3Dutf-8");</div><div><span style=3D"w=
hite-space:pre-wrap"> </span>$self->abort();</div><div>};</div></div>
<div><br></div><div>Is there a better way to handle this?</div></div><div c=
lass=3D"HOEnZb"><div class=3D"h5"><div class=3D"gmail_extra"><br><br><div c=
lass=3D"gmail_quote">On Thu, Aug 14, 2014 at 10:29 AM, James Orr <span dir=
=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_blank">jame=
[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">I had some utf8 issues as w=
ell, I used the plugin at the stackoverflow link which worked for the most =
part, except any parameters passed in to a page were getting double encoded=
.<div>
<br></div><div>I made this change to the around 'run' in Mason::Plu=
gin::UTF8::Request which seems to fix the problem ...</div>
<div><br></div><div><div class=3D"gmail_extra"><div class=3D"gmail_extra">a=
round 'run' =3D> sub {</div><div class=3D"gmail_extra"><span sty=
le=3D"white-space:pre-wrap"> </span>my $orig =3D shift;</div><div class=3D"=
gmail_extra">
<span style=3D"white-space:pre-wrap"> </span>my $self =3D shift;</div><div =
class=3D"gmail_extra"><br></div><div class=3D"gmail_extra"><span style=3D"w=
hite-space:pre-wrap"> </span>my %params =3D @_;</div><div class=3D"gmail_ex=
tra">
<span style=3D"white-space:pre-wrap"> </span>while (my($key, $value) =3D ea=
ch(%params)) {</div><div class=3D"gmail_extra"><span style=3D"white-space:p=
re-wrap"> </span>while (my ($k, $v) =3D each(%$value)) {</div><div class=
=3D"gmail_extra">
<span style=3D"white-space:pre-wrap"> </span>$params{$key}->{$k} =3D d=
ecode_utf8($v);</div><div class=3D"gmail_extra"><span style=3D"white-space:=
pre-wrap"> </span>}</div><div class=3D"gmail_extra"><span style=3D"white-s=
pace:pre-wrap"> </span>}</div>
<div class=3D"gmail_extra"><span style=3D"white-space:pre-wrap"> </span>$se=
lf->$orig(%params);</div><div class=3D"gmail_extra">};</div><div><div><b=
r><div class=3D"gmail_quote">On Wed, Aug 13, 2014 at 8:19 AM, Jonathan Swar=
tz <span dir=3D"ltr"><<a href=3D"mailto:[email protected]" target=3D"_bla=
nk">[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div style=3D"word-wrap:break-word"><div>Ladislav - you ca=
n find some starting points here =E2=80=94</div>
<div><br></div><div><a href=3D"http://stackoverflow.com/questions/5858596/h=
ow-to-make-mason2-utf-8-clean" target=3D"_blank">http://stackoverflow.com/q=
uestions/5858596/how-to-make-mason2-utf-8-clean</a></div><div><a href=3D"ht=
tps://www.mail-archive.com/[email protected]/msg03450.html"=
target=3D"_blank">https://www.mail-archive.com/[email protected]=
ge.net/msg03450.html</a></div>
<div><br></div><div>There ought to be a Mason::Plugin::Utf8 that handles th=
is, but I never had the time and it appears no one else did either=E2=80=A6=
:)</div><div><br></div><div>Best</div><div>Jon</div><div><div><br><div>
<div>On Aug 13, 2014, at 5:13 AM, Adam Sj=C3=B8gren <<a href=3D"mailto:a=
[email protected]" target=3D"_blank">[email protected]</a>> wrote:</div><=
br><blockquote type=3D"cite">Ladislav Laska <<a href=3D"mailto:laska@kam=
.mff.cuni.cz" target=3D"_blank">[email protected]</a>> writes:<br>
<br><blockquote type=3D"cite">So my question to the community is: what is t=
he right way to do it?<br></blockquote><br>In the old HTML::Mason (1) days,=
I would put 'use utf8;' in the preamble<br>parameter:<br><br> * <a=
href=3D"https://metacpan.org/pod/distribution/HTML-Mason/lib/HTML/Mason/Pa=
rams.pod#preamble" target=3D"_blank">https://metacpan.org/pod/distribution/=
HTML-Mason/lib/HTML/Mason/Params.pod#preamble</a><br>
<br><br>I don't know if Mason (2) has changed anything in this regard.<=
br><br><br> =C2=A0Best regards,<br><br> =C2=A0=C2=A0=C2=A0Adam<br><br>-- <b=
r> "Repetition is the death of magic." =C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0Adam Sj=C3=B8gren<br>
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0<a href=3D"mailto:[email protected]"=
target=3D"_blank">[email protected]</a><br>
<br><br>-------------------------------------------------------------------=
-----------<br>_______________________________________________<br>Mason-use=
rs mailing list<br><a href=3D"mailto:[email protected]" tar=
get=3D"_blank">[email protected]</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/mason-users" target=
=3D"_blank">https://lists.sourceforge.net/lists/listinfo/mason-users</a><br=
></blockquote></div><br></div></div></div><br>-----------------------------=
-------------------------------------------------<br>
<br>_______________________________________________<br>
Mason-users mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">Maso=
[email protected]</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/mason-users" target=
=3D"_blank">https://lists.sourceforge.net/lists/listinfo/mason-users</a><br=
>
<br></blockquote></div><br></div></div></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
--20cf301cbea2fae66a0500af1867--
--===============5661674545957823869==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
--===============5661674545957823869==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
Mason-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mason-users
--===============5661674545957823869==--