Re: [viewvc-dev] ViewVC 1.1.13 viewvc.get_result tricked by unusually long patch message

------=_Part_5462_1020013737.1333135659375
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable

On 03/30/2012 03:00 PM, Robert C. Kahlert wrote:
> Hi,
>=20
> this may or may not be the right list to report this problem to, so
> apologies if it is not.

You've hit the right list!

> We are running into a situation with ViewVC 1.1.13 where the get_result()
> method in viewvc.py (line 1210) is being bitten by an update with a very
> long VC log message (~250K).
>=20
> Specifically, the maxlen parameter does not seem to be passed down to the
> self._tokenize_text() method, which is going to town on the 250K message,
> calling the search method millions of times, for emitting a few hundred
> characters in the summary view.

[...]

> Two ideas come to mind:
>=20
> (a) pre-truncating the message (e.g. to 10 x the max len intended, to pic=
k a
> heuristic)
> (b) caching the tokenization summary, since patches usually affect multip=
le
> files
>=20
> Either solution would fix our situation.

The pre-truncation approach would (obviously) be easier to implement.

Caching the tokenization results is a pretty good idea, too, but obviously
more complicated.  What to use as the cache key?  For Subversion, you could
use the revision number.  Not so for CVS.  So ... perhaps a SHA1 checksum o=
f
the log message?

Of course, even if the same log message *doesn't* appear on multiple files,
I realize now that ViewVC will still process the log message twice per file
-- once without the maxlen set, once with.  Since no caller ever wants
*only* the truncated log message, it seems that at a minimum I could make
format_log() return a 2-tuple containing the full log message and the
shortened form (and sometimes they'd be exactly the same string).  If more
advanced caching was added, it could be added around the results of
format_log().

-- C-Mike

PS:  How did you perform your profiling?  That's a trick I've only done wit=
h
C programs ... haven't played with it in Python yet.

--=20
C. Michael Pilato <[email protected]>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

------------------------------------------------------
http://viewvc.tigris.org/ds/viewMessage.do?dsForumId=3D4251&dsMessageId=3D2=
942587

To unsubscribe from this discussion, e-mail: [[email protected]=
.org].
------=_Part_5462_1020013737.1333135659375
Content-Type: application/pgp-signature; name=signature.asc
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=signature.asc
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEARECAAYFAk92CSUACgkQokEGqRcG/W7YXACgituGVZ81hRFpt9R6PFYArVe+
nCIAnif+zRXTUZZP5gg2v4y19XAyB6jZ
=k6Fm
-----END PGP SIGNATURE-----

------=_Part_5462_1020013737.1333135659375--