Re: [PATCH 1/3] Add caching to catpkgsplit function

On 7/6/20 11:03 AM, Zac Medico wrote:
> On 7/6/20 10:30 AM, Chun-Yu Shei wrote:
>> I finally got a chance to try Sid's lru_cache suggestion, and the
>> results were really good.  Simply adding it on catpkgsplit and moving
>> the body of use_reduce into a separate function (that accepts tuples
>> instead of unhashable lists/sets) and decorating it with lru_cache
>> gets a similar 40% overall speedup for the upgrade case I tested.  It
>> seems like even a relatively small cache size (1000 entries) gives
>> quite a speedup, even though in the use_reduce case, the cache size
>> eventually reaches almost 20,000 entries if no limit is set.  With
>> these two changes, adding caching to match_from_list didn't seem to
>> make much/any difference.
> 
> That's great!
> 
>> The catch is that lru_cache is only available in Python 3.2, so would
>> it make sense to add a dummy lru_cache implementation for Python < 3.2
>> that does nothing?  There is also a backports-functools-lru-cache
>> package that's already available in the Portage tree, but that would
>> add an additional external dependency.
>>
>> I agree that refactoring could yield an even bigger gain, but
>> hopefully this can be implemented as an interim solution to speed up
>> the common emerge case of resolving upgrades.  I'm happy to submit new
>> patches for this, if someone can suggest how to best handle the Python
>> < 3.2 case. :)
>>
>> Thanks,
>> Chun-Yu
> 
> We can safely drop support for < Python 3.6 at this point. Alternatively
> we could add a compatibility shim for Python 2.7 that does not perform
> any caching, but I really don't think it's worth the trouble to support
> it any longer.

We've dropped Python 2.7, so now the minimum version is Python 3.6.
-- 
Thanks,
Zac

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQKTBAEBCgB9FiEE8OgXaltWzqgSupCu0HX7jBBKPSAFAl8D7vZfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEYw
RTgxNzZBNUI1NkNFQTgxMkJBOTBBRUQwNzVGQjhDMTA0QTNEMjAACgkQ0HX7jBBK
PSDCuRAAzMuZFFMM4Q3pQka17RstSib44nZ2uKI7Z64rKTtA1nUqzPWZmRd3Jfur
nX4Y1/rb90Zf/huqghmlOu0WILKRpnWfXNSui6wbQ66rzqNIH/ZtIGHsSFwtgBkJ
AB44tZnYXFZoqxZ/+Fk8a4R4FB1gdtNJsRvh4nxv+wxZWX7jpUfb7y7kEqgSVm4m
bu8W9wsIaP5cMIp5DGlHIdkzVcvVCNyRZHQHovSy8GbLuEVYN2/9xd2DEsnVGqio
e+7sJsgsFuxql1toMo0FPewI7iIGHt1uJhzw2vzKhTErotFsjbe2UGZhl2SLpfMj
PhPBgkvyyolVHxRQ8sNa9vLcddruv2LrveX0kG4qypKpgS3TiRy5MCscFsju1cta
pYZoyCeS9gixYtnMVBJtoTgBkzvpxYp5ozEp2s+fxGXAreZRJ5TEGBQBKGBxfhKL
GI0dpLJzUSPhpc9jYNwrC1pB7+ScRHvTxcNv0quedJHLLznrFIazIF67oBHtCB8C
ejkwjXCbdcVjmPSHG26DMBwsN0URvg9lQOS45ewYs5NYet8NpFvtcFNoedHuyKgL
p671HhKJL7dD0gAfDOZd1SnIu66+aiiQJRu5dZl+LKZ+4hlcugfYHSuwiIL3Wv9w
YjqgL3UahE2M7xuuXe+NMhdZfCBLtqjBz4djO9BpnVtuXZMPu9M=
=QClA
-----END PGP SIGNATURE-----