Re: [Tiki-devel] composer.tiki.org slightly borken?

Hi Victor, here is the result of some more drilling down:
http://composer.tiki.org/packages-phpstan.txt
http://composer.tiki.org/packages-phpunit.txt
http://composer.tiki.org/packages-plotly.txt
http://composer.tiki.org/packages-sebastian.txt
http://composer.tiki.org/packages-vimeo.txt
http://composer.tiki.org/packages-aws.txt
http://composer.tiki.org/packages-google.txt
http://composer.tiki.org/packages-mpdf.txt


I don't usually run unit-tests but I can't see why we keep a cache of old
versions of dev tools.

Cheers,
J-M

On Fri, Apr 8, 2022 at 11:28 AM Victor Emanouilov <[email protected]> wrote:

> JM, sorry, one more request. These are several extremely large, packages
> themselves are about 20-30mb but these take 10+GB on disk, can you check
> them as well (drill down)?
>
> 100G	dist/phpstan
> 66G	dist/phpunit
> 30G	dist/plotly
> 29G	dist/sebastian
> 26G	dist/vimeo
> 25G	dist/aws
> 15G	dist/google
> 13G	dist/mpdf
>
> Thanks,
> Victor
> On 4/7/22 7:10 PM, Jean-Marc Libs wrote:
>
> Hi Victor,
>
> It's a huge file so I left it online:
> That's a simple du -sh :
> http://composer.tiki.org/packages_du-sh.txt
> That one is sorted by size:
> http://composer.tiki.org/packages_du-sh_sorted.txt
>
> No problem,
> J-M
>
> On Wed, Apr 6, 2022 at 2:05 PM Victor Emanouilov via TikiWiki-devel <
> [email protected]> wrote:
>
>> JM, can you `du -sh` on the satis storage dirs and send us a list of the
>> directories that take the most amount of space? We might limit semantic
>> versions of those packages to what Tiki really needs...
>>
>> Thanks,
>> Victor
>>
>> On 4/5/22 6:42 PM, Jonny Bradley via TikiWiki-devel wrote:
>> > Hi Victor
>> >
>> > Yes, that was going to be plan B, if just listing what's needed
>> currently in all the branches didn't result in freeing up enough space we
>> could keep just the supported ones (somehow, we don't actually have that
>> info in a format we can access easily afaik).
>> >
>> > The main thing needed is to change from using the actual used version
>> to working out all the possible required ones, i expect we can use whatever
>> composer uses to calculate that but i probably need to know more about how
>> satis working internally...
>> >
>> > And have more free time! ;)
>> >
>> > jb
>> >
>> >
>> >
>> >> On 5 Apr 2022, at 11:44, Victor Emanouilov via TikiWiki-devel <
>> [email protected]> wrote:
>> >>
>> >> Hi Jonny, Jean-Marc,
>> >>
>> >> Thanks for fixing this! It has been a lingering problem for quite some
>> time. We definitely don't need all the packages that are downloaded on
>> composer.tiki.org but Jonny, that's a static repository generator which
>> means it can't fall back to download a missing package. I am thinking why
>> don't we reduce to the number of official Tiki releases we support - we get
>> all semantic versions of packages and generate only those in satis? Symfony
>> packages, for example, will be reduced to v3 or v4 only and not everything
>> from 2 to 6 (https://composer.tiki.org/#symfony/config) for example.
>> Each official Tiki release can have Jonny's improved script run and
>> generate the correct satis config. I believe we should decrease storage
>> requirements several times this way.
>> >>
>> >> Regards,
>> >> Victor
>> >>
>> >> On 4/5/22 12:15 PM, Jonny Bradley via TikiWiki-devel wrote:
>> >>> Hi Jean-Marc and all
>> >>>
>> >>> Had a spare hour last night so started on this:
>> >>>
>> >>>
>> https://gitlab.com/tikiwiki/tiki/-/blob/master/doc/devtools/composer_packages_in_use.php
>> >>>
>> >>> which at the moment lists all the versions currently in use by all
>> the branches in tiki on gitlab.
>> >>>
>> >>> However, it then occurred to me that many of these branches will have
>> outstanding automatic updates pending, e.g. if we ran `composer update` on
>> tiki 22.x there almost certainly would be updates to newer packages, so
>> maybe this isn't the right strategy after all...
>> >>>
>> >>> I guess we need to take into account the semantic versioning in all
>> the composer.json files rather than the actual ones in use in the lock
>> files, but at least it's a start?
>> >>>
>> >>> Would it be possible to remove some packages form satis on
>> composer.t.o, but if something requests one of them it re-downloads it?
>> Then we could be more confident in deleting stuff...
>> >>>
>> >>> Or just put a larger drive on it‽ :p
>> >>>
>> >>> Thanks for fixing it again!
>> >>>
>> >>> jonny
>> >>>
>> >>>
>> >>>
>> >>>> On 5 Apr 2022, at 01:09, Jean-Marc Libs <[email protected]>
>> wrote:
>> >>>>
>> >>>> Hi folks,
>> >>>>
>> >>>> root partition was full on composer.tiki.org 😞 I managed to find
>> 26G and it works again but the volume is staggering (409G, 103G once
>> compressed for backups) and goes up all the time.
>> >>>>
>> >>>> Do we need so many third-party libs?
>> >>>>
>> >>>> Cheers,
>> >>>> J-M
>> >>>>
>> >>>> On Mon, Apr 4, 2022 at 5:17 PM Victor Emanouilov via TikiWiki-devel <
>> [email protected]> wrote:
>> >>>> +1 same for me ... that URL returns 304 not modified - maybe some
>> >>>> apache/nginx config is needed? Composer expects a valid json file
>> there...
>> >>>>
>> >>>> Regards,
>> >>>> Victor
>> >>>>
>> >>>> On 4/4/22 3:03 PM, Jonny Bradley via TikiWiki-devel wrote:
>> >>>>> Hi all
>> >>>>>
>> >>>>> I'm trying to update composer for the bootstrap5 branch following
>> some rebasing and it seem composer is failing now, and it's the same in
>> trunk/master/main now.
>> >>>>>
>> >>>>> When i do:
>> >>>>>
>> >>>>> $ php74 temp/composer.phar update  --working-dir vendor_bundled
>> --prefer-dist --optimize-autoloader
>> >>>>>
>> >>>>> I get:
>> >>>>>
>> >>>>>> In JsonFile.php line 345:
>> >>>>>>
>>                                                           "
>> https://composer.tiki.org/include/all%2419ac295e729a70b8272116cec8bdb0716f3dd8d2.json"
>> does not contain valid JSON
>> >>>>>>     Parse error on line 1:
>> >>>>>>
>> >>>>>>     ^
>> >>>>>>     Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE',
>> '{', '['
>> >>>>>>
>> >>>>>
>> https://composer.tiki.org/include/all%2419ac295e729a70b8272116cec8bdb0716f3dd8d2.json
>> is an empty file, which presumably is the problem.
>> >>>>>
>> >>>>> Does anyone know why? (probably a Jean-Marc thing i expect, maybe
>> satis needs updating/restarting/flushing?)
>> >>>>>
>> >>>>> jonny
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> TikiWiki-devel mailing list
>> >>>>> [email protected]
>> >>>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>> >>>> _______________________________________________
>> >>>> TikiWiki-devel mailing list
>> >>>> [email protected]
>> >>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>> >>>> _______________________________________________
>> >>>> TikiWiki-devel mailing list
>> >>>> [email protected]
>> >>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>> >>>
>> >>> _______________________________________________
>> >>> TikiWiki-devel mailing list
>> >>> [email protected]
>> >>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>> >>
>> >> _______________________________________________
>> >> TikiWiki-devel mailing list
>> >> [email protected]
>> >> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>> >
>> >
>> > _______________________________________________
>> > TikiWiki-devel mailing list
>> > [email protected]
>> > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>>
>>
>> _______________________________________________
>> TikiWiki-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel
>>
>

_______________________________________________
TikiWiki-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel