Re: [Tiki-devel] composer.tiki.org slightly borken?
Victor Emanouilov via TikiWiki-devel <[email protected]>
| Newsgroups | gmane.comp.cms.tiki.devel |
|---|---|
| Message-ID | <[email protected]> |
JM, sorry, one more request. These are several extremely large, packages themselves are about 20-30mb but these take 10+GB on disk, can you check them as well (drill down)? 100G dist/phpstan 66G dist/phpunit 30G dist/plotly 29G dist/sebastian 26G dist/vimeo 25G dist/aws 15G dist/google 13G dist/mpdf Thanks, Victor On 4/7/22 7:10 PM, Jean-Marc Libs wrote: > Hi Victor, > > It's a huge file so I left it online: > That's a simple du -sh : > http://composer.tiki.org/packages_du-sh.txt > That one is sorted by size: > http://composer.tiki.org/packages_du-sh_sorted.txt > > No problem, > J-M > > On Wed, Apr 6, 2022 at 2:05 PM Victor Emanouilov via TikiWiki-devel > <[email protected]> wrote: > > JM, can you `du -sh` on the satis storage dirs and send us a list > of the > directories that take the most amount of space? We might limit > semantic > versions of those packages to what Tiki really needs... > > Thanks, > Victor > > On 4/5/22 6:42 PM, Jonny Bradley via TikiWiki-devel wrote: > > Hi Victor > > > > Yes, that was going to be plan B, if just listing what's needed > currently in all the branches didn't result in freeing up enough > space we could keep just the supported ones (somehow, we don't > actually have that info in a format we can access easily afaik). > > > > The main thing needed is to change from using the actual used > version to working out all the possible required ones, i expect we > can use whatever composer uses to calculate that but i probably > need to know more about how satis working internally... > > > > And have more free time! ;) > > > > jb > > > > > > > >> On 5 Apr 2022, at 11:44, Victor Emanouilov via TikiWiki-devel > <[email protected]> wrote: > >> > >> Hi Jonny, Jean-Marc, > >> > >> Thanks for fixing this! It has been a lingering problem for > quite some time. We definitely don't need all the packages that > are downloaded on composer.tiki.org <http://composer.tiki.org> but > Jonny, that's a static repository generator which means it can't > fall back to download a missing package. I am thinking why don't > we reduce to the number of official Tiki releases we support - we > get all semantic versions of packages and generate only those in > satis? Symfony packages, for example, will be reduced to v3 or v4 > only and not everything from 2 to 6 > (https://composer.tiki.org/#symfony/config) for example. Each > official Tiki release can have Jonny's improved script run and > generate the correct satis config. I believe we should decrease > storage requirements several times this way. > >> > >> Regards, > >> Victor > >> > >> On 4/5/22 12:15 PM, Jonny Bradley via TikiWiki-devel wrote: > >>> Hi Jean-Marc and all > >>> > >>> Had a spare hour last night so started on this: > >>> > >>> > https://gitlab.com/tikiwiki/tiki/-/blob/master/doc/devtools/composer_packages_in_use.php > >>> > >>> which at the moment lists all the versions currently in use by > all the branches in tiki on gitlab. > >>> > >>> However, it then occurred to me that many of these branches > will have outstanding automatic updates pending, e.g. if we ran > `composer update` on tiki 22.x there almost certainly would be > updates to newer packages, so maybe this isn't the right strategy > after all... > >>> > >>> I guess we need to take into account the semantic versioning > in all the composer.json files rather than the actual ones in use > in the lock files, but at least it's a start? > >>> > >>> Would it be possible to remove some packages form satis on > composer.t.o, but if something requests one of them it > re-downloads it? Then we could be more confident in deleting stuff... > >>> > >>> Or just put a larger drive on it‽ :p > >>> > >>> Thanks for fixing it again! > >>> > >>> jonny > >>> > >>> > >>> > >>>> On 5 Apr 2022, at 01:09, Jean-Marc Libs > <[email protected]> wrote: > >>>> > >>>> Hi folks, > >>>> > >>>> root partition was full on composer.tiki.org > <http://composer.tiki.org> 😞 I managed to find 26G and it works > again but the volume is staggering (409G, 103G once compressed for > backups) and goes up all the time. > >>>> > >>>> Do we need so many third-party libs? > >>>> > >>>> Cheers, > >>>> J-M > >>>> > >>>> On Mon, Apr 4, 2022 at 5:17 PM Victor Emanouilov via > TikiWiki-devel <[email protected]> wrote: > >>>> +1 same for me ... that URL returns 304 not modified - maybe some > >>>> apache/nginx config is needed? Composer expects a valid json > file there... > >>>> > >>>> Regards, > >>>> Victor > >>>> > >>>> On 4/4/22 3:03 PM, Jonny Bradley via TikiWiki-devel wrote: > >>>>> Hi all > >>>>> > >>>>> I'm trying to update composer for the bootstrap5 branch > following some rebasing and it seem composer is failing now, and > it's the same in trunk/master/main now. > >>>>> > >>>>> When i do: > >>>>> > >>>>> $ php74 temp/composer.phar update --working-dir > vendor_bundled --prefer-dist --optimize-autoloader > >>>>> > >>>>> I get: > >>>>> > >>>>>> In JsonFile.php line 345: > >>>>>> > "https://composer.tiki.org/include/all%2419ac295e729a70b8272116cec8bdb0716f3dd8d2.json" > does not contain valid JSON > >>>>>> Parse error on line 1: > >>>>>> > >>>>>> ^ > >>>>>> Expected one of: 'STRING', 'NUMBER', 'NULL', 'TRUE', > 'FALSE', '{', '[' > >>>>>> > >>>>> > https://composer.tiki.org/include/all%2419ac295e729a70b8272116cec8bdb0716f3dd8d2.json > is an empty file, which presumably is the problem. > >>>>> > >>>>> Does anyone know why? (probably a Jean-Marc thing i expect, > maybe satis needs updating/restarting/flushing?) > >>>>> > >>>>> jonny > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> _______________________________________________ > >>>>> TikiWiki-devel mailing list > >>>>> [email protected] > >>>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > >>>> _______________________________________________ > >>>> TikiWiki-devel mailing list > >>>> [email protected] > >>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > >>>> _______________________________________________ > >>>> TikiWiki-devel mailing list > >>>> [email protected] > >>>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > >>> > >>> _______________________________________________ > >>> TikiWiki-devel mailing list > >>> [email protected] > >>> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > >> > >> _______________________________________________ > >> TikiWiki-devel mailing list > >> [email protected] > >> https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > > > > > > _______________________________________________ > > TikiWiki-devel mailing list > > [email protected] > > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > > > _______________________________________________ > TikiWiki-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel > _______________________________________________ TikiWiki-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/tikiwiki-devel