Some update - I can compile quite a lot of KDE now, up to KDE konsole 17.08.1 at the least; and I should be able to auto-generate valid GoboLinux recipes (but this part is not yet finished)
Robert Alexander <[email protected]> Tue, 19 Sep 2017 21:26:51 +0200
| Newsgroups | gmane.linux.distributions.gobo.general |
|---|---|
| Message-ID | <CAFie_ADvqibB6symbsvKfTQKYDjpn28Y9indnKRG4KpiqPwxbw@mail.gmail.com> |
--===============1981902342944925311== Content-Type: multipart/alternative; boundary="f403045c18845671ce05598fd73f" --f403045c18845671ce05598fd73f Content-Type: text/plain; charset="UTF-8" Hello GoboFolks, Sorry for this very long mail. The line breaks are at or ~before 80 characters mostly because I type this in my editor. This is a ... semi-follow up on my earlier KDE-related mail/question from some time ago. I've actually looked at LFS/BLFS and am tracking quite a lot of KDE stuff now in my two major ruby gems "cookbooks" and "rbt". (Lucas also pointed towards UpdateKdeRecipe but I have not yet had a look at it. I'll have a look once I am able to keep up with KDE-releases; right now, I am not easily able to but I think this may change in the coming weeks or so.) As this mail is quite long, I will attempt to focus on only two things that may be somewhat interesting to other people perhaps: (1) The LFS/BLFS team actually did a pretty good job at tracking KDE-related stuff and dependencies. In particular, if you want to have a go at it too, here are two very useful websites IMO: http://www.linuxfromscratch.org/blfs/view/cvs/kde/krameworks5.html http://www.linuxfromscratch.org/blfs/view/cvs/kde/plasma-all.html There are further links there, within the BLFS subsite which can be used to track dependencies and stuff, providing instructions how to compile something. The biggest part is qt - if you manage to compile qt, then the rest is not too difficult. (On a side note, qt - 5.8.x did not compile for me but qt - 5.9.1 did... upon investigating my error, I found out that the error was fixed in 5.9.1 but not in the 5.8.x branch, makes you wonder a bit what the qt team considers "stable" when compilation does not work, despite fixes being already integrated into the "unstable" branch....) In my main ruby gem there at, https://rubygems.org/gems/cookbooks , I am currently tracking 3084 programs. This includes most of the KDE stack, excluding the kde plasma stack right now of which the kde devs are very proud of (as a matter of fact, though, I am currently adding the plasma-related stack to the cookbooks gem and also compiling them once added). Not sure if I will manage to finish adding the rest of plasma this week/weekend, and what else is missing in the kde stack. A LOT of stuff is missing in the cookbooks gem and the whole gem has to be polished a lot more before it is really usable for other people; but I use that gem since quite some time by now and it's quite useful in general. One idea is to eventually be able to autogenerate "all the things", at the least the common formats such as debian, rpm and so forth - a similar ruby gem exists here at https://github.com/jordansissel/fpm. GoboLinux recipes also have to be autogenerated of course. On a brighter note, a few hours ago, I managed to compile konsole: https://download.kde.org/stable/applications/17.08.1/src/konsole-17.08.1.tar.xz from source - and it is actually working. :) (There is some strange bug which I reported to the KDE team, where I can not pick another profile without KDE konsole crashing... but anyway, if I don't fiddle with the profile, konsole 17.08.1 works fine.) (2) More relevant to GoboLinux, I am actually able to autogenerate (non-conforming) GoboLinux recipes. So I could autogenerate 3084 GoboLinux recipes in theory. :) None of them are tested so I don't recommend anyone else to test - I'll have a look at their validity lateron, but I first want to write some scripts that check for problems etc... and help auto-correct stuff too. I had a dry-run a few days ago, without the whole kde KF5 stack as of yet; the upload archive can be seen here: http://shevegen.square7.ch/gobolinux-recipes-16.09.2017.tar.xz The nice thing is that the archive is automatically generated and automatically uploaded, too. So when I am able to generate valid GoboLinux recipes, I should be able to autogenerate all the recipes registered there. Unfortunately, the big caveat here is that ... a lot of things are currently not working in the autogeneration part, in particular when it comes to shell logic. It's not trivial to auto-generate valid shell code. But the dependency chain is also empty (I do not track the dependencies completely yet). What does, somewhat work, is autogenerating files such as "Recipe" or the file "Resources/Description". I think "Recipe" is the most important one probably, since it has the url: entry, which Compile probably uses to download remote programs. I know that in my own two gems, when you have the remote url, you can infer all the other information; and you can even automatically detect the configure type usually - the GoboLinux scripts try this already to some extent if I recall correctly. Not sure if Gobolinux supports scons, waf and meson though. In particular gnome moved towards meson. At a later time, I intend to expand the code that autogenerates stuff to not only generate valid GoboLinux Recipes, but to also automatically upload all the recipes into extracted, remote archives at that FTP site (or perhaps some other site, I'll have a closer look once the autogeneration part works). When that part works, then in theory, people could navigate to the respective URL and download a given recipe from there. But the *really* nice part would be that this could be done by other people too, so there could be countless "GoboLinux secondary recipe stores". :) (The cookbooks gem currently has a few other limitations though, so it is not really useful in any collaborative environment; I'll have a look at this again at a later point. In principle people should be able to update the official cookbooks gem too, but for now I'll handle things.) But again, this is all not working that well ... since I am also somewhat slow, I'll try to have things work better before ... the end of this year. There is also usually a new ruby release at around x-mas so I'll aim for a bit after x-mas too with improvements there, in particular being able to autogenerate VALID and COMPLETE GoboLinux recipes.. :D Some parts will be a bit difficult to autogenerate, such as the shell logic, due to two reasons - one, my shell foo is still bad, after all these years, even though the GoboLinux scripts are surprisingly clean compared to other shell scripts. (I remember looking at GNU sourcemage many years ago, and their shell code was a lot uglier than the GoboLinux shell scripts.) Two, and probably the bigger reason, is that I am not entirely sure what is all needed and when in the various GoboLinux recipes. I think that the recipes currently require quite a lot of expert domain knowledge. Joel Roth did a great job at the wiki but I think a lot is still missing. Like more details. But anyway. Some recipes do not need any modifications at all, but I looked at other ones such as e. g. dbus and qt and they need quite a few modifications here or there as can be seen: http://recipes.gobolinux.org/r/?list=DBus&ver=1.10.8-r3 http://recipes.gobolinux.org/r/?list=Qt&ver=5.9.0-r2&file=Recipe I do not yet know what the various entries mean (environment, install_variables, pre_link) and although I understand shell code to some extent, I don't have a great idea how to autogenerate all of that ... but I think that in theory, once I understand these parts fully, I should be able to autogenerate most of them. I tested this first on slackware though, since my specific GoboLinux knowledge is currently still too limited, but I will test it on GoboLinux once I have finished with the whole kde stack in the cookbooks; in the past, I did not even manage to compile qt/kde at all, which was really frustrating. I also managed to compile all of mate-desktop from source, though there also was some problem with the settings; but mate-terminal works and mate-desktop starts up fine too. Note that with autogenerated GoboLinux recipes, this is not aimed at all to bypass the manual review process in any way whatsoever. But it is supposed to help speed up some things. I can probably also add a linter to check for conforming recipes too. I am also using some other scripts to help track updates that happen to source code repositories in general, such as via the blfs changelog page at http://www.linuxfromscratch.org/blfs/view/cvs/introduction/changelog.html. While there is an initial time investment in adding new entries to the cookbooks project, and making sure that the code logic works - once the entry is registered, updating is very easy for me. One script called UpdateEntry accepts an URL and will update the local program + download. (In theory it can also be used to automatically compile, but I usually first update, before then invoking the compile process via the ruby scripts.) Another script also autogenerates individual .html pages that I could upload too. And I could probably combine this with existing GoboLinux recipes... a bit similar to how the BLFS homepage works too. But anyway, I think I am writing too much and having too little action ;) - the archive at gobolinux-recipes-16.09.2017.tar.xz shows that it works at the least in principle, as demo. The descriptions I use from the cookbooks gem, but I can also write a script that uses the descriptions found in the gobolinux recipes (the github page allows one to easily download the recipes... there are quite a LOT of recipes there, some of the recipes are many years old, with URLs that no longer exist hehe) I also have an auto-update script for the gnome stack although it is a bit buggy - the URL that I am checking for gnome stuff is http://ftp.gnome.org/pub/gnome/sources/?C=M;O=D Meson is currently annoying me here, so I am waiting until the LFS/BFLS team shows how to use meson properly. :D Here is a simple program version dump of projects that I am currently tracking, though I have to update it with kde stuff yet (it is updated whenever I push a new release to the cookbooks gem but I broke it a bit recently, so I have to update manually at a later time): http://shevegen.square7.ch/programs_version.html Anyway, I wanted to give a bit more feedback as to what I am doing. My idea is to use the "secondary gobostore", and from that one, once it all works on my gobolinux machine, push updates to the recipe store once it all works. I'll try to use a weekend that is not so busy to batch-test this via GoboLinux but I will also first test that the recipes work on GoboLinux individually, which will take some time. Cheers, Robert --f403045c18845671ce05598fd73f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div>Hello GoboFolks,<br><br>Sorry for this very long mail= . The line breaks are at or ~before 80 characters<br></div>mostly because I= type this in my editor.<br><div><br>This is a ... semi-follow up on my ear= lier KDE-related mail/question from some<br>time ago.<br><br>I've actua= lly looked at LFS/BLFS and am tracking quite a lot of KDE<br>stuff now in m= y two major ruby gems "cookbooks" and "rbt". (Lucas<br>= also pointed towards UpdateKdeRecipe but I have not yet had a look<br>at it= . I'll have a look once I am able to keep up with KDE-releases; right<b= r>now, I am not easily able to but I think this may change in the coming<br= >weeks or so.)<br><br>As this mail is quite long, I will attempt to focus o= n only two things that<br>may be somewhat interesting to other people perha= ps:<br><br>(1) The LFS/BLFS team actually did a pretty good job at tracking= <br>KDE-related stuff and dependencies. In particular, if you want to<br>ha= ve a go at it too, here are two very useful websites IMO:<br><br>=C2=A0 <a = href=3D"http://www.linuxfromscratch.org/blfs/view/cvs/kde/krameworks5.html"= >http://www.linuxfromscratch.org/blfs/view/cvs/kde/krameworks5.html</a><br>= =C2=A0 <a href=3D"http://www.linuxfromscratch.org/blfs/view/cvs/kde/plasma-= all.html">http://www.linuxfromscratch.org/blfs/view/cvs/kde/plasma-all.html= </a><br><br>There are further links there, within the BLFS subsite which ca= n be<br>used to track dependencies and stuff, providing instructions how to= <br>compile something. The biggest part is qt - if you manage to compile<br= >qt, then the rest is not too difficult. (On a side note, qt - 5.8.x did no= t<br></div><div>compile for me but qt - 5.9.1 did... upon investigating my = error, I<br></div><div>found out that the error was fixed in 5.9.1 but not = in the 5.8.x branch,<br></div><div>makes you wonder a bit what the qt team = considers "stable" when <br></div><div>compilation does not work,= despite fixes being already integrated <br></div><div>into the "unsta= ble" branch....)<br></div><div><br>In my main ruby gem there at, <a hr= ef=3D"https://rubygems.org/gems/cookbooks">https://rubygems.org/gems/cookbo= oks</a> ,<br>I am currently tracking 3084 programs. This includes most of t= he<br>KDE stack, excluding the kde plasma stack right now of which the kde<= br>devs are very proud of (as a matter of fact, though, I am currently addi= ng<br>the plasma-related stack to the cookbooks gem and also compiling<br>t= hem once added).<br><br>Not sure if I will manage to finish adding the rest= of plasma this<br>week/weekend, and what else is missing in the kde stack.= A LOT of stuff<br>is missing in the cookbooks gem and the whole gem has t= o be polished a<br>lot more before it is really usable for other people; bu= t I use that gem since<br>quite some time by now and it's quite useful = in general. One idea is to<br>eventually be able to=C2=A0 autogenerate &quo= t;all the things", at the least the common<br>formats=C2=A0 such as de= bian, rpm and so forth - a similar ruby gem exists here<br>at <a href=3D"ht= tps://github.com/jordansissel/fpm">https://github.com/jordansissel/fpm</a>.= GoboLinux recipes also have to<br></div><div>be autogenerated of course.<b= r></div><div><br>On a brighter note, a few hours ago, I managed to compile = konsole:<br><br>=C2=A0 <a href=3D"https://download.kde.org/stable/applicati= ons/17.08.1/src/konsole-17.08.1.tar.xz">https://download.kde.org/stable/app= lications/17.08.1/src/konsole-17.08.1.tar.xz</a><br><br>from source - and i= t is actually working. :)<br><br>(There is some strange bug which I reporte= d to the KDE team, where I can <br>not pick another profile without KDE kon= sole crashing... but anyway,<br>if I don't fiddle with the profile, kon= sole 17.08.1 works fine.)<br><br>(2) More relevant to GoboLinux, I am actua= lly able to autogenerate<br>(non-conforming) GoboLinux recipes. So I could = autogenerate 3084<br>GoboLinux recipes in theory. :)<br><br></div><div>None= of them are tested so I don't recommend anyone else to test -<br></div= ><div>I'll have a look at their validity lateron, but I first want to w= rite some<br></div><div>scripts that check for problems etc... and help aut= o-correct stuff <br></div><div>too.<br></div><div><br>I had a dry-run a few= days ago, without the whole kde KF5 stack as<br>of yet; the upload archive= can be seen here:<br><br>=C2=A0 <a href=3D"http://shevegen.square7.ch/gobo= linux-recipes-16.09.2017.tar.xz">http://shevegen.square7.ch/gobolinux-recip= es-16.09.2017.tar.xz</a><br><br>The nice thing is that the archive is autom= atically generated<br>and automatically uploaded, too. So when I am able to= generate <br></div><div>valid GoboLinux recipes, I should be able to autog= enerate all<br></div><div>the recipes registered there.<br></div><div><br>U= nfortunately, the big caveat here is that ... a lot of things are<br>curren= tly not working in the autogeneration part, in particular when<br>it comes = to shell logic. It's not trivial to auto-generate valid shell<br>code.<= br><br>But the dependency chain is also empty (I do not track the dependenc= ies<br>completely yet). What does, somewhat work, is autogenerating<br>file= s such as "Recipe" or the file "Resources/Description".= I think<br>"Recipe" is the most important one probably, since it= has the url:<br>entry, which Compile probably uses to download remote prog= rams.<br>I know that in my own two gems, when you have the remote url, you<= br>can infer all the other information; and you can even automatically dete= ct<br>the configure type usually - the GoboLinux scripts try this already t= o some<br>extent if I recall correctly. Not sure if Gobolinux supports scon= s,<br>waf and meson though. In particular gnome moved towards meson.<br><br= >At a later time, I intend to expand the code that autogenerates stuff<br>t= o not only generate valid GoboLinux Recipes, but to also automatically<br>u= pload all the recipes into extracted, remote archives at that FTP<br>site (= or perhaps some other site, I'll have a closer look once the autogenera= tion<br>part works). When that part works, then in theory, people could nav= igate<br>to the respective URL and download a given recipe from there. But = the<br>*really* nice part would be that this could be done by other people = too,<br>so there could be countless "GoboLinux secondary recipe stores= ". :)<br><br></div><div>(The cookbooks gem currently has a few other l= imitations though, so<br></div><div>it is not really useful in any collabor= ative environment; I'll have a look<br></div><div>at this again at a la= ter point. In principle people should be able to <br></div><div>update the = official cookbooks gem too, but for now I'll handle things.)<br></div><= div><br>But again, this is all not working that well ... since I am also so= mewhat<br>slow, I'll try to have things work better before ... the end = of this year.<br><br>There is also usually a new ruby release at around x-m= as so I'll aim for <br>a bit after x-mas too with improvements there, i= n particular being able<br>to autogenerate VALID and COMPLETE GoboLinux rec= ipes.. :D<br><br>Some parts will be a bit difficult to autogenerate, such a= s the shell<br>logic, due to two reasons - one, my shell foo is still bad, = after all<br>these years, even though the GoboLinux scripts are surprisingl= y clean<br>compared to other shell scripts. (I remember looking at GNU sour= cemage<br>many years ago, and their shell code was a lot uglier than the Go= boLinux<br>shell scripts.)<br><br>Two, and probably the bigger reason, is t= hat I am not entirely sure what<br>is all needed and when in the various Go= boLinux recipes. I think that<br>the recipes currently require quite a lot = of expert domain knowledge.<br>Joel Roth did a great job at the wiki but I = think a lot is still <br>missing. Like more details. But anyway.<br><br>Som= e recipes do not need any modifications at all, but I looked at other<br>on= es such as e. g. dbus and qt and they need quite a few modifications <br>he= re or there as can be seen:<br><br>=C2=A0 <a href=3D"http://recipes.gobolin= ux.org/r/?list=3DDBus&ver=3D1.10.8-r3">http://recipes.gobolinux.org/r/?= list=3DDBus&ver=3D1.10.8-r3</a><br>=C2=A0 <a href=3D"http://recipes.gob= olinux.org/r/?list=3DQt&ver=3D5.9.0-r2&file=3DRecipe">http://recipe= s.gobolinux.org/r/?list=3DQt&ver=3D5.9.0-r2&file=3DRecipe</a><br><b= r>I do not yet know what the various entries mean (environment, <br>install= _variables, pre_link) and although I understand shell code to some<br>exten= t, I don't have a great idea how to autogenerate all of that ... but<br= >I think that in theory, once I understand these parts fully, I should be<b= r>able to autogenerate most of them.<br><br>I tested this first on slackwar= e though, since my specific GoboLinux<br>knowledge is currently still too l= imited, but I will test it on GoboLinux<br>once I have finished with the wh= ole kde stack in the cookbooks; in the<br>past, I did not even manage to co= mpile qt/kde at all, which was really<br>frustrating. I also managed to com= pile all of mate-desktop from source,<br>though there also was some problem= with the settings; but mate-terminal<br>works and mate-desktop starts up f= ine too. <br><br>Note that with autogenerated GoboLinux recipes, this is no= t aimed at all<br>to bypass the manual review process in any way whatsoever= .<br><br>But it is supposed to help speed up some things. I can probably al= so add<br>a linter to check for conforming recipes too.<br><br>I am also us= ing some other scripts to help track updates that happen to<br>source code = repositories in general, such as via the blfs changelog page at<br><a href= =3D"http://www.linuxfromscratch.org/blfs/view/cvs/introduction/changelog.ht= ml">http://www.linuxfromscratch.org/blfs/view/cvs/introduction/changelog.ht= ml</a>.<br><br>While there is an initial time investment in adding new entr= ies to the<br>cookbooks project, and making sure that the code logic works = - once the<br>entry is registered, updating is very easy for me. One script= called<br>UpdateEntry accepts an URL and will update the local program + d= ownload.<br>(In theory it can also be used to automatically compile, but I = usually<br>first update, before then invoking the compile process via the r= uby<br>scripts.)<br><br></div><div>Another script also autogenerates indivi= dual .html pages that I could<br></div><div>upload too. And I could probabl= y combine this with existing GoboLinux<br></div><div>recipes... a bit simil= ar to how the BLFS homepage works too. But anyway,<br></div><div>I think I = am writing too much and having too little action ;) - the <br>archive at go= bolinux-recipes-16.09.2017.tar.xz shows that it works at<br></div><div>the = least in principle, as demo. The descriptions I use from the cookbooks<br><= /div><div>gem, but I can also write a script that uses the descriptions fou= nd in<br></div><div>the gobolinux recipes (the github page allows one to ea= sily download<br></div><div>the recipes... there are quite a LOT of recipes= there, some of the recipes<br></div><div>are many years old, with URLs tha= t no longer exist hehe)<br></div><div><br>I also have an auto-update script= for the gnome stack although it is a<br>bit buggy - the URL that I am chec= king for gnome stuff is<br><br><a href=3D"http://ftp.gnome.org/pub/gnome/so= urces/?C=3DM;O=3DD">http://ftp.gnome.org/pub/gnome/sources/?C=3DM;O=3DD</a>= <br><br>Meson is currently annoying me here, so I am waiting until the LFS/= BFLS<br>team shows how to use meson properly. :D<br><br>Here is a simple pr= ogram version dump of projects that I am currently<br>tracking, though I ha= ve to update it with kde stuff yet (it is updated<br>whenever I push a new = release to the cookbooks gem but I broke it<br>a bit recently, so I have to= update manually at a later time):<br><br>=C2=A0 <a href=3D"http://shevegen= .square7.ch/programs_version.html">http://shevegen.square7.ch/programs_vers= ion.html</a><br><br>Anyway, I wanted to give a bit more feedback as to what= I am doing. My<br></div><div>idea is to use the "secondary gobostore&= quot;, and from that one, once it<br></div><div>all works on my gobolinux m= achine, push updates to the recipe store<br></div><div>once it all works.<b= r></div><div><br>I'll try to use a weekend that is not so busy to batch= -test this<br>via GoboLinux but I will also first test that the recipes wor= k on<br>GoboLinux individually, which will take some time.<br><br>Cheers,<b= r>Robert<br></div></div> --f403045c18845671ce05598fd73f-- --===============1981902342944925311== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZ29ib2xpbnV4 LXVzZXJzIG1haWxpbmcgbGlzdApnb2JvbGludXgtdXNlcnNAbGlzdHMuZ29ib2xpbnV4Lm9yZwpo dHRwOi8vbGlzdHMuZ29ib2xpbnV4Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2dvYm9saW51eC11c2Vy cwo= --===============1981902342944925311==--