Re: Gallery 3.1 - auto-upgrading, directory structure

Gallery 2 has a means of doing this.  It worked, but was heavy and ran into
a lot of issues around things like:

1) API compatibility across major/minor boundaries
2) Filesystem permissions
3) Dealing with local code modifications made by the user
4) Added complexity in the code to deal with multiple possible locations of
modules

I propose that we do the following:
1) Leave our structure exactly the way it is
2) Push out a really simple PHP file that does the following
2a) check filesystem permissions everywhere and make sure that the webapp
can update everything
2b) download a tarball
2c) unpack it on top of all the existing code (and maybe delete extra files
we know to be problems - this saves us the trouble of having to tell users
to blow away all their old stuff)
2d) runs "system php upgrader.php" or something equally API-simple

This is simple and fast.  We can write the code whenever we want.  It's not
guaranteed to be super reliable and we can add tests, etc to try to verify
the environment before we do anything radical.  It should also be very
little code.

thoughts?

On Fri, Apr 12, 2013 at 11:44 AM, Shad Laws <shad-xpYdmXCiSuZWk0Htik3J/[email protected]> wrote:

> Hey everyone,
>
> I feel rather strongly that it'd be a *big* boon to Gallery if we made
> v3.1+ self-upgrading.  Of course, this is a big undertaking, but I think
> it'd be well worth it.
>
> To me, it seems that there are three parts to the problem:
> - infrastructure on www.galleryproject.org
> - self-upgrading code
> - an application structure that makes it easier to self-upgrade without
> destroying user mods
>
> I've been thinking a lot about the last item, and I feel like it should be
> the first to be tackled.  Below are my current thoughts on it...
>
> ----
>
> OVERVIEW
>
> Kohana 3 is pretty agnostic when it comes to module/theme locations.
>  APPPATH and SYSPATH are important, but MODPATH really isn't.  In fact,
> grepping for MODPATH will find one match in Debug::path() which can (and
> should!) be easily overriden.  However, there's a big boon to having
> *everything* (including SYSPATH and APPPATH) use the same module structure.
>
> We leveraged these facts some in Gallery 3.0.x (e.g. MODPATH vs.
> THEMEPATH), but I say we should do so more in Gallery 3.1.x.
>
> So, here's my new idea.  There are eight main subdirectories, each with a
> different purpose.
>
> First, the installer.  This should never be modified by either the end
> user or upgraders.  This is where we should put our core upgrade code.
> - installer
>
> Next, the three "core" directories.  These should never be modified by the
> end user.  A gallery-wide upgrade simply deletes and replaces these.
> - core: application, system, and all third-party modules; no module.infofiles here
> - modules-gallery: all other modules gallery ships with
> - themes-gallery: wind, admin_wind, and lib
>
> Then, the two "contrib" directories.  The end user may need to *install*
> these, but should never *modify* them.  A contrib module installer/updater
> can operate by deleting/rebuilding subdirs here.  These are shipped only
> with README.md files.
> - modules-contrib
> - themes-contrib
>
> Then, the two "site-specific" directories.  This is where all config,
> custom code, and other tweaks should go.  The end user is highly encouraged
> to modify stuff here :-).
> - bin: shipped with only README.md and .htaccess files
> - site: all custom code goes here; see more info below
>
> Finally, the var directory.  This is where all of our data goes.  The end
> user shouldn't need to modify things directly here... and neither should
> Gallery upgraders.
> - var
>
> Once we do this, we can remove every instance of MODPATH and
> Gallery::find_file() in the core code and replace it with
> Kohana::find_file().
>
> ----
>
> DETAILS - MAIN
>
> Here's a sketch of what Gallery would look like.  For brevity, I've
> omitted all files and subdirs with the exception of vendor.
>
> gallery
> |-- bin
> |-- core
> |   |-- application                 [Note 1]
> |   |   `-- vendor
> |   |       `-- htmlpurifier
> |   |-- cache
> |   |-- database
> |   |-- formo
> |   |-- orm
> |   |-- image
> |   |-- pagination
> |   |-- unittest
> |   `-- system
> |-- installer
> |-- modules-contrib
> |-- modules-gallery
> |   |-- akismet
> |   |-- comment
> |   |-- exif
> |   |   `-- vendor
> |   |       `-- exifer
> |   |-- g2_import
> |   |-- gallery                     [Note 2]
> |   |   `-- vendor
> |   |       |-- jquery-file-upload  [Note 3]
> |   |       |-- mediaelementjs
> |   |       `-- joomla
> |   |-- gallery_unittest
> |   |-- image_block
> |   |-- info
> |   |-- notification
> |   |-- organize
> |   |-- recaptcha
> |   |-- rest
> |   |-- rss
> |   |-- search
> |   |-- server_add
> |   |-- slideshow
> |   |-- tag
> |   |-- user
> |   |-- watermark
> |-- site                            [Note 4]
> |-- themes-contrib
> |-- themes-gallery
> |   |-- admin_wind
> |   |-- lib                         [Note 5]
> |   `-- wind
> `-- var                             [Note 6]
>     |-- albums
>     |-- backups                     [Note 7]
>     |-- caches                      [Note 8]
>     |-- logs
>     |-- modules
>     |-- resizes
>     |-- thumbs
>     |-- tmp
>     `-- uploads
>
> Note 1: application includes bootstrap, purifier, and
> load_modules/load_themes code.  This stuff can never be overridden.
> Note 2: gallery includes code to swap the active theme for admin and/or
> override if needed.
> Note 3: jquery-file-upload will replace uploadify/swfobject.
> Note 4: site details are below.
> Note 5: lib details are below.
> Note 6: var root directory is empty, and the database config has been
> moved to site.
> Note 7: should use this for backup SQL dumps :-)
> Note 8: cache is new for Kohana 3 caches.
>
> ----
>
> DETAILS - SITE AND LIB
>
> My goal is to encourage end users to use the site directory for *all*
> tweaks of any kind.  It's full of a bunch of templates, which they can
> modify (and then remove .tmpl) to change gallery without actually touching
> the core or contrib files.
>
> site
> |-- README.md   (tells how to use this directory)
> |-- init.tmpl   (this takes the place of the old local.php file)
> |-- assets
> |   |-- apple-touch-icon.png.tmpl
> |   |-- favicon.ico.tmpl
> |   |-- logo.png.tmpl
> |   |-- site.css.tmpl     (custom css, which is loaded last, goes here)
> |   `-- site.js.tmpl      (custom js, which is loaded last, goes here)
> |-- classes
> |   |-- Example.php.tmpl  (tells how to override a class instead of
> modifying it directly)
> |   `-- Hook              (each template includes a list of possible
> functions)
> |       |-- Rest
> |       |   `-- Site.php.tmpl
> |       |-- SiteEvent.php.tmpl
> |       |-- SiteRss.php.tmpl
> |       |-- SiteTask.php.tmpl
> |       `-- SiteTheme.php.tmpl
> `-- config
>     `-- database.php      (this was var/database.php in 3.0.x)
>
> Also, I'd like to move the lib directory around a bit to make it like the
> other modules, remove the things that aren't specific to the theme
> (uploader and movie player), and then put it as a theme subdir.  I used a
> couple wildcards for brevity below.  The end result would look like this:
>
> lib
> |-- assets
> |   |-- apple-touch-icon.png
> |   |-- favicon.ico
> |   |-- logo.png
> |   |-- gallery*.js
> `-- vendor
>     |-- jquery*.js
>     |-- json*.js
>     |-- superfish
>     `-- yui
>
> ----
>
> MODULE ORDER
>
> Our complete module order would look like this:
>
> core/application (APPPATH req'd by Kohana)
> modules-gallery/gallery_unittest (if TEST_MODE)
> core/unittest (if TEST_MODE)
> site
> [active theme]
> themes-gallery/lib
> [active modules...]
> [active identity provider]
> modules-gallery/gallery
> core/pagination
> core/formo
> core/image
> core/cache
> core/orm
> core/database
> core/system (SYSPATH req'd by Kohana)
>
> Additionally, during bootstrap and configuration, we load a few modules
> before the whole list:
>
> core/application (APPPATH req'd by Kohana)
> site (allows init to be run)
> modules-gallery/gallery
> core/cache
> core/orm
> core/database
> core/system (SYSPATH req'd by Kohana)
>
> ----
>
> Whew - that took awhile to write up!  So... whaddayathink?
>
> Thanks for readin'!
>
> Take care,
> Shad
>
> ------------------------------------------------------------------------------
> Precog is a next-generation analytics platform capable of advanced
> analytics on semi-structured data. The platform includes APIs for building
> apps and a phenomenal toolset for data science. Developers can use
> our toolset for easy data analysis & visualization. Get a free account!
> http://www2.precog.com/precogplatform/slashdotnewsletter
> __[ g a l l e r y - d e v e l ]_________________________
>
> [ list info/archive --> http://gallery.sf.net/lists.php ]
> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

__[ g a l l e r y - d e v e l ]_________________________

[ list info/archive --> http://gallery.sf.net/lists.php ]
[ gallery info/FAQ/download --> http://gallery.sf.net ]