Gallery 3.1 - auto-upgrading, directory structure

Hey everyone,

I feel rather strongly that it'd be a *big* boon to Gallery if we made
v3.1+ self-upgrading.  Of course, this is a big undertaking, but I think
it'd be well worth it.

To me, it seems that there are three parts to the problem:
- infrastructure on www.galleryproject.org
- self-upgrading code
- an application structure that makes it easier to self-upgrade without
destroying user mods

I've been thinking a lot about the last item, and I feel like it should be
the first to be tackled.  Below are my current thoughts on it...

----

OVERVIEW

Kohana 3 is pretty agnostic when it comes to module/theme locations.
 APPPATH and SYSPATH are important, but MODPATH really isn't.  In fact,
grepping for MODPATH will find one match in Debug::path() which can (and
should!) be easily overriden.  However, there's a big boon to having
*everything* (including SYSPATH and APPPATH) use the same module structure.

We leveraged these facts some in Gallery 3.0.x (e.g. MODPATH vs.
THEMEPATH), but I say we should do so more in Gallery 3.1.x.

So, here's my new idea.  There are eight main subdirectories, each with a
different purpose.

First, the installer.  This should never be modified by either the end user
or upgraders.  This is where we should put our core upgrade code.
- installer

Next, the three "core" directories.  These should never be modified by the
end user.  A gallery-wide upgrade simply deletes and replaces these.
- core: application, system, and all third-party modules; no
module.infofiles here
- modules-gallery: all other modules gallery ships with
- themes-gallery: wind, admin_wind, and lib

Then, the two "contrib" directories.  The end user may need to *install*
these, but should never *modify* them.  A contrib module installer/updater
can operate by deleting/rebuilding subdirs here.  These are shipped only
with README.md files.
- modules-contrib
- themes-contrib

Then, the two "site-specific" directories.  This is where all config,
custom code, and other tweaks should go.  The end user is highly encouraged
to modify stuff here :-).
- bin: shipped with only README.md and .htaccess files
- site: all custom code goes here; see more info below

Finally, the var directory.  This is where all of our data goes.  The end
user shouldn't need to modify things directly here... and neither should
Gallery upgraders.
- var

Once we do this, we can remove every instance of MODPATH and
Gallery::find_file() in the core code and replace it with
Kohana::find_file().

----

DETAILS - MAIN

Here's a sketch of what Gallery would look like.  For brevity, I've omitted
all files and subdirs with the exception of vendor.

gallery
|-- bin
|-- core
|   |-- application                 [Note 1]
|   |   `-- vendor
|   |       `-- htmlpurifier
|   |-- cache
|   |-- database
|   |-- formo
|   |-- orm
|   |-- image
|   |-- pagination
|   |-- unittest
|   `-- system
|-- installer
|-- modules-contrib
|-- modules-gallery
|   |-- akismet
|   |-- comment
|   |-- exif
|   |   `-- vendor
|   |       `-- exifer
|   |-- g2_import
|   |-- gallery                     [Note 2]
|   |   `-- vendor
|   |       |-- jquery-file-upload  [Note 3]
|   |       |-- mediaelementjs
|   |       `-- joomla
|   |-- gallery_unittest
|   |-- image_block
|   |-- info
|   |-- notification
|   |-- organize
|   |-- recaptcha
|   |-- rest
|   |-- rss
|   |-- search
|   |-- server_add
|   |-- slideshow
|   |-- tag
|   |-- user
|   |-- watermark
|-- site                            [Note 4]
|-- themes-contrib
|-- themes-gallery
|   |-- admin_wind
|   |-- lib                         [Note 5]
|   `-- wind
`-- var                             [Note 6]
    |-- albums
    |-- backups                     [Note 7]
    |-- caches                      [Note 8]
    |-- logs
    |-- modules
    |-- resizes
    |-- thumbs
    |-- tmp
    `-- uploads

Note 1: application includes bootstrap, purifier, and
load_modules/load_themes code.  This stuff can never be overridden.
Note 2: gallery includes code to swap the active theme for admin and/or
override if needed.
Note 3: jquery-file-upload will replace uploadify/swfobject.
Note 4: site details are below.
Note 5: lib details are below.
Note 6: var root directory is empty, and the database config has been moved
to site.
Note 7: should use this for backup SQL dumps :-)
Note 8: cache is new for Kohana 3 caches.

----

DETAILS - SITE AND LIB

My goal is to encourage end users to use the site directory for *all*
tweaks of any kind.  It's full of a bunch of templates, which they can
modify (and then remove .tmpl) to change gallery without actually touching
the core or contrib files.

site
|-- README.md   (tells how to use this directory)
|-- init.tmpl   (this takes the place of the old local.php file)
|-- assets
|   |-- apple-touch-icon.png.tmpl
|   |-- favicon.ico.tmpl
|   |-- logo.png.tmpl
|   |-- site.css.tmpl     (custom css, which is loaded last, goes here)
|   `-- site.js.tmpl      (custom js, which is loaded last, goes here)
|-- classes
|   |-- Example.php.tmpl  (tells how to override a class instead of
modifying it directly)
|   `-- Hook              (each template includes a list of possible
functions)
|       |-- Rest
|       |   `-- Site.php.tmpl
|       |-- SiteEvent.php.tmpl
|       |-- SiteRss.php.tmpl
|       |-- SiteTask.php.tmpl
|       `-- SiteTheme.php.tmpl
`-- config
    `-- database.php      (this was var/database.php in 3.0.x)

Also, I'd like to move the lib directory around a bit to make it like the
other modules, remove the things that aren't specific to the theme
(uploader and movie player), and then put it as a theme subdir.  I used a
couple wildcards for brevity below.  The end result would look like this:

lib
|-- assets
|   |-- apple-touch-icon.png
|   |-- favicon.ico
|   |-- logo.png
|   |-- gallery*.js
`-- vendor
    |-- jquery*.js
    |-- json*.js
    |-- superfish
    `-- yui

----

MODULE ORDER

Our complete module order would look like this:

core/application (APPPATH req'd by Kohana)
modules-gallery/gallery_unittest (if TEST_MODE)
core/unittest (if TEST_MODE)
site
[active theme]
themes-gallery/lib
[active modules...]
[active identity provider]
modules-gallery/gallery
core/pagination
core/formo
core/image
core/cache
core/orm
core/database
core/system (SYSPATH req'd by Kohana)

Additionally, during bootstrap and configuration, we load a few modules
before the whole list:

core/application (APPPATH req'd by Kohana)
site (allows init to be run)
modules-gallery/gallery
core/cache
core/orm
core/database
core/system (SYSPATH req'd by Kohana)

----

Whew - that took awhile to write up!  So... whaddayathink?

Thanks for readin'!

Take care,
Shad

------------------------------------------------------------------------------
Precog is a next-generation analytics platform capable of advanced
analytics on semi-structured data. The platform includes APIs for building
apps and a phenomenal toolset for data science. Developers can use
our toolset for easy data analysis & visualization. Get a free account!
http://www2.precog.com/precogplatform/slashdotnewsletter

__[ g a l l e r y - d e v e l ]_________________________

[ list info/archive --> http://gallery.sf.net/lists.php ]
[ gallery info/FAQ/download --> http://gallery.sf.net ]