How I'm Organizing a Large Io Project

In my game engine I now have over 64 Io scripts and probably 100 protos.  I reference many 3rd-party libraries, and I want to know what scripts are importing what types and organize the scripts so that as much as possible of the rest of the game engine is still functional if a 3rd party library is removed.  (For instance, a game server might retain the physics library but not load the graphics library.)

It's taken me a long time to work out how I want to organize this, and refactoring to support this new way of organizing is still in progress.  I believe the way I've designed will solve a lot of my problems, but I'd be interested in what others think.

My old way of organizing things was just lots of calls to "appendProto" to get 3rd-party dependencies' namespaces, and my startup script was just a long list of "doRelativeFile".  I had to run the scripts in a certain order so that protos one script needed to clone would existed when they were needed.  I also had problems where two scripts developed mutual dependency on each other so that either order of running them did not work.

I wanted to achieve three things with the new system:  run "doFile" automatically by recursively finding all the ".io" files in all my script directories, make scripts declare their dependencies and have them passed in rather than the script reaching out to grab it on its own, and have the system automatically wire up these dependencies.  The scheme I came up with should achieve the first two goals, but does not address the third goal.  The programmer still has to manually pass in the dependencies.

For this new style of organization, I tried to combine an idea floated on the mailing list a while ago of folders-as-objects and files-as-methods, with the concept from Gilad Bracha's Newspeak of not having a global namespace so that scripts would be required to take their dependencies as arguments instead of reaching out and getting them.

My new script loader creates a clone of a proto called "Module" for each filesystem folder.  The module is named the same as the folder name.  Within each module it creates slots named the same as the Io files in the folder, and assigns to that slot the result of doFile on the script.  (And sub-folders become sub-modules of the module, with the slot named the same as the folder name.)

I've taken all my scripts and wrapped the entire code with a method block.  That way instead of creating the proto directly, when doFile is run it just creates a method which will be called later to create the proto.  Since Io is lazy and dynamic, it doesn't try to resolve any names in the method body when the method is declared, only when it is called.  So all the doFiles can run in any order and create all the methods, without worrying about dependencies or mutual dependencies at load time.

All of the dependencies the script has are method arguments to the outer method.  Therefore it has no dependencies on a global namespace; instead they are all method arguments.

Inside the methods in my scripts files I've changed my proto declarations from "name := proto clone" to "self name := proto clone".  Since "name" is the same as the name of the file, which was used as the slot name, what happens is running the method the first time *replaces* the slot which was a method that creates the proto, with the proto itself.  So, in a configuration script you must call the slot at least once and pass in the dependencies.  After that, the slot in the module refers to the actual type created, instead of the method.

An example:

// in file "~/MyModule/Test.io"
method(Dependency, someValue,
self Test := Dependency clone do( x := someValue ) 
)

// in file "~/MyModule/Config.io"
method(Dependency,
Test(Dependency, 1234)
)

// in file "~/Startup.io"
MyModule := Module clone setDirectory("~/MyModule") doAllScripts
MyModule Config(SomeProto)

After the call to "doAllScripts" in startup, MyModule looks like this:
MyModule( Test := method(...) )

And after running "Config(SomeProto)", MyModule looks like this:
MyModule( Test := SomeProto( x := 1234 ) )

An important property of this system is sub-modules do NOT automatically get access to the namespace of more-global modules.  I'm intentionally using a capabilities-model to prevent that; submodules do not have any slot that references their parent modules.  Instead, they must have things passed in from higher up as method arguments in the Config script.  Config scripts at higher levels pass needed dependencies down to Config scripts at lower levels.