third-party package tips and developer concerns (Was: New version of file-watchers)

First, to Sage, thank you for going to the work to open source 
third-party packages like this.

Here are a few suggestions that came to mind from this post, which I'd 
like to mention to Racket third-party package developers in general. 
(Only suggestions, not presuming to tell anyone what to do; just 
avoiding real work this early Saturday morning. :)  Also, as I was 
writing this, I realized one of the points is an opportunity to give 
some context on recent governance concerns.

Note that some of these suggestions are biased to a development approach 
that encourages developing all software in such a way that there's often 
low friction to project/organization-internal component reuse, or even 
to selectively open-sourcing individual modules.

* Normally, you'll be developing your own large systems, and a given 
open source package will just be a small part of it, and you might have 
many such open source packages.  You probably don't want to be slowing 
yourself down with offering people previews of new versions, lead times 
for input, etc. -- when you're ready to push a new version, just do it.  
There are other mechanisms for not breaking users of your package.  Of 
course you can ask for input if you're contemplating big or breaking 
changes, and not sure what to do.

* Try not to make identifiers be simple generic terms, like `watch`, 
that might better be used for key language features, or in users' own 
programs.  (Note that, historically in Racket, there was an idea that 
modules might provide such simple generic identifiers, and then modules 
using them would add a prefix or otherwise transform them. But that had 
a couple problems: it was more work for the module user in practice, and 
it also meant that reading unfamiliar code or looking up unfamiliar 
identifiers in documentation/Google search was harder.  There will 
always be a lot of name collisions in Racket documentation search of 
core Racket and whatever third-party modules are installed, but 
non-generic names improves that in a worse-is-better kind of way.)

* Of course, you can name your identifiers however you want, and you 
might want to make them fairly descriptive, in the scope of all Racket 
packages.  One convention that sometimes (not always) seems good is to 
include the package name in each exported identifier, which helps search 
and recognizing which package an identifier comes from when reading 
code, and also reduces name collisions between packages.  For example, I 
think every identifier from the `roomba` package contains `roomba` in 
its name (and if you wanted shorter names for this package, to make it 
more like, say, Logo turtle graphics, that might be a `#lang`), 
"https://www.neilvandyke.org/racket/roomba/  That convention might not 
be appropriate for `file-watchers`, but I just wanted to mention it, 
because it seems to often help.

* Try not to remove deprecated interfaces from within a package, unless 
keeping them is a significant burden for you.  You can move the 
documentation for them to some dark corner of the manual (like the end 
of the manual, or end of a section), and use the `deprecated` Scribble 
procedure on them.  But if it's practical to keep them in, that reduces 
breaking users of your modules.  Think of a little deprecated API as 
like scars of a battle-tested and enduring package.

* When you do want to break the interfaces, there's currently no great 
way.  Racket's original PLaneT package system had a very simple yet very 
nice SemVer-like versioning system that some third-party package authors 
used,[1] and multiple versions of the same package could be installed at 
once.  That was particularly helpful for low-encumbrance third-party 
packages.  Users of the package could keep using the old version until 
they were ready to make changes to use the new one, and the package 
developer (usually altruistically releasing some part of their system 
that was their real job) wasn't unduly burdened.  And being able to have 
multiple versions of a package installed at once could often (though not 
always) let you move incrementally, of packages you control, and from 
the graph of interdependent third-party packages you use.  With the new 
package system, that version-related functionality was lost, and it was 
emphasized as policy that new package versions should always be 
backwards compatible, and if you wanted to break an interface, you 
should make a new package (with a new name).  Losing that various 
version-related functionality made sense for core Racket's policy change 
to support backward-compatibility, but I think it didn't make sense for 
the breadth of third-party developers for whom we'd like to encourage 
low-friction release and updating of their packages.[2]


[1] See 
"http://planet.racket-lang.org/display.ss?package=javascript.plt&owner=dherman", 
which was up non-backward-compatible version 9.  Dave Herman was able to 
keep doing his work, but also keep releasing new versions. People who 
used that could depend on it and update at their convenience.  You'll 
also see use of that in many of his other packages at 
"http://planet.racket-lang.org/display.ss?owner=dherman", and also use 
of non-backward-compatible versioning by other people at 
"http://planet.racket-lang.org/".  I also used this feature for various 
big and small packages, 
"http://planet.racket-lang.org/display.ss?owner=neil".

[2] This was a great learning moment for Racket open source, and 
relevant to the disruption with governance and Racket2.  How things 
happened was very costly for some third-party developers and for morale, 
and we lost at least one of our prolific contributors and a nice book 
over it.  If anyone is confused why some Racketeers recently see to feel 
strongly about being clear about exactly what will drive Racket 
decisions and how that will be made (top-level requirements, process, 
etc.) -- one reason is the history of Racket itself (this and other 
events), besides all the things we've learned from countless other open 
source and industry projects.  It's easy to see fairly new Racketeers 
understandably making unwarranted assumptions about the goals and where 
things will go.  Being unambiguously clear about what drives Racket 
decisions, and being able to trace everything back to those goals 
(through multiple decisions/requirements), seems very healthy for both 
new and remaining long-time Racketeers.  It will also help focus 
discussion about Racket2, once things get past brainstorming (e.g., save 
people a lot of time working on details that would trace back to 
mutually-exclusive higher-level requirements).

-- 
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/[email protected]
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/5b352f83-fb65-2272-ef8a-2846261aae62%40neilvandyke.org.