IMPORTANT: Macros in top-level .h files are no longer by default visible outside core

Hello Porters,

An important change has been made that you should be aware of regarding 
XS code.  Macros that you create from now on in header files like 
'perl.h' will no longer be visible to that XS code by default.  They 
still are automatically visible to the Perl core.

tl;dr

Document new macros (using apidoc) or else no external code can use 
them.  Document new public functions you create or else no external code 
can use its short name.  Help remove current name space pollution by 
scanning the list beginning about line 120 in regen/embed.pl for items 
that clearly shouldn't be visible to external code.

Macros are effectively global symbols, and they are now effectively made 
private unless you take deliberate action to make them public.  They 
essentially become like static functions, but local to the entire Perl core.

This has important implications

1.  We now have much better control over the interface to XS code. 
Module writers can no longer inadvertently (or deliberately) use a macro 
intended only for internal use.

2.  We stop inadvertently polluting the namespace of the XS code.  That 
means you don't have to worry about naming your internal macros with 
clumsy spellings just to avoid the possibility of name collisions.  (It 
also saves you from polluting the name space if you forgot to worry.) 
You also find out early about mistakes where a macro gets used outside 
core when you hadn't considered that possibility.

3.  The main way to get a new symbol to be visible is to document it 
(how is described below).  Documenting isn't just for people to know 
they exist and how to use them; it allows for some automatic code 
generation.  Devel::PPPort currently cannot test many elements because 
it doesn't know the calling conventions.  The embed regeneration also 
can use this to automatically generate code to make 
-DPERL_NO_SHORT_NAMES work, and further enhancements are available for 
it to automatically generate macros that must manually be done now.

4.  It reduces the number of macros in use.  The C Standard says that a 
conformant compiler only need support 4095 macros defined at a time.  We 
are THOUSANDS above that value, so it's clear modern compilers have much 
higher limits than the bare minimum.  Still, a large module could add 
many more macros and this gives them more of a chance of not exceeding 
the limit.

5.  There is a list of macros that are currently externally visible, but 
the new mechanism doesn't see why they should be.  This list allows us 
to see at a glance the current name space pollution we have.  Some items 
on the list should not be visible, and the ones that do need to be 
visible should have names that are unlikely to clash with the users', or 
be documented as being reserved for perl's use.

In order to avoid the possibility of breaking any code that may rely on 
some symbol, all symbols on that list remain visible.  The list contains 
over 3500 macros.  (This is scarily close to the C standard 4095, and 
this is just the macros we don't know about!)  It would be good for the 
project to get this list shortened as much as possible.

I'm asking you to help out by scanning the list for symbols you know 
about, and either submitting pull request(s) or posting what you know. 
The list begins around line 120 in regen/embed.pl in blead.  It is named 
@unresolved_visibility_overrides.

The items on the list can be categorized as:

a) They aren't actually externally visible, and the new mechanism 
mistakenly thinks they are.  This can be because of preprocessor 
directives like #ifdef or #undef that affect the symbol.  Point these 
out to me, so I can fix it.

b) They need to be visible outside core, but aren't documented.  If you 
know enough to adequately document them, please, please submit a pull 
request to do so.  If you don't have the wherewithal to do that, at 
least point them out publicly.  Maybe I should create some mechanism, so 
that you can easily add them to a list.  If the names could easily 
clash, maybe we need to change them.

c) They need to be visible outside core, and are documented, but the new 
mechanism isn't seeing that.  Point these out to me so that I can fix it.

d) They need to be visible outside core, but just to Perl extensions. 
The best way to fix that is a p.r. documenting them with a 'E' 
visibility flag.  But another way is to surround the symbol definition in a

#if defined(PERL_CORE) || defined(PERL_EXT)
...
#endif

e) They need to be visible outside core just to the regex engine.  I 
will be looking for these myself, and will fix them by surrounding them with

#if defined(PERL_CORE) || defined(PERL_EXT_RE_BUILD)
...
#endif

f) We are confident, or merely think that, they shouldn't be visible 
outside core.  The solutions to this are to surround them with an #ifdef 
PER_CORE, or simply remove them from the list via p.r.s, and smoke cpan. 
  If failures arise, we can revert the changes, or negotiate with the 
maintainer.

The mechanism for documenting is described in embed.fnc in the section
AUTOMATIC DOCUMENTATION GENERATION.  The visibility flags are:
     A   means it is part of the API
     C   means it needs to be visible, but is not part of the API
     E   means it needs to be visible to Perl extensions (and the core) only

This change continues the work to create better boundaries between perl 
and XS code.  Internal functions now are hidden from the outside world 
(on participating compilers and loaders), for example.  What remains 
uncontrolled as far as I know are typedefs, enums, structs, and unions. 
(Let me know what I'm missing.)  I don't know how to automatically gain 
control of them, except that the regen process could display all the 
ones that are unexpectedly visible, so that #ifdef's could manually be 
added.  (Again, let me know if there is another way.)

The commit that made this change is 92dcf59a90bfbb545599098d7043c96abb783ee5