Re: Documentation tokenization

>   ## some annoation
>
>   ## more annotation
>
> Ultimately this would be combined to a single documentation. My curiousity
> on this point is simply about correctness.

Additionally, when documentation is separated by a comment does it still
compress to a single documentation? For example:

## some annotation
# A comment
## more annotation

Would this compress to a single documentation item? If so, would changing
"indent" to "separators" be more accurate? Effectively, it seems it would
be. However in terms of process I suppose not. Even if (Documentation, Sep,
Documentation) becomes (Documentation) it may still be important that it was
initially tokenized otherwise, correct?

Thanks again,
Jeff Rafter