Re: bogofilter closing stdin prematurely in passthrough mode

This is a multi-part message in MIME format.
--------------vIV9e8oBzXx1X50ZRPBDfbmy
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 21. 09. 25 23:58, Matthias Andree via bogofilter wrote:
> Am 21.09.25 um 13:03 schrieb Tomaž Šolc via bogofilter:
>> Hi
>>
>> I have a bogofilter setup where procmail pipes messages through 
>> bogofilter for sorting. I've been looking into a problem where a 
>> certain email causes procmail to error out ("error while writing to 
>> bogofilter") and repeatedly defer the delivery.
>>
>> It appears that the reason is that bogofilter prematurely closes stdin 
>> and exits while procmail still has bytes to write. Indeed, if I just 
>> cat the offending email through "bogofilter -p" the email printed on 
>> stdout is truncated somewhere in the middle.
>>
>> Otherwise bogofilter appears to work normally. It doesn't return an 
>> error code. There's no segfault. The only weird thing I see is that if 
>> I do a strace it attempts to do an invalid seek on fd=0 before it 
>> exits, but this might be benign.
>>
>> I think the email is hitting somekind of a token or input buffer 
>> limit. It has a few very long sequences of U+3164 "HANGUL FILLER" 
>> characters. I've been able to reproduce the problem by creating an 
>> email with such a sequence. Bogofilter cuts it off after 3640 
>> characters (but, for example, it will not cut off an identical email 
>> with an ascii "a" instead of U+3164).
>>
>> This is with bogofilter 1.2.5 as packaged in Debian Bookworm.
>>
>> I'll check if the same issue appears with 1.3.0 once I get it running. 
> 
> 
> Tomaž, thanks for the report.
> 
> I believe 1.3.0 is bound to fix that, as the result of reports from - 
> among others - Jonathan Kamens (Gitlab issues #7 and #11) in the beta 
> phase,
> and if you want to avoid messing with the 1.3.0 build if that's too 
> troublesome, you can also send me your reduced test case off-list and I 
> can test it.

Thank you both. I've been out of the loop for a while and only realized 
there's an issue tracker after I sent my last mail.

Yes, issue #7 describes the same symptoms I see. Probably lexer tries to 
stuff the long sequence of U+3164 characters into the buffer in a 
similar way described there.

I've tested the current git main branch and it works as expected.

I've attached a patch with a test case (based on 
t.passthrough-truncation) that fails on 1.2.5 and passes on 1.3.0. Feel 
free to include it if you think it will help prevent future regressions.

Best regards
Tomaž
--------------vIV9e8oBzXx1X50ZRPBDfbmy
Content-Type: text/x-patch; charset=UTF-8;
 name="0001-Add-another-test-for-passthrough-truncation.patch"
Content-Disposition: attachment;
 filename="0001-Add-another-test-for-passthrough-truncation.patch"
Content-Transfer-Encoding: base64

RnJvbSAyZmNkNGVhMjA1ZDBiMzQ3MmZmODY1Zjc1YzFmNzQ4NWUwMTUxMGZlIE1vbiBTZXAg
MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBUb21heiBTb2xjIDx0b21hei5zb2xjQHRhYmxpeC5v
cmc+CkRhdGU6IE1vbiwgMjIgU2VwIDIwMjUgMTA6MjE6MzkgKzAyMDAKU3ViamVjdDogW1BB
VENIXSBBZGQgYW5vdGhlciB0ZXN0IGZvciBwYXNzdGhyb3VnaCB0cnVuY2F0aW9uLgoKVGVz
dCBtZXNzYWdlIGNvbnNpc3RzIG9mIHNvbWUgQVNDSUkgdGV4dCwgMTVrIG9mIFUrMzE2NCBj
aGFyYWN0ZXJzIGFuZCBzb21lCkFTQ0lJIHRleHQgKHRleHQvcGxhaW4gd2l0aCB1dGY4KS4K
CkJvZ29maWx0ZXIgMS4yLjUgc2lsZW50bHkgdHJ1bmNhdGVzIHRoaXMgbWVzc2FnZSBpbiBw
YXNzdGhyb3VnaCBtb2RlLgotLS0KIGJvZ29maWx0ZXIvc3JjL3Rlc3RzL01ha2VmaWxlLmFt
ICAgICAgICAgICAgICB8ICAgMyArKy0KIC4uLi9pbnB1dHMvdC5wYXNzdGhyb3VnaC0xNWst
dTMxNjQtaW4uZ3ogICAgICB8IEJpbiAwIC0+IDM5MiBieXRlcwogYm9nb2ZpbHRlci9zcmMv
dGVzdHMvdC5wYXNzdGhyb3VnaC0xNWstdTMxNjQgIHwgIDE5ICsrKysrKysrKysrKysrKysr
KwogMyBmaWxlcyBjaGFuZ2VkLCAyMSBpbnNlcnRpb25zKCspLCAxIGRlbGV0aW9uKC0pCiBj
cmVhdGUgbW9kZSAxMDA2NDQgYm9nb2ZpbHRlci9zcmMvdGVzdHMvaW5wdXRzL3QucGFzc3Ro
cm91Z2gtMTVrLXUzMTY0LWluLmd6CiBjcmVhdGUgbW9kZSAxMDA3NTUgYm9nb2ZpbHRlci9z
cmMvdGVzdHMvdC5wYXNzdGhyb3VnaC0xNWstdTMxNjQKCmRpZmYgLS1naXQgYS9ib2dvZmls
dGVyL3NyYy90ZXN0cy9NYWtlZmlsZS5hbSBiL2JvZ29maWx0ZXIvc3JjL3Rlc3RzL01ha2Vm
aWxlLmFtCmluZGV4IGQ1MzQ1YjQyLi4zM2Y4NmQ2YiAxMDA2NDQKLS0tIGEvYm9nb2ZpbHRl
ci9zcmMvdGVzdHMvTWFrZWZpbGUuYW0KKysrIGIvYm9nb2ZpbHRlci9zcmMvdGVzdHMvTWFr
ZWZpbGUuYW0KQEAgLTM0LDcgKzM0LDcgQEAgUEFSU0lOR19URVNUUyA9IFwKIAl0Lmlnbm9y
ZV9zcGFtX2hlYWRlciBcCiAJdC5udWxsc3RhdHNwcmVmaXggXAogCXQuaW50ZWdyaXR5IHQu
aW50ZWdyaXR5MiB0LmludGVncml0eTMgXAotCXQucGFzc3Rocm91Z2gtaGIgdC5wYXNzdGhy
b3VnaC10cnVuY2F0aW9uIFwKKwl0LnBhc3N0aHJvdWdoLWhiIHQucGFzc3Rocm91Z2gtdHJ1
bmNhdGlvbiB0LnBhc3N0aHJvdWdoLTE1ay11MzE2NCBcCiAJdC5lc2NhcGVkLmh0bWwgdC5l
c2NhcGVkLnVybCBcCiAJdC5iYXNlNjQgdC5zcGxpdCB0LnBhcnNpbmcgXAogCXQubGV4ZXIg
dC5sZXhlci5tYnggdC5sZXhlci5xcGNyIHQubGV4ZXIuZW9oIFwKQEAgLTEwNSw2ICsxMDUs
NyBAQCBFWFRSQV9ESVNUPSQoVEVTVFNDUklQVFMpIHQuZnJhbWUgdC5zYXZlIHQuc2tlbCBc
CiAJaW5wdXRzL21zZy5zcGxpdC5ncy4wMTE5LnRleHQgXAogCWlucHV0cy9zcGFtLm1ieCBc
CiAJaW5wdXRzL3QucGFzc3Rocm91Z2gtdHJ1bmNhdGlvbi1pbi5neiBcCisJaW5wdXRzL3Qu
cGFzc3Rocm91Z2gtMTVrLXUzMTY0LWluLmd6IFwKIAlpbnB1dHMvaW5wdXQtc2YtYnVnLTEy
NC15eWlucHV0LXRtaW4uZ3ogXAogCWlucHV0cy9pbnB1dC1zZi1idWctMTI0LWNvdW50LXRt
aW4uZ3ogXAogCWlucHV0cy9pbnB1dC1zZi1idWctMTI0LWJ1ZmZzaGlmdC10bWluLmd6IFwK
ZGlmZiAtLWdpdCBhL2JvZ29maWx0ZXIvc3JjL3Rlc3RzL2lucHV0cy90LnBhc3N0aHJvdWdo
LTE1ay11MzE2NC1pbi5neiBiL2JvZ29maWx0ZXIvc3JjL3Rlc3RzL2lucHV0cy90LnBhc3N0
aHJvdWdoLTE1ay11MzE2NC1pbi5negpuZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAw
MDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwLi42NmEzMmQxYjY0OTk1Yjhl
ZjNiMWQ3ODUzNTUyZjI3ZmJjMGFjNGQ2CkdJVCBiaW5hcnkgcGF0Y2gKbGl0ZXJhbCAzOTIK
emNtYjJ8PTNvRT09Q0B+VHlfcD51U1Q1QW5BRCNHYlFJZGNxci1xX2VteHh6S0xpeFBlezYx
eGVAM0tESEZWNCs8CnooQz43KGtNV3Mzb19AN0h3TDU9VTk5aXxeQGspRV5IP01NY1kwbVAh
VGVlWjJ6d2hzPUojc0I0emkoT195dUI7ZQp6KCR9fXtfT2pCQEo9YkZRTlZXZFZ2RDJOdEAk
RUx0KDQkOGAqeGNEKDg4NW5lUFg2UFNfdkZoOFojPTF1YyhVSTYKel9oUm18UGNKWD8/MDxk
biNtflIlSXNaMnZOSENEI1ZRVzQoRVR8M2xqakpRJTtAXkY7czlfUjZ3UURFO0l8Yz4tCkRL
N2cwNwoKbGl0ZXJhbCAwCkhjbVY/ZDAwMDAxCgpkaWZmIC0tZ2l0IGEvYm9nb2ZpbHRlci9z
cmMvdGVzdHMvdC5wYXNzdGhyb3VnaC0xNWstdTMxNjQgYi9ib2dvZmlsdGVyL3NyYy90ZXN0
cy90LnBhc3N0aHJvdWdoLTE1ay11MzE2NApuZXcgZmlsZSBtb2RlIDEwMDc1NQppbmRleCAw
MDAwMDAwMC4uNDUyMTdkNDAKLS0tIC9kZXYvbnVsbAorKysgYi9ib2dvZmlsdGVyL3NyYy90
ZXN0cy90LnBhc3N0aHJvdWdoLTE1ay11MzE2NApAQCAtMCwwICsxLDE5IEBACisjISAvYmlu
L3NoCisKKy4gJHtzcmNkaXI6PS59L3QuZnJhbWUKKworIyB0LnBhc3N0aHJvdWdoLTE1ay11
MzE2NAorIworIwl0ZXN0IGZvciBjb3JyZWN0IHBhc3N0aHJvdWdoOiBBU0NJSSB0ZXh0LCAx
NWsgb2YgVSszMTY0IGNoYXJhY3RlcnMsCisjCUFTQ0lJIHRleHQuCisKK2d6aXAgLWMgLWQg
PCIkc3JjZGlyL2lucHV0cy90LnBhc3N0aHJvdWdoLTE1ay11MzE2NC1pbi5neiIgPiIkVE1Q
RElSL2lucHV0IgorJEJPR09GSUxURVIgLWUgLXAgLUMgLUkgIiRUTVBESVIvaW5wdXQiID4i
JHtUTVBESVJ9L2ludGVybWVkaWF0ZSIKKyRHUkVQIC12ICJeWC1Cb2dvc2l0eTogVW5zdXJl
LCIgPCIke1RNUERJUn0vaW50ZXJtZWRpYXRlIiA+IiRUTVBESVIvb3V0cHV0IgorCitpZiAg
WyAkdmVyYm9zZSAtZXEgMCBdOyB0aGVuCisgICAgY21wICIkVE1QRElSL2lucHV0IiAiJFRN
UERJUi9vdXRwdXQiCitlbHNlCisgICAgc2V0ICtlCisgICAgZGlmZiAkRElGRl9CUklFRiAi
JFRNUERJUi9pbnB1dCIgIiRUTVBESVIvb3V0cHV0IgorZmkKLS0gCjIuMzkuNQoK

--------------vIV9e8oBzXx1X50ZRPBDfbmy
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
bogofilter mailing list
[email protected]
https://www.bogofilter.org/mailman/listinfo/bogofilter

--------------vIV9e8oBzXx1X50ZRPBDfbmy--