Re: Disable all checksum verification in flac/metaflac
"Manuel Bertsch" <[email protected]> Fri, 30 Jun 2023 12:48:12 +0200
| Newsgroups | gmane.comp.audio.compression.flac.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multipart message in MIME format. --===============3038988820503109178== Content-Type: multipart/alternative; boundary="----=_NextPart_000_0003_01D9AB51.218CDA40" Content-Language: de This is a multipart message in MIME format. ------=_NextPart_000_0003_01D9AB51.218CDA40 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello, =20 thank you for the response, I=E2=80=99ll try compiling later with these = lines removed. I=E2=80=99d be in favor of a proper compile-time option = for this. =20 > I think providing an API function and flac+metaflac command line = option is going to be confusing and they will probably be misused. A = compile time option might be possible though, libogg provides that too. I don=E2=80=99t see how the option would be misused or be confusing. As = I=E2=80=99ve discovered, popular FLAC-compatible tools like ffmpeg (or = libavcodec=E2=80=99s FLAC implementation respectively) don=E2=80=99t = strictly adhere to the standard here in order to support weirder files, = and the FLAC tools themselves already have options to adhere or deviate = from the streamable subset, i.e. a particularly simple part of the FLAC = format. If the option has a name that cannot be used by accident, and if = it=E2=80=99s properly documented that it may lead to mis-syncing, = glitchy audio etc., I don=E2=80=99t see the harm in a run-time option. = Though I won=E2=80=99t die on this hill, so feel free to ignore me here. =20 Regards and thanks for your help, =20 kleines Filmr=C3=B6llchen =20 Von: Martijn van Beurden <[email protected]>=20 Gesendet: Freitag, 30. Juni 2023 07:23 An: Manuel Bertsch <[email protected]> Cc: [email protected] Betreff: Re: [flac-dev] Disable all checksum verification in = flac/metaflac =20 Hi, =20 Is it possible for you to compile flac/metaflac yourself? Then you'll = only need to remove a few lines, namely these 4: = https://github.com/xiph/flac/blob/28e4f0528c76b296c561e922ba67d4375199059= 9/src/libFLAC/stream_decoder.c#L2118-L2121 and these 7: = https://github.com/xiph/flac/blob/28e4f0528c76b296c561e922ba67d4375199059= 9/src/libFLAC/stream_decoder.c#L2141-L2147 As you can see these lines = are also disabled on fuzzing. =20 I think providing an API function and flac+metaflac command line option = is going to be confusing and they will probably be misused. A compile = time option might be possible though, libogg provides that too. =20 Kind regards, =20 Martijn van Beurden =20 Op do 29 jun. 2023 22:28 schreef Manuel Bertsch <[email protected] = <mailto:[email protected]> >: Hello all, =20 I=E2=80=99m trying to use flac and metaflac as tooling to analyze the = generated FLAC files of a new encoder implementation. However, while my = own third-party decoder can handle the output just fine, flac, metaflac, = and other tools discard the output since the frame checksum (footer = CRC-16) is invalid. This is expected, since I have no checksum = generation implemented apart from the frame header checksum (all other = checksums are written as 0), but it prevents me from making development = process without being able to check my outputs against the reference = implementation. I looked around and I couldn=E2=80=99t find a way to = turn off either tool=E2=80=99s checksum verification. So I=E2=80=99d = like to know how one could turn off checksum verification in the tools, = and if that=E2=80=99s not currently possible, I=E2=80=99d like to see = that as a feature in a future version of flac. (In that case I would = also try to disable the checks in code permanently, though my solution = is most likely not going to be upstreamable in any capacity.) =20 FYI, the third-party decoder is highly compliant according to the spec = test suite, producing identical audio playback on almost all of the = files, but it also doesn=E2=80=99t verify or error on any checksums. = ffplay and ffmpeg both accept the file and can play and transcode it = correctly, although they ignore the sample count in the streaminfo = header and will read all frames they can find. The latter leads me to = believe that checksums are the actual issue, even though flac reports = the generic =E2=80=9CBad Header=E2=80=9D and =E2=80=9CLost = Sync=E2=80=9D. =20 Greetings and thanks in advance, =20 Kleines Filmr=C3=B6llchen _______________________________________________ flac-dev mailing list [email protected] <mailto:[email protected]>=20 http://lists.xiph.org/mailman/listinfo/flac-dev ------=_NextPart_000_0003_01D9AB51.218CDA40 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta = http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta = name=3DGenerator content=3D"Microsoft Word 15 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4;} @font-face {font-family:"Yu Gothic"; panose-1:2 11 4 0 0 0 0 0 0 0;} @font-face {font-family:"\@Yu Gothic"; panose-1:2 11 4 0 0 0 0 0 0 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0cm; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman",serif;} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {mso-style-priority:34; margin-top:0cm; margin-right:0cm; margin-bottom:0cm; margin-left:36.0pt; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman",serif;} p.msonormal0, li.msonormal0, div.msonormal0 {mso-style-name:msonormal; mso-margin-top-alt:auto; margin-right:0cm; mso-margin-bottom-alt:auto; margin-left:0cm; font-size:12.0pt; font-family:"Times New Roman",serif;} span.E-MailFormatvorlage18 {mso-style-type:personal; font-family:"Calibri",sans-serif; color:#1F497D;} span.E-MailFormatvorlage20 {mso-style-type:personal-compose; font-family:"Calibri",sans-serif; color:windowtext;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri",sans-serif;} @page WordSection1 {size:612.0pt 792.0pt; margin:70.85pt 70.85pt 2.0cm 70.85pt;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DDE link=3Dblue = vlink=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'>Hello,<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>thank you = for the response, I=E2=80=99ll try compiling later with these lines = removed. I=E2=80=99d be in favor of a proper compile-time option for = this.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US>> I think providing an API = function and flac+metaflac command line option is going to be confusing = and they will probably be misused. A compile time option might be = possible though, libogg provides that too.<o:p></o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>I = don=E2=80=99t see how the option would be misused or be confusing. As = I=E2=80=99ve discovered, popular FLAC-compatible tools like ffmpeg (or = libavcodec=E2=80=99s FLAC implementation respectively) don=E2=80=99t = strictly adhere to the standard here in order to support weirder files, = and the FLAC tools themselves already have options to adhere or deviate = from the streamable subset, i.e. a particularly simple part of the FLAC = format. If the option has a name that cannot be used by accident, and if = it=E2=80=99s properly documented that it may lead to mis-syncing, = glitchy audio etc., I don=E2=80=99t see the harm in a run-time option. = Though I won=E2=80=99t die on this hill, so feel free to ignore me = here.<o:p></o:p></span></p><p class=3DMsoNormal><span lang=3DEN-US = style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>Regards and = thanks for your help,<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><span lang=3DEN-US style=3D'color:#1F497D'>kleines = Filmr=C3=B6llchen<o:p></o:p></span></p><p class=3DMsoNormal><span = lang=3DEN-US style=3D'color:#1F497D'><o:p> </o:p></span></p><p = class=3DMsoNormal><b>Von:</b> Martijn van Beurden = <[email protected]> <br><b>Gesendet:</b> Freitag, 30. Juni 2023 = 07:23<br><b>An:</b> Manuel Bertsch = <[email protected]><br><b>Cc:</b> = [email protected]<br><b>Betreff:</b> Re: [flac-dev] Disable all checksum = verification in flac/metaflac<o:p></o:p></p><p = class=3DMsoNormal><o:p> </o:p></p><div><div><p = class=3DMsoNormal>Hi,<o:p></o:p></p><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>Is it possible for you to compile flac/metaflac = yourself? Then you'll only need to remove a few lines, namely these 4: = <a = href=3D"https://github.com/xiph/flac/blob/28e4f0528c76b296c561e922ba67d43= 751990599/src/libFLAC/stream_decoder.c#L2118-L2121">https://github.com/xi= ph/flac/blob/28e4f0528c76b296c561e922ba67d43751990599/src/libFLAC/stream_= decoder.c#L2118-L2121</a> and these 7: <a = href=3D"https://github.com/xiph/flac/blob/28e4f0528c76b296c561e922ba67d43= 751990599/src/libFLAC/stream_decoder.c#L2141-L2147">https://github.com/xi= ph/flac/blob/28e4f0528c76b296c561e922ba67d43751990599/src/libFLAC/stream_= decoder.c#L2141-L2147</a> As you can see these lines are also disabled = on fuzzing.<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p class=3DMsoNormal>I = think providing an API function and flac+metaflac command line option is = going to be confusing and they will probably be misused. A compile time = option might be possible though, libogg provides that = too.<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>Kind regards,<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>Martijn van Beurden<o:p></o:p></p></div></div><p = class=3DMsoNormal><o:p> </o:p></p><div><div><p class=3DMsoNormal>Op = do 29 jun. 2023 22:28 schreef Manuel Bertsch <<a = href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a>>:<o:p></o:p></p></div><bl= ockquote style=3D'border:none;border-left:solid #CCCCCC = 1.0pt;padding:0cm 0cm 0cm = 6.0pt;margin-left:4.8pt;margin-right:0cm'><div><div><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hello = all,<o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p><= /o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US>I=E2=80=99m trying to use flac and metaflac as tooling to = analyze the generated FLAC files of a new encoder implementation. = However, while my own third-party decoder can handle the output just = fine, flac, metaflac, and other tools discard the output since the frame = checksum (footer CRC-16) is invalid. This is expected, since I have no = checksum generation implemented apart from the frame header checksum = (all other checksums are written as 0), but it prevents me from making = development process without being able to check my outputs against the = reference implementation. I looked around and I couldn=E2=80=99t find a = way to turn off either tool=E2=80=99s checksum verification. So = I=E2=80=99d like to know how one could turn off checksum verification in = the tools, and if that=E2=80=99s not currently possible, I=E2=80=99d = like to see that as a feature in a future version of flac. (In that case = I would also try to disable the checks in code permanently, though my = solution is most likely not going to be upstreamable in any = capacity.)</span><o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US>FYI, the third-party decoder is highly compliant according = to the spec test suite, producing identical audio playback on almost all = of the files, but it also doesn=E2=80=99t verify or error on any = checksums. ffplay and ffmpeg both accept the file and can play and = transcode it correctly, although they ignore the sample count in the = streaminfo header and will read all frames they can find. The latter = leads me to believe that checksums are the actual issue, even though = flac reports the generic =E2=80=9CBad Header=E2=80=9D and =E2=80=9CLost = Sync=E2=80=9D.</span><o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US>Greetings and thanks in advance,</span><o:p></o:p></p><p = class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US> </span><o:p></o:p></p><p class=3DMsoNormal = style=3D'mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span = lang=3DEN-US>Kleines = Filmr=C3=B6llchen</span><o:p></o:p></p></div></div><p = class=3DMsoNormal>_______________________________________________<br>flac= -dev mailing list<br><a href=3D"mailto:[email protected]" = target=3D"_blank">[email protected]</a><br><a = href=3D"http://lists.xiph.org/mailman/listinfo/flac-dev" = target=3D"_blank">http://lists.xiph.org/mailman/listinfo/flac-dev</a><o:p= ></o:p></p></blockquote></div></div></div></body></html> ------=_NextPart_000_0003_01D9AB51.218CDA40-- --===============3038988820503109178== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ flac-dev mailing list [email protected] http://lists.xiph.org/mailman/listinfo/flac-dev --===============3038988820503109178==--