Re: [Csnd-dev] [EXTERNAL] [Csnd-dev] Unity

Yes unfortunately it doesn't work on mobile, the start promise is timed out.

This is the log:

TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet,
Single-threaded
TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb 17
2023
TestBuild_0.framework.js:383 [commit: HEAD]
TestBuild_0.framework.js:383 libsndfile-1.1.0
TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
TestBuild_0.framework.js:383 0dBFS level = 1.0, A4 tuning = 440.0
TestBuild_0.framework.js:383 orch now loaded
TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
TestBuild_0.framework.js:383 SECTION 1:
TestBuild_0.framework.js:374 start promise timed out

This is the stack of the promise timeout error:

(anonymous) @ TestBuild_0.framework.js:374
setTimeout (async)
(anonymous) @ TestBuild_0.framework.js:373
createStartPromise @ TestBuild_0.framework.js:373
a @ TestBuild_0.framework.js:898
_csoundInitialize @ TestBuild_0.framework.js:913
await in _csoundInitialize (async)
$func33262 @ 0670e506:0x97e895
$func34985 @ 0670e506:0x9abd39
$func36433 @ 0670e506:0x9c759b
$dynCall_iiii @ 0670e506:0x170a42a
invoke_iiii @ TestBuild_0.framework.js:913
$func36417 @ 0670e506:0x9c6632
$func1111 @ 0670e506:0xa6bf1
$func60024 @ 0670e506:0x16c471c
$func57642 @ 0670e506:0x1674cc4
$func38105 @ 0670e506:0xaabc30
$func38106 @ 0670e506:0xaabc45
$func57359 @ 0670e506:0x1664efe
$func57067 @ 0670e506:0x163f9af
$func38930 @ 0670e506:0xaf67e8
$func38930 @ 0670e506:0xaf685c
$func60744 @ 0670e506:0x16f7e08
$dynCall_v @ 0670e506:0x170a466
(anonymous) @ TestBuild_0.framework.js:913
_JS_CallAsLongAsNoExceptionsSeen @ TestBuild_0.framework.js:913
$func60497 @ 0670e506:0x16e3bc4
$dynCall_iiii @ 0670e506:0x170a42a
(anonymous) @ TestBuild_0.framework.js:913
touchEventHandlerFunc @ TestBuild_0.framework.js:913
jsEventHandler @ TestBuild_0.framework.js:913

What could be the cause?

About “HTTPS response Header”, I will investigate the cause, it should have
no bad effect though.

Il giorno gio 20 giu 2024 alle ore 14:36 Victor Lazzarini <
[email protected]> ha scritto:

> Tested it here on the Mac and the interaction is working well. On the
> mobile, I guess the touch screen interaction does not replace
> the mouse. I get the csoundInitialize dialogue, but that’s about it.
>
> On both I get this “HTTPS response Header” message, but that goes away
> after a while.
>
> ========================
> Prof. Victor Lazzarini
> Maynooth University
> Ireland
>
> > On 20 Jun 2024, at 11:45, Giovanni Bedetti <[email protected]>
> wrote:
> >
> > *Warning*
> > This email originated from outside of Maynooth University's Mail System.
> Do not reply, click links or open attachments unless you recognise the
> sender and know the content is safe.
> > I have updated the repo and the build, same links as above.
> > I have been able to create multiple instances of Csound from Unity,
> reference a specific instance and set a channel there, so I am making good
> progress!
> > I had to rely on callbacks so yes the method signature (and workflow)
> will probably be very different from the original CsoundUnity
> implementation.
> > At the moment there is no sound at all on the editor, so you'll have to
> build every time after a change.
> >
> >
> >
> >
> > Il giorno mer 19 giu 2024 alle ore 03:13 Dr. Richard Boulanger <
> [email protected]> ha scritto:
> > I too heard the sine tone in my browser - nice.
> > - now - a button and a fader in the browser will be something!
> >
> > - Dr.B
> >
> > Dr. Richard Boulanger
> > Professor
> > Electronic Production and Design
> > Berklee College of Music
> > Professional Writing & Technology Division
> >
> >
> > On Tue, Jun 18, 2024 at 9:12 PM Dr. Richard Boulanger <
> [email protected]> wrote:
> > wow - you are getting there.  this will be awesome too - an important
> next step.
> >
> > - Dr.B
> >
> > Dr. Richard Boulanger
> > Professor
> > Electronic Production and Design
> > Berklee College of Music
> > Professional Writing & Technology Division
> >
> >
> > On Tue, Jun 18, 2024 at 1:36 PM Giovanni Bedetti <
> [email protected]> wrote:
> > I made some progress, we have sound!
> > Check it out in the repo.
> > At this link you can test the working build! (it's safe I swear)
> >
> >
> >
> > Il giorno mar 18 giu 2024 alle ore 13:56 Giovanni Bedetti <
> [email protected]> ha scritto:
> > I have no idea, this is what the docs say about this, no mention about
> future plans anywhere:
> > Because Unity uses FMODto manage audio for platforms, the Web platform
> supports limited audio functionality, which only includes the basic
> features. FMOD relies on threads, which the WebGL API doesn’t support. For
> this reason, Unity uses an implementation based on the internal Web Audio
> API, which enables the browser to handle audio playback and mixing.
> >
> > Il Mar 18 Giu 2024, 13:41 Rory Walsh <[email protected]> ha scritto:
> > Do they plan to add support for the OnAudioFilterRead, is it on any
> roadmaps? That's a fairly major part of things.
> >
> > On Tue 18 Jun 2024, 12:25 Giovanni Bedetti, <[email protected]>
> wrote:
> > Thanks for your replies!
> > Dr.B let's chat soon! I'll drop you an email ;)
> >
> > I'm getting closer! I was able to load Csound! Woho!
> >
> > I have created a repository with a test project and a build folder if
> anybody wants to take a look.
> >
> > I grabbed the 6.18.7 dist/csound.js file as Steven suggested, renamed it
> to .jspre and placed it inside Assets/Plugins.
> > If I build without any changes it builds ok, but as soon as the game
> starts on the browser I get this message:
> > Unable to parse Build/TestBuild_0.framework.js! The file is corrupt, or
> compression was misconfigured? (check Content-Encoding HTTP Response Header
> on web server)
> >
> > I tried using the other extension, .jslib, and I got this error instead:
> > Library/Bee/artifacts/WebGL/build/debug_WebGL_wasm/build.js: failure to
> execute js library "/.../TestCsoundUnityWebGL/Assets/Plugins/csound.jslib":
> SyntaxError: Unexpected token 'export',
> >
> > So I renamed the file back to csound.jspre, and commented the export
> lines at the end.
> >
> > const Csound = Csound$$module$src$index;
> > Csound.toString = () => 'async (options) => CsoundObj;';
> > //export { Csound }
> > //export default Csound
> > //# sourceMappingURL=csound.js.map
> >
> > It builds and runs ok, of course no Csound yet as nothing is calling it.
> >
> > So I added a .jslib file and wrote an initialize method (grabbed from
> the test.js in the 6.18.7 dist above)
> >
> >
> > mergeInto(LibraryManager.library, {
> >
> > csoundInitialize: async function(flags) {
> > window.alert("csoundInitialize");
> >
> > const csoundVariations = [
> > { useWorker: false, useSPN: false, name: "SINGLE THREAD, AW" },
> > { useWorker: false, useSPN: true, name: "SINGLE THREAD, SPN" },
> > { useWorker: true, useSAB: true, name: "WORKER, AW, SAB" },
> > { useWorker: true, useSAB: false, name: "WORKER, AW, Messageport" },
> > { useWorker: true, useSAB: false, useSPN: true, name: "WORKER, SPN,
> MessagePort" },
> > ];
> >
> > //Csound.csoundInitialize(flags); // this doesn't work
> > console.log("starting to await for Csound with flag: + flags")
> > const cs = await Csound(csoundVariations[flags]);
> > console.log(`Csound version: ${cs.name}`);
> > const startReturn = await cs.start();
> > console.log(startReturn);
> > // await cs.stop(); // cs.terminateInstance && (await
> cs.terminateInstance()); },
> > });
> >
> > Then call csoundInitialize from a C# script
> > void Start()
> > {
> > Debug.Log("Calling csoundInitialize");
> > csoundInitialize(0);
> > }
> >
> > and bang we got it!!
> >
> > TestBuild_0.framework.js:913 Calling csoundInitialize
> > TestBuild_0.framework.js:913 starting to await for Csound with flag: 0
> > TestBuild_0.framework.js:913 Csound version: Csound: Audio Worklet,
> Single-threaded
> > TestBuild_0.framework.js:383 --Csound version 6.18 (double samples) Feb
> 17 2023
> > TestBuild_0.framework.js:383 [commit: HEAD]
> > TestBuild_0.framework.js:383 libsndfile-1.1.0
> > TestBuild_0.framework.js:383 graphics suppressed, ascii substituted
> > TestBuild_0.framework.js:383 sr = 48000.0, kr = 3000.000, ksmps = 16
> > TestBuild_0.framework.js:383 0dBFS level = 32768.0, A4 tuning = 440.0
> > TestBuild_0.framework.js:383 orch now loaded
> > TestBuild_0.framework.js:383 audio buffered in 256 sample-frame blocks
> > TestBuild_0.framework.js:383 SECTION 1:
> > TestBuild_0.framework.js:913 0
> >
> > Only the SINGLE_THREAD csoundVariations print the Csound version, for
> the other 3 it says undefined, but the rest of the logs are still printed
> (sr and kr could differ).
> > Unity apparently doesn't support multi-threading on WebGL yet (this is
> an interesting post on the subject), so workers don't work? :D
> > I'll dig deeper inside this as soon as I have sound output.
> >
> > So let's get a bit into the details. It was "nice" that I was able to
> call an async method (csoundInitialize) from a non-async context (the
> Start() function).
> > But of course the Start function is not waiting for it to complete
> before progressing.
> >
> > All of C# methods in the CsoundUnity package are synchronous, and
> sometimes called in a sequence.
> > Have a look at the initialization sequence of the CsoundUnityBridge
> > So I guess that to integrate WebGL into the existing CsoundUnity package
> we will have to use a lot of directives to handle that and it could become
> hard to maintain.
> > I mean, the way we use the API in C# is very different from what we
> should do with the CsoundObj API.
> > Furthermore, promises don't exist in the .NET environment out of the
> box, so we should rely on an external package to have them in C#.
> > We could add callbacks on completion for the methods that need to be
> awaited but it doesn't look very stylish.
> > Should the CsoundUnityWebGL be a completely different package?
> >
> > The other very important thing to keep in mind is that the
> OnAudioFilterRead function (what we use to do business in the Audio Thread,
> grabbing audio data from Csound and writing it into the AudioSource buffer
> - and the opposite) is not triggered at all on WebGL, and it means that it
> is not triggered on the editor too (I was very surprised to discover this).
> > So you won't have the normal CsoundUnity behaviour on the Editor if the
> selected platform is WebGL. This means that the normal CsoundUnity workflow
> on the Editor on WebGL is not possible. I mean, the inspector with the
> control channels grabbed from the csd will still work, but pressing play on
> the Editor won't produce any sound.
> > I'm thinking of workarounds but have no ideas at the moment.
> >
> > But for now let's focus on the fact that there was some progress! It's a
> win! :D
> >
> > I'll keep you posted as I continue to experiment with this,
> > thanks again for now!
> >
> >
> > Il giorno dom 16 giu 2024 alle ore 17:59 Steven Yi <[email protected]>
> ha scritto:
> > Webassembly Csound already compiles to a single JS file that embeds
> > the WASM binary into it. I assume this would work with Unity but am
> > unsure. I think you would only need the dist/csound.js file. You could
> > also try using the one from 6.18.7:
> >
> > https://unpkg.com/browse/@csound/[email protected]/
> >
> > It might be possible to rename it .jspre and create a test .jslib that
> > uses the Csound API.
> >
> > On Sat, Jun 15, 2024 at 2:02 PM Giovanni Bedetti
> > <[email protected]> wrote:
> > >
> > > Quick update on this: no luck yet!
> > >
> > > Sorry for the long email that follows. Any help would be much
> appreciated!
> > >
> > >
> > > Unity uses Emscripten for the WebGL build.
> > >
> > > There are 2 options using Emscripten:
> > >
> > > Compile Csound as a static library to be used as Unity plug-in
> > > First thing I tried the old emscripten build for Web Csound from the
> master branch (6.18.1), but I couldn't build it on macOs with this error
> message:
> > >
> > > Error: Could not resolve './libcsound' from
> src/CsoundScriptProcessorNode.js
> > >
> > >
> > > I also tried older versions of emscripten, but I cannot install the
> ones older than version 2.0, so no luck there.
> > > This should be the preferred way of creating the required files for
> Unity.
> > >
> > > The supported native plug-in file formats are
> > >
> > > LLVM Bitcode file format (.bc)
> > > GNU archive file format (.a)
> > >
> > > I noticed that the Csound wasm build was exporting a file with that
> extension so I went this route.
> > > I was able to build the fix-csound-wasm branch (off CS7) and I can
> confirm that it works on macOS (commit bac483d).
> > > I ran yarn build in the wasm folder.
> > > In the wasm/lib folder I could find a "libcsound-wasm.a" file, that
> will be accepted by Unity as a native WebGL plug-in.
> > >
> > > I was hoping that the .a file would be enough, but it appears that I
> cannot reach its methods from C#, I guess I need some js glue code with the
> explicit exports.
> > > In fact, If I try to build the Unity project with only the above .a
> file it builds ok, but as soon as I import the CsoundUnity package (with a
> little change to try and load the library for WebGL) I get loads of build
> errors (one for each native method we have, like "error: undefined symbol:
> csoundAddSpinSample (referenced by top-level compiled C/C++ code").
> > >
> > > So I started to wonder if I needed the wasm/browser part to be able to
> use the above library.
> > > I tried to build the browser package. First of all I ran yarn link
> "@csound/wasm-bin" to link it with the previously built lib, then yarn
> build.
> > > When built, it creates a "dist" folder, with several .js and .map
> files with the name starting with "__compiled", like
> "__compiled.vanilla.worker.js", "__compiled.vanilla.worker.inline.js",
> "__compiled.vanilla.worker.js.map", together with
> "__csound_wasm.inline.js", "csound.js" and "csound.js.map".
> > > I'm not sure how to import those files in Unity, I have very little
> experience on the WebGL platform.
> > > Unity supports two JavaScript plug-in file types that let you add
> JavaScript code to your Unity project
> > >
> > > .jslib A JavaScript library to use in addition to those in
> Emscripten’s core libraries:
> emscripten.org/docs/tools_reference/emcc.html#emcc-js-library
> > > .jspre A a file whose contents are added before the emitted code:
> emscripten.org/docs/tools_reference/emcc.html#emcc-pre-js
> > >
> > > Furthermore, Unity expects this syntax to retrieve the native methods
> that can be used in C#:
> > >
> > > const hello = {
> > >     Hello: function() {
> > >         window.alert("Hello, world!");
> > >     }
> > > };
> > > mergeInto(LibraryManager.library, hello);
> > >
> > > to be called in C# like this:
> > >
> > > [DllImport("__Internal")]
> > >     private static extern void Hello();
> > >
> > > Bundle the C/C++ code in the Unity project
> > > This option is very undocumented and there's no mention of any build
> script or makefile to edit, so at the moment I have no idea how Csound will
> be built by Unity.
> > > Out of curiosity, I tried to import some folders from the Csound repo:
> > > - Engine
> > > - H
> > > - include
> > > - InOut
> > > - interfaces
> > > - OOps
> > > - Opcodes
> > > - po
> > > - util
> > > - util1
> > > - util2
> > >
> > > When I build I get the following error, "Plugins colliding with each
> other.":
> > >
> > > Plugin 'mp3dec_internal.h' is used from several locations:
> > >  Assets/Plugins/InOut/libmpadec/mp3dec_internal.h would be copied to
> <PluginPath>/mp3dec_internal.h
> > >  Assets/Plugins/H/mp3dec_internal.h would be copied to
> <PluginPath>/mp3dec_internal.h
> > > Plugin 'mpadec_internal.h' is used from several locations:
> > >  Assets/Plugins/H/mpadec_internal.h would be copied to
> <PluginPath>/mpadec_internal.h
> > >  Assets/Plugins/InOut/libmpadec/mpadec_internal.h would be copied to
> <PluginPath>/mpadec_internal.h
> > > Plugin 'stdopcod.h' is used from several locations:
> > >  Assets/Plugins/Opcodes/stdopcod.h would be copied to
> <PluginPath>/stdopcod.h
> > >  Assets/Plugins/Opcodes/gab/stdopcod.h would be copied to
> <PluginPath>/stdopcod.h
> > >
> > > I had a look at the two mp3dec_internal.h and they are identical
> except for one line, the Assets/Plugins/InOut/libmpadec/mp3dec_internal.h
> has an additional #include "mpadec_config.h".
> > > I'm not sure what to try next here, I guess removing one of the two
> folders is too simple to be an option :D but maybe I'll try it later!
> > >
> > > There is also the option to use a pure Javascript plugin, but how to
> create a javascript version of Csound?
> > >
> > > Last option, Unity native audio plugins, but they can't be used for
> WebGL, see this comment on the official Native plugins repo
> > >
> > > Hope someone has some tips here, I'm lost in the Web(GL)!
> > >
> > >
> > > Il giorno ven 31 mag 2024 alle ore 10:07 Giovanni Bedetti <
> [email protected]> ha scritto:
> > >>
> > >> Not yet, there are limitations in the Unity FMOD implementation for
> the WebGL platform. Specifically, it doesn't support the OnAudioFilterRead
> callback, which is the function we use to fill the Unity AudioSources
> content with the Csound output.
> > >> Technically though it shouldn't be hard to import a Csound library
> for the Web, and use the API. The difficulty is sending the Csound content
> to the AudioSources, and so being able to move those sources in the 3D
> space and having them spatialized (Unity spatializer is first order
> ambisonic).
> > >>
> > >> In the next days I will try something and report here my findings.
> > >>
> > >>
> > >> Il Ven 31 Mag 2024, 09:29 Michael Gogins <[email protected]>
> ha scritto:
> > >>>
> > >>> Is there WebAssembly support for Csound in Unity yet?
>
>