[Question] Is there a way to specify an offset to the loaded symbols?

Hello,
for some reason that is not important here I need to debug a binary that is executed by calling the linker manually.
In my case I do `/lib/ld-musl-x86_64.so.1 binary`.
The problem with this is that gdb now obviously thinks that I want to debug `/lib/ld-musl-x86_64.so.1`. This is not the case.
Loading the symbols from `binary` does not help since the the linker loads the process into memory and the jumps to its main.

In other words I have the symbols for the binary to be debugged but the binary is loaded into memory with an offset so the symbols are offseted.
Is there any way to get this working?

Thank you!
Marten Ringwelski