Re: [PATCH v2 07/50] helper-to-tcg: Introduce get-llvm-ir.py

On Thu, 30 Jul 2026 05:09:41 +0200
Anton Johansson via qemu development <[email protected]> wrote:

> ---
>  subprojects/helper-to-tcg/get-llvm-ir.py | 145 +++++++++++++++++++++++
>  subprojects/helper-to-tcg/meson.build    |   8 ++
>  2 files changed, 153 insertions(+)
>  create mode 100755 subprojects/helper-to-tcg/get-llvm-ir.py
> 
> diff --git a/subprojects/helper-to-tcg/get-llvm-ir.py b/subprojects/helper-to-tcg/get-llvm-ir.py
> new file mode 100755
> index 0000000000..982b87f791
> --- /dev/null
> +++ b/subprojects/helper-to-tcg/get-llvm-ir.py
> @@ -0,0 +1,145 @@
> +#!/usr/bin/env python3
> +
> +##
> +##  Copyright(c) 2026 rev.ng Labs Srl. All Rights Reserved.
> +##
> +##  This program is free software; you can redistribute it and/or modify
> +##  it under the terms of the GNU General Public License as published by
> +##  the Free Software Foundation; either version 2 of the License, or
> +##  (at your option) any later version.
> +##
> +##  This program is distributed in the hope that it will be useful,
> +##  but WITHOUT ANY WARRANTY; without even the implied warranty of
> +##  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +##  GNU General Public License for more details.
> +##
> +##  You should have received a copy of the GNU General Public License
> +##  along with this program; if not, see <http://www.gnu.org/licenses/>.
> +##
> +
> +import argparse
> +import json
> +import os
> +import shlex
> +import sys
> +import subprocess
> +
> +
> +def log(msg):
> +    print(msg, file=sys.stderr)
> +
> +
> +def run_command(command):
> +    proc = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
> +    out = proc.communicate()
> +    if proc.wait() != 0:
> +        log(f"Command: {' '.join(command)} exited with {proc.returncode}\n")
> +        log(f"output:\n{out}\n")

The whole program should fail if a subcommand fails, printing about it
is not enough.

> +
> +def find_compile_commands(compile_commands_path, clang_path, input_path, target):
> +    with open(compile_commands_path, "r") as f:
> +        compile_commands = json.load(f)
> +        for compile_command in compile_commands:
> +            path = compile_command["file"]
> +            if os.path.basename(path) != os.path.basename(input_path):
> +                continue
> +
> +            os.chdir(compile_command["directory"])
> +            command = compile_command["command"]
> +
> +            # If building multiple targets there's a chance
> +            # input files share the same path and name.
> +            # This could cause us to find the wrong compile
> +            # command, we use the target path to distinguish
> +            # between these.
> +            if not target in command:
> +                continue
> +
> +            argv = shlex.split(command)
> +            argv[0] = clang_path
> +
> +            return argv
> +
> +    raise ValueError(f"Unable to find compile command for {input_path}")
> +
> +
> +def generate_llvm_ir(
> +    compile_commands_path, clang_path, output_path, input_path, target
> +):
> +    command = find_compile_commands(
> +        compile_commands_path, clang_path, input_path, target
> +    )
> +
> +    flags_to_remove = {
> +        "-ftrivial-auto-var-init=zero",
> +        "-fzero-call-used-regs=used-gpr",
> +        "-Wimplicit-fallthrough=2",
> +        "-Wold-style-declaration",
> +        "-Wno-psabi",
> +        "-Wshadow=local",
> +        "-c",
> +    }

How did you come up with this list?
We should put some indication to make its maintenance easier.
For instance, explicitly mention what part of QEMU introduced it, so
one can easily check if it's still relevant or if it disappeared.

> +
> +    # Remove
> +    #   - output of makefile rules (-MQ,-MF target);
> +    #   - output of object files (-o target);
> +    #   - excessive zero-initialization of block-scope variables
> +    #     (-ftrivial-auto-var-init=zero);
> +    #   - and any optimization flags (-O).
> +    for i, arg in reversed(list(enumerate(command))):
> +        if arg in {"-MQ", "-o", "-MF"}:
> +            del command[i : i + 2]
> +        elif arg.startswith("-O") or arg in flags_to_remove:
> +            del command[i]
> +
> +    # Define a HELPER_TO_TCG macro for translation units wanting to
> +    # conditionally include or exclude code during translation to TCG.
> +    # Disable optimization (-O0) and make sure clang doesn't emit optnone
> +    # attributes (-disable-O0-optnone) which inhibit further optimization.
> +    # Optimization will be performed at a later stage in the helper-to-tcg
> +    # pipeline.
> +    command += [
> +        "-S",
> +        "-emit-llvm",
> +        "-DHELPER_TO_TCG_IR_GEN",
> +        "-O0",
> +        "-g",
> +        "-Xclang",
> +        "-disable-O0-optnone",
> +    ]
> +    if output_path:
> +        command += ["-o", output_path]
> +
> +    run_command(command)
> +
> +
> +def main():
> +    parser = argparse.ArgumentParser(
> +        description="Produce the LLVM IR of a given .c file."
> +    )
> +    parser.add_argument(
> +        "--compile-commands", required=True, help="Path to compile_commands.json"
> +    )

I'm not super happy that we use `compile_commands.json`, however:

1. `compile_commands.json` is emitted unconditionally by meson at build
   time.

2. There's no good alternative. In rev.ng we configure a dedicated
   (throwaway) clang QEMU build with `-fembed-bitcode` and then extract
   the IR downstream.

   Maybe here we could manage to build a set of sources with
   `-fembed-bitcode`, but that won't work unless the compiler is clang.

   We could configure a temporary clang QEMU build in a subdirectory,
   but that's not very nice.

So, in the end, this makes sense to me.

The proper solution would to have `meson` provide the full invocation
used to produce a certain object file that we can then manipulate, but
AFAIU there's not such a feature.

One day we could maybe explore adjusting meson, but I'd say not today.

> +    parser.add_argument("--clang", default="clang", help="Path to clang.")
> +    parser.add_argument("--llvm-link", default="llvm-link", help="Path to llvm-link.")
> +    parser.add_argument("-o", "--output", required=True, help="Output .ll file path")
> +    parser.add_argument(
> +        "--target-path", help="Path to QEMU target dir. (e.q. target/i386)"
> +    )
> +    parser.add_argument("inputs", nargs="+", help=".c file inputs")
> +    args = parser.parse_args()
> +
> +    outputs = []
> +    for input in args.inputs:
> +        output = os.path.basename(input) + ".ll"
> +        generate_llvm_ir(
> +            args.compile_commands, args.clang, output, input, args.target_path
> +        )
> +        outputs.append(output)
> +
> +    run_command([args.llvm_link] + outputs + ["-S", "-o", args.output])

Maybe we should emit bitcode (binary form) instead of textual LLVM IR.
If you go this route, don't forget to rename the output to `.bc`.

I know this is easy for debugging, but bitcode is what one should use
"in production" and you're one `opt -S` away from getting the textual
IR again.

> +
> +
> +if __name__ == "__main__":
> +    sys.exit(main())
> diff --git a/subprojects/helper-to-tcg/meson.build b/subprojects/helper-to-tcg/meson.build
> index 8ab58adb39..97bce186fe 100644
> --- a/subprojects/helper-to-tcg/meson.build
> +++ b/subprojects/helper-to-tcg/meson.build
> @@ -42,6 +42,14 @@ endif
>  sources = [
>  ]
>  
> +clang = bindir / 'clang'
> +llvm_link = bindir / 'llvm-link'
> +
> +get_llvm_ir_cmd = [python, meson.current_source_dir() / 'get-llvm-ir.py',
> +                   '--compile-commands', 'compile_commands.json',
> +                   '--clang', clang,
> +                   '--llvm-link', llvm_link]
> +
>  # NOTE: Add -Wno-template-id-cdtor for GCC versions >= 14.  This warning is
>  # related to a change in the C++ standard in C++20, that also applies to C++14
>  # for some reason. See defect report DR2237 and commit
> -- 
> 2.52.0

Reviewed-by: Alessandro Di Federico <[email protected]>

-- 
Alessandro Di Federico
rev.ng Labs