Re: L4Re: Allocating pinned memory and obtaining its physical address

On Tuesday, 23 May 2023 16:03:28 CEST Philipp Eppelt wrote:
> 
> The path you are on sounds correct. A reason for a missing DMA domain 
> resource on the vbus is (usually) the lack of a DMA capable device on 
> said vbus.
> 
> https://github.com/kernkonzept/manifest/wiki/NVMeWithLinux#vbus-configuration-file
>
> shows a vbus configuration file that places all PCI/storage devices on 
> the vbus. I presume your MIPS board lacks PCI, so you have to write the 
> entries for System_bus and the Vbus yourself.

Yes, it involves a system-on-chip (SoC) without the usual PC architecture 
technologies like PCI.

> The helpful sections in another tutorial are:
> https://github.com/kernkonzept/manifest/wiki/HwPassThrough#ios-config-file
> https://github.com/kernkonzept/manifest/wiki/HwPassThrough#vbus-configuration-file

This is pretty similar to, but clearer than, the previous L4Re documentation 
about Io:

https://l4re.org/doc/io.html

> I found your mercurial instance for the l4re topic but didn't find any 
> io & vbus configuration. Can you point me to your config files?
> And/Or send along the Io output with maximum verbosity (-vvvvvvv)?

Thank you for going to the trouble of locating my repositories!

Currently, there is no single location of the files for this effort, but I aim 
to tidy this up somewhat. To explain briefly, there is the set of patches to 
get the software working on these MIPS-based boards (which is thankfully 
diminishing over time), and then there is a framework called Landfall which 
contains example programs. You can see the .io file for the specific example 
program here:

https://hg.boddie.org.uk/Landfall/file/7aa21a758551/conf/landfall-examples/
mips-ci20-hdmi-i2c.io

Since it is short, here is the essential part:

----
local hw = Io.system_bus()

local bus = Io.Vi.System_bus
{
  CPM = wrap(hw:match("jz4780-cpm"));
  GPIO = wrap(hw:match("jz4780-gpio"));
  LCD = wrap(hw:match("jz4780-lcd"));
  HDMI = wrap(hw:match("jz4780-hdmi"));
}

Io.add_vbus("vbus", bus)
----

Meanwhile, the underlying board-level file is actually defined in a patch 
against L4Re, and it isn't currently in a public repository, but it just looks 
like this:

----
local Res = Io.Res
local Hw = Io.Hw

Io.hw_add_devices(function()

  CPM = Hw.Device(function()
    Property.hid = "jz4780-cpm";
    -- compatible = {"mips,jz4780-cpm"};
    Resource.regs = Res.mmio(0x10000000, 0x10000fff);
    -- Property.exclk_freq = 48000000; -- 48 MHz
    -- Property.rtclk_freq = 32768; -- 32.768 kHz
  end);

  DMA = Hw.Device(function()
    Property.hid = "jz4780-dma";
    Resource.regs = Res.mmio(0x13420000, 0x1342103f);
    Resource.irq = Res.irq({59, 56}, Io.Resource.Irq_type_level_high);
  end);

  ...

end)
----

Note that the DMA device in the above is actually the DMA controller 
peripheral which is used to perform DMA involving the different SoC 
peripherals.

What I have tried to do in the .io file for the specific program is the 
following before the final statement adding the new bus as a vbus:

----
for resource in hw:resources() do
  print("resource name=" .. resource:id());
  if resource:id() == 1145130308 then
    bus:add_resource(resource)
  end
end
----

Here, I can confirm that there are two resources with the appropriate 
designation (1145130308, or 0x44414d44, or "DMAD"), and the trace from Io 
shows that the resource is copied to the vbus:

IO      | vbus: [N12_GLOBAL__N_112Virtual_sbusE]
IO      |   Resources: ==== start ====
IO      |   DMADOM  [00000000000000-00000000000000 1] (align=0 flags=6)
IO      |   Resources: ===== end =====
IO      |   L4ICU: [N2Vi6Sw_icuE]
IO      |     Resources: ==== start ====
IO      |     Resources: ===== end =====
IO      |   HDMI: [N2Vi9Proxy_devE]
IO      |     Resources: ==== start ====
IO      |     IOMEM   [00000010180000-0000001019ffff 20000] 32-bit non-pref 
(align=1ffff flags=300002)
IO      |     IRQ     [00000000000003-00000000000004 2] level high (align=1 
flags=300001)
IO      |     Resources: ===== end =====

But the DMA domain is still not found when assigning the space. I did wonder 
whether I might need to create a device to hold the DMA domain resource, but 
my attempt to implement this failed:

for resource in hw:resources() do
  print("resource name=" .. resource:id())
  if resource:id() == 1145130308 then
    device = Io.Vi.System_bus()
    device:add_resource(resource)
    bus:add_child(device)
  end
end

Here, Io crashes...

IO      | L4Re[rm]: unhandled read page fault at 0x31 pc=0x102aba0
IO      | L4Re: rom/io: Unhandled exception: PC=0x102aba0 PFA=0x30 LdrFlgs=0x0

...which is the following line in Dma_domain::add_to_group:

  if (!_v_domain && !g->_set)

I imagine that I am making this harder than it should be, though.

Paul