Boot hangs "Probing PCI hardware"

Dear All:

I've found RHL7.2 boot sometimes hangs with "Probing PCI hardware" message
on our Itanium based server.

I also conclude it is a kernel problem in PCI device initialization.

There is the following architectural difference between IA-32 and IA-64.
IA-64 has SAPIC for interrupts and its messages are generated by Memory write 
transaction to System reserved region on PCI Bus, instead of using APIC and 
APIC Bus in IA-32.

Therefore, from this point of view, BAR(Base Address Register) opertions in PCI
device initialization must be changed between IA-32 and IA-64, but it has not 
actually revised yet.

I'll explain that in detail below and also provide the patch. If somebody 
reflect it in Kernel source tree, I would be appreciated so much. I also hope 
the next Kernel-2.4.19 includes following fixes.

Our machine has two Itaniums (733MHz 2MB cache) and 512MB RAM. It also has
three PCI Buses and the problem occurs in case of putting a video card (ATI Rage
128 GL) on the first PCI Bus which also has SAPIC.

I've also found it occurs on RHL7.1(Kernel-2.4.3-12), RHL7.2(Kernel-2.4.9-18)
and the latest Kernel-2.4.18.

Next, I'll describe the error sequence.

There are some BAR operations in pci_read_bases() function of pci.c to get memory
and IO space ranges assigned for each PCI device by writing 0xffffffff on BAR and
reading it again. If an interrupt from SAPIC occurs during BAR operations, the 
first PCI Bus hangs.

In this case, the video device gets 64MB memory space from 0xFC000000 to 
0xFFFFFFFF improperly, and it conflicts with System reserve region (0xFCxxxxxx -
0xFExxxxxx) for SAPIC interrupt messages. After that the video device reacts to 
an SAPIC interrupt improperly.

According to PCI specifications, it is necessary to disable Memory IO target
function during BAR operations by setting zero to bit0/1 of PCI_COMMAND, but
current pci_read_bases() doesn't do that.

I'd like to provide the following patch, and I've already found it works well 
on both RHL7.2 and Kernel-2.4.18.


*** pci.c.orig  Tue Apr  2 19:56:06 2002
--- pci.c       Tue Apr  2 19:56:14 2002
***************
*** 939,946 ****
--- 939,950 ----
  {
        unsigned int pos, reg, next;
        u32 l, sz;
+       u16 pcicmd_sv;
        struct resource *res;

+       pci_read_config_word(dev, PCI_COMMAND, &pcicmd_sv);
+       pci_write_config_word(dev, PCI_COMMAND, pcicmd_sv & 0xfffc);
+
        for(pos=0; pos<howmany; pos = next) {
                next = pos+1;
                res = &dev->resource[pos];
***************
*** 1004,1009 ****
--- 1008,1014 ----
                }
                res->name = dev->name;
        }
+       pci_write_config_word(dev, PCI_COMMAND, pcicmd_sv);
  }

  void __devinit  pci_read_bridge_bases(struct pci_bus *child)


Finally, I think this problem may occur theoretically in case of having memory 
consuming devices (more than 32MB) like a video card and SAPIC on the same PCI
Bus. Does anybody know the same problem on a Lion box ? Any comments especially
including how to avoid it would be appreciated.

Best regards,
Tatsuya Tsurukawa
Mitsubishi Electric

----
Tatsuya   [email protected]