ethernet smp_affinity vs /proc/interrupts vs /sys/class/net/ethX/device

http://serverfault.com/questions/254355/ethernet-smp-affinity-vs-proc-interrupts-vs-sys-class-net-ethx-device

 

My goal is to configure our CentOS ("free" RHEL) 5.x servers for custom low-latency network programs. I would like to experiment with binding ethernet NIC interrupt handling to the same CPU on which the program runs (to hopefully improve cache utilization). The first step in this process is to determine the NIC‘s IRQ.

Here is the contents of /proc/interrupts on one server (note that I deleted CPUs 2 through 14 for brevity):

    CPU0 CPU1 CPU15 
 0: 600299726 0 0 IO-APIC-edge timer 
 1: 3 0 0 IO-APIC-edge i8042 
 8: 1 0 0 IO-APIC-edge rtc 
 9: 0 0 0 IO-APIC-level acpi 
 12: 4 0 0 IO-APIC-edge i8042 
 50: 0 0 0 IO-APIC-level uhci_hcd:usb6, uhci_hcd:usb8 
 58: 6644 25103 0 IO-APIC-level ioc0 
 66: 0 0 0 IO-APIC-level ata_piix 
 74: 221 533830 0 IO-APIC-level ata_piix 
 98: 35 0 2902361 PCI-MSI-X eth1-0 
106: 61 11 3841 PCI-MSI-X eth1-1 
114: 28 0 61452 PCI-MSI-X eth1-2 
122: 24 1586 22 PCI-MSI-X eth1-3 
130: 2912 0 337 PCI-MSI-X eth1-4 
138: 21 0 28 PCI-MSI-X eth1-5 
146: 21 0 56 PCI-MSI-X eth1-6 
154: 34 1 1 PCI-MSI-X eth1-7 
209: 23 0 0 IO-APIC-level ehci_hcd:usb1 
217: 0 0 0 IO-APIC-level ehci_hcd:usb2, uhci_hcd:usb5, uhci_hcd:usb7 
225: 0 0 0 IO-APIC-level uhci_hcd:usb3 
233: 0 0 0 IO-APIC-level uhci_hcd:usb4 
NMI: 7615 2989 2931 LOC: 600328144 600328099 600327086 
ERR: 0 
MIS: 0

Why are there multiple entries for "eth1" in the form of "eth1-X"?

Furthermore, the contents of "/sys/class/net/eth1/device/irq" is "90". But there‘s no 90 in the interrupt list above.

So let‘s say I look at just "eth1-0", which is IRQ 98. The contents of /proc/irq/98/smp_affinity is:

00000000,00000000,00000000,00000000,00000000,00000000,00000000,00008000

That‘s a list of numbers, rather than just one number.

So how do I set eth1‘s smp_affinity?

None of the online examples and documentation I could find mentioned any cases like this; they always have exactly one "ethX" entry in /proc/interrupts; the indicated interrupt matches /sys/class/net/ethX/device/irq; and there is only one number in /proc/irq/N/smp_affinity.

FWIW, I‘ll add that this application is extremely latency sensitive. To the point where we disable C-states and processor frequency scaling (because those features induce too much latency). Micro seconds make a difference here.

郑重声明:本站内容如果来自互联网及其他传播媒体,其版权均属原媒体及文章作者所有。转载目的在于传递更多信息及用于网络分享,并不代表本站赞同其观点和对其真实性负责,也不构成任何其他建议。