@@ -6,6 +6,7 @@ S/390 PCI
66
77Authors:
88 - Pierre Morel
9+ - Niklas Schnelle
910
1011Copyright, IBM Corp. 2020
1112
@@ -27,14 +28,16 @@ Command line parameters
2728debugfs entries
2829---------------
2930
30- The S/390 debug feature (s390dbf) generates views to hold various debug results in sysfs directories of the form:
31+ The S/390 debug feature (s390dbf) generates views to hold various debug results
32+ in sysfs directories of the form:
3133
3234 * /sys/kernel/debug/s390dbf/pci_*/
3335
3436For example:
3537
3638 - /sys/kernel/debug/s390dbf/pci_msg/sprintf
37- Holds messages from the processing of PCI events, like machine check handling
39+
40+ holds messages from the processing of PCI events, like machine check handling
3841 and setting of global functionality, like UID checking.
3942
4043 Change the level of logging to be more or less verbose by piping
@@ -47,87 +50,134 @@ Sysfs entries
4750
4851Entries specific to zPCI functions and entries that hold zPCI information.
4952
50- * /sys/bus/pci/slots/XXXXXXXX
53+ * /sys/bus/pci/slots/XXXXXXXX:
5154
52- The slot entries are set up using the function identifier (FID) of the
53- PCI function. The format depicted as XXXXXXXX above is 8 hexadecimal digits
54- with 0 padding and lower case hexadecimal digits.
55+ The slot entries are set up using the function identifier (FID) of the PCI
56+ function as slot name . The format depicted as XXXXXXXX above is 8 hexadecimal
57+ digits with 0 padding and lower case hexadecimal digits.
5558
5659 - /sys/bus/pci/slots/XXXXXXXX/power
5760
5861 A physical function that currently supports a virtual function cannot be
5962 powered off until all virtual functions are removed with:
60- echo 0 > /sys/bus/pci/devices/XXXX :XX: XX.X /sriov_numvf
63+ echo 0 > /sys/bus/pci/devices/DDDD :BB: dd.f /sriov_numvf
6164
62- * /sys/bus/pci/devices/XXXX :XX: XX.X/
65+ * /sys/bus/pci/devices/DDDD :BB: dd.f/:
6366
64- - function_id
65- A zPCI function identifier that uniquely identifies the function in the Z server.
67+ - function_id:
68+ The zPCI function identifier (FID) is a 32-bit hexadecimal value that
69+ uniquely identifies the PCI function. Unless the hypervisor provides
70+ a virtual FID e.g. on KVM this identifier is unique across the machine even
71+ between different partitions.
6672
67- - function_handle
68- Low-level identifier used for a configured PCI function.
69- It might be useful for debugging.
73+ - function_handle:
74+ This 32-bit hexadecimal value is a low-level identifier used for a PCI
75+ function. Note that the function handle may be changed and become invalid
76+ on PCI events and when enabling/disabling the PCI function.
7077
71- - pchid
72- Model-dependent location of the I/O adapter.
78+ - pchid:
79+ This 16-bit hexadecimal value encodes a model-dependent location for
80+ the PCI function.
7381
74- - pfgid
75- PCI function group ID, functions that share identical functionality
82+ - pfgid:
83+ PCI function group ID; functions that share identical functionality
7684 use a common identifier.
7785 A PCI group defines interrupts, IOMMU, IOTLB, and DMA specifics.
7886
79- - vfn
87+ - vfn:
8088 The virtual function number, from 1 to N for virtual functions,
8189 0 for physical functions.
8290
83- - pft
84- The PCI function type
85-
86- - port
87- The port corresponds to the physical port the function is attached to.
88- It also gives an indication of the physical function a virtual function
89- is attached to.
90-
91- - uid
92- The user identifier (UID) may be defined as part of the machine
93- configuration or the z/VM or KVM guest configuration. If the accompanying
94- uid_is_unique attribute is 1 the platform guarantees that the UID is unique
95- within that instance and no devices with the same UID can be attached
96- during the lifetime of the system.
97-
98- - uid_is_unique
99- Indicates whether the user identifier (UID) is guaranteed to be and remain
100- unique within this Linux instance.
101-
102- - pfip/segmentX
91+ - pft:
92+ The PCI function type is an s390-specific type attribute. It indicates
93+ a more general, usage oriented, type than PCI Specification
94+ class/vendor/device identifiers. That is PCI functions with the same pft
95+ value may be backed by different hardware implementations. At the same time
96+ apart from unclassified functions (pft is 0x00) the same pft value
97+ generally implies a similar usage model. At the same time the same
98+ PCI hardware device may appear with different pft values when in a
99+ different usage model. For example NETD and NETH VFs may be implemented
100+ by the same PCI hardware device but in NETD the parent Physical Function
101+ is user managed while with NETH it is platform managed.
102+
103+ Currently the following PFT values are defined:
104+
105+ - 0x00 (UNC): Unclassified
106+ - 0x02 (ROCE): RoCE Express
107+ - 0x05 (ISM): Internal Shared Memory
108+ - 0x0a (ROC2): RoCE Express 2
109+ - 0x0b (NVMe): NVMe
110+ - 0x0c (NETH): Network Express hybrid
111+ - 0x0d (CNW): Cloud Network Adapter
112+ - 0x0f (NETD): Network Express direct
113+
114+ - port:
115+ The port is a decimal value corresponding to the physical port the function
116+ is attached to. Virtual Functions (VFs) share the port with their parent
117+ Physical Function (PF). A value of 0 indicates that the port attribute is
118+ not applicable for that PCI function type.
119+
120+ - uid:
121+ The user-defined identifier (UID) for a PCI function is a 32-bit
122+ hexadecimal value. It is defined on a per instance basis as part of the
123+ partition, KVM guest, or z/VM guest configuration. If UID Checking is
124+ enabled the platform ensures that the UID is unique within that instance
125+ and no two PCI functions with the same UID will be visible to the instance.
126+
127+ Independent of this guarantee and unlike the function ID (FID) the UID may
128+ be the same in different partitions within the same machine. This allows to
129+ create PCI configurations in multiple partitions to be identical in the
130+ UID-namespace.
131+
132+ - uid_is_unique:
133+ A 0 or 1 flag indicating whether the user-defined identifier (UID) is
134+ guaranteed to be and remain unique within this Linux instance. This
135+ platform feature is called UID Checking.
136+
137+ - pfip/segmentX:
103138 The segments determine the isolation of a function.
104139 They correspond to the physical path to the function.
105140 The more the segments are different, the more the functions are isolated.
106141
142+ - fidparm:
143+ Contains an 8-bit-per-PCI function parameter field in hexadecimal provided
144+ by the platform. The meaning of this field is PCI function type specific.
145+ For NETH VFs a value of 0x01 indicates that the function supports
146+ promiscuous mode.
147+
148+ * /sys/firmware/clp/uid_checking:
149+
150+ In addition to the per-device uid_is_unique attribute this presents a
151+ global indication of whether UID Checking is enabled. This allows users
152+ to check for UID Checking even when no PCI functions are configured.
153+
107154Enumeration and hotplug
108155=======================
109156
110157The PCI address consists of four parts: domain, bus, device and function,
111- and is of this form: DDDD:BB: dd.f
158+ and is of this form: DDDD:BB: dd.f.
112159
113- * When not using multi-functions (norid is set, or the firmware does not
114- support multi-functions):
160+ * For a PCI function for which the platform does not expose the RID, the
161+ pci=norid kernel parameter is used, or a so-called isolated Virtual Function
162+ which does have RID information but is used without its parent Physical
163+ Function being part of the same PCI configuration:
115164
116165 - There is only one function per domain.
117166
118- - The domain is set from the zPCI function's UID as defined during the
119- LPAR creation.
167+ - The domain is set from the zPCI function's UID if UID Checking is on;
168+ otherwise the domain ID is generated dynamically and is not stable
169+ across reboots or hot plug.
120170
121- * When using multi-functions (norid parameter is not set),
122- zPCI functions are addressed differently :
171+ * For a PCI function for which the platform exposes the RID and which
172+ is not an Isolated Virtual Function :
123173
124174 - There is still only one bus per domain.
125175
126- - There can be up to 256 functions per bus.
176+ - There can be up to 256 PCI functions per bus.
127177
128- - The domain part of the address of all functions for
129- a multi-Function device is set from the zPCI function's UID as defined
130- in the LPAR creation for the function zero .
178+ - The domain part of the address of all functions within the same topology is
179+ that of the configured PCI function with the lowest devfn within that
180+ topology .
131181
132- - New functions will only be ready for use after the function zero
133- (the function with devfn 0) has been enumerated .
182+ - Virtual Functions generated by an SR-IOV capable Physical Function only
183+ become visible once SR-IOV is enabled .
0 commit comments