Skip to content

Commit

Permalink
Updated v2023.2.0 Linux kernel documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
raghusha-xlnx committed Sep 29, 2023
1 parent d2a5f2c commit 2aa255b
Show file tree
Hide file tree
Showing 36 changed files with 335 additions and 113 deletions.
35 changes: 33 additions & 2 deletions master/QDMA/linux-kernel/html/_sources/dma-latency.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,42 @@
DMA Latency Application (dma-latency)
*************************************

Xilinx-developed custom tool ``dma-latency`` is used to collect the latency metrics for unidirectional and bidirectional traffic.
Xilinx-developed custom tool ``dma-latency`` is used to collect the latency metrics for unidirectional and bidirectional traffic.

::

usage: dma-latency [OPTIONS

-c (--config) config file that has configuration for IO

Example

::

[xilinx@]# dma-latency -c dmalat.conf
dma-ctl qdma06000 global_csr | grep Global| cut -d : -f 2 > glbl_rng_sz
qdma06000-ST-0 H2C added.
Added 1 Queues.
1 Queues started, idx 0 ~ 0.
qdma06000-ST-0 C2H added.
Added 1 Queues.
1 Queues started, idx 0 ~ 0.
dmautils(1) threads
Stopped Queues 0 -> 0.
Deleted Queues 0 -> 0.
Stopped Queues 0 -> 0.
Deleted Queues 0 -> 0.
qdma06000:statistics
Total MM H2C packets processed = 0
Total MM C2H packets processed = 0
Total ST H2C packets processed = 2288284
Total ST C2H packets processed = 2288284
Min Ping Pong Latency = 0
Max Ping Pong Latency = 0
Avg Ping Pong Latency = 0
[xilinx@]#


dma-latency tool takes a configuration file as input. The configuration file format is as below.

::
Expand All @@ -30,6 +58,9 @@ dma-latency tool takes a configuration file as input. The configuration file for
bidir_en=1
num_pkt=64
pkt_sz=64
pci_bus=06
pci_device=00



**Parameters**
Expand All @@ -49,4 +80,4 @@ dma-latency tool takes a configuration file as input. The configuration file for
- num_threads : number of threads to be used in dma-perf application to pump the traffic to queues
- bidir_en : Enable or Disable the bi-direction mode \( 0: Disable, 1: Enable \)
- num_pkt : number of packets
- pkt_sz : Packet size
- pkt_sz : Packet size
12 changes: 11 additions & 1 deletion master/QDMA/linux-kernel/html/_sources/dma_xfer_app.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,16 @@ Testing/Creation of cmpt queues is outside the scope of this application as tran
-v (--version), to print version name


Example

::

[xilinx@]# dma-xfer -c sample_qdma_xfer_config.txt
PF :0 Queue :0 H2C Async transfer success
PF :0 Queue :0 C2H ASync transfer success



dma-xfer application takes a configuration file as input. The configuration file format is as below.

::
Expand Down Expand Up @@ -85,4 +95,4 @@ dma-xfer application takes a configuration file as input. The configuration file
- pkt_sz : packet size, MM mode: <1 – 512MB> ST mode: <1 – 28KB>.
- io_type: synchronous or asynchronous DMA transfers <io_sync, io_async>.
- inputfile: input file name from where Host to Card dma data taken from <file name of length upto 127>.
- Outputfile: output file name to where Card to Host dma data to be written <file name of length upto 127>.
- Outputfile: output file name to where Card to Host dma data to be written <file name of length upto 127>.
53 changes: 51 additions & 2 deletions master/QDMA/linux-kernel/html/_sources/dmaperf.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
DMA Performance Tool (dma-perf)
*******************************

Xilinx-developed custom tool``dma-perf`` is used to collect the performance metrics for unidirectional and bidirectional traffic.
Xilinx-developed custom tool``dma-perf`` is used to collect the performance metrics for unidirectional and bidirectional traffic. This tool is used with AXI Stream Loopback Example Design only.

=====
Note:
=====
The name of the application in previous releases before 2020.1 was ``dmaperf``. If user installed ``dmaperf`` application aleady in ``/usr/local/sbin`` area, make sure to uninstall the old application(s). Using ``dmaperf`` aginst latest driver would lead to undefined behaviour and errors may be observed.

Standard IO tools such as ``fio`` can be used for performing IO operations using the char device interface.
Standard IO tools such as ``fio`` can be used for performing IO operations using the char device interface.

However, most of the tools are limited to sending / receiving 1 packet at a time and wait for the processing of the packet to complete, so they are not able to keep the driver/ HW busy enough for performance measurement. Although fio also supports asynchronous interfaces, it does not continuously submit IO requests while polling for the completion in parallel.

Expand All @@ -22,6 +22,55 @@ To overcome this limitation, Xilinx developed dma-perf tool. It leverages the as
-c (--config) config file that has configuration for IO


Example

::

[xilinx@]# dma-perf -c perf_config.txt
qdma65000-MM-0 H2C added.
Added 1 Queues.
Queues started, idx 0 ~ 0.
qdma65000-MM-0 C2H added.
Added 1 Queues.
Queues started, idx 0 ~ 0.
dmautils(16) threads
Exit Check: tid =8, req_sbmitted=1495488 req_completed=1495488 dir=H2C, intime=0 loop_count=0,
Exit Check: tid =13, req_sbmitted=1482752 req_completed=1482752 dir=C2H, intime=0 loop_count=0,
Exit Check: tid =14, req_sbmitted=1494720 req_completed=1494720 dir=H2C, intime=0 loop_count=0,
Exit Check: tid =8, req_sbmitted=1495488 req_completed=1495488 dir=H2C, intime=0 loop_count=0,
Exit Check: tid =14, req_sbmitted=1494720 req_completed=1494720 dir=H2C, intime=0 loop_count=0,
Exit Check: tid =6, req_sbmitted=1495488 req_completed=1495488 dir=H2C, intime=1495360 loop_count=1,
Exit Check: tid =5, req_sbmitted=1485568 req_completed=1485568 dir=C2H, intime=1485440 loop_count=1,
Exit Check: tid =11, req_sbmitted=1454208 req_completed=1454208 dir=C2H, intime=1454080 loop_count=1,
Exit Check: tid =13, req_sbmitted=1482944 req_completed=1482944 dir=C2H, intime=1482752 loop_count=1,
Exit Check: tid =0, req_sbmitted=1495168 req_completed=1495168 dir=H2C, intime=1494976 loop_count=2,
Exit Check: tid =10, req_sbmitted=1495104 req_completed=1495104 dir=H2C, intime=1494912 loop_count=2,
Exit Check: tid =12, req_sbmitted=1494592 req_completed=1494592 dir=H2C, intime=1494400 loop_count=2,
Exit Check: tid =9, req_sbmitted=1486784 req_completed=1486784 dir=C2H, intime=1486592 loop_count=2,
Exit Check: tid =15, req_sbmitted=1485248 req_completed=1485248 dir=C2H, intime=1485056 loop_count=2,
Exit Check: tid =1, req_sbmitted=1486656 req_completed=1486656 dir=C2H, intime=1486592 loop_count=1,
Exit Check: tid =4, req_sbmitted=1495872 req_completed=1495872 dir=H2C, intime=1495744 loop_count=1,
Exit Check: tid =3, req_sbmitted=1486336 req_completed=1486336 dir=C2H, intime=1486208 loop_count=2,
Exit Check: tid =7, req_sbmitted=1486400 req_completed=1486400 dir=C2H, intime=1486208 loop_count=2,
Exit Check: tid =2, req_sbmitted=1495744 req_completed=1495744 dir=H2C, intime=1495616 loop_count=2,
Exit Check: tid =10, req_sbmitted=1495296 req_completed=1495104 dir=H2C, intime=1494912 loop_count=10000,
Exit Check: tid =11, req_sbmitted=1454464 req_completed=1454336 dir=C2H, intime=1454080 loop_count=10000,
Exit Check: tid =5, req_sbmitted=1485632 req_completed=1485504 dir=C2H, intime=1485440 loop_count=10000,
Exit Check: tid =0, req_sbmitted=1495616 req_completed=1495424 dir=H2C, intime=1494976 loop_count=10000,
Exit Check: tid =12, req_sbmitted=1494912 req_completed=1494720 dir=H2C, intime=1494400 loop_count=10000,
Exit Check: tid =6, req_sbmitted=1495616 req_completed=1495488 dir=H2C, intime=1495360 loop_count=10000,
Stopped Queues 0 -> 0.
Exit Check: tid =9, req_sbmitted=1486912 req_completed=1486720 dir=C2H, intime=1486592 loop_count=10000,
Exit Check: tid =15, req_sbmitted=1485952 req_completed=1485760 dir=C2H, intime=1485056 loop_count=10000,
Exit Check: tid =13, req_sbmitted=1483456 req_completed=1483264 dir=C2H, intime=1482752 loop_count=10000,
Stopped Queues 0 -> 0.
Deleted Queues 0 -> 0.
Deleted Queues 0 -> 0.
WRITE: total pps = 3987072 BW = 255.172608 MB/sec
READ: total pps = 3950976 BW = 252.862464 MB/sec
[xilinx@]#


dma-perf tool takes a configuration file as input. The configuration file format is as below.

::
Expand Down
52 changes: 37 additions & 15 deletions master/QDMA/linux-kernel/html/_sources/features.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ QDMA Hardware Features
* User Interrupts
* Error Interrupts

* Mailbox communication between PF and VF
* Mailbox communication between PF and VF
* HW Error reporting
* Immediate data transfers
* Streaming H2C to C2H and C2H to H2C loopback support
Expand All @@ -48,7 +48,7 @@ Features Supported only in QDMA3.1 Design
=========================================

* Flexible BAR mapping for QDMA configuration register space

Features Supported only in QDMA4.0 Design
=========================================

Expand All @@ -72,11 +72,33 @@ Features Supported only in Versal CPM5 Design
* 256 C2H (Card-to-Host) descriptor rings
* 256 Completion Rings
* Tandem boot support


For details on Hardware Features refer to QDMA_Product_Guide_.
Features Supported only in Versal CPM4 Design
=============================================

* SRIOV with 4 Physical Functions (PF) and 252 Virtual Functions (VF)
* Descriptor Engine can be configured in Internal Mode only
* Supports 2048 queue sets for Physical Functions

- 2048 H2C (Host-to-Card) descriptor rings
- 2048 C2H (Card-to-Host) descriptor rings
- 2048 completion rings
* Supports 256 queue sets for Virtual Functions

- 256 H2C (Host-to-Card) descriptor rings
- 256 C2H (Card-to-Host) descriptor rings
- 256 Completion Rings
* Tandem boot support


For details on QDMA Hardware Features refer to QDMA_Soft_Product_Guide_.

.. _QDMA_Soft_Product_Guide: https://docs.xilinx.com/viewer/book-attachment/n6fBd_xlVtlFE96gGACfJA/gbwHDtOsU98Gh7U6BMS5rg

For details on CPM4 and CPM5 Hardware Features refer to QDMA_Hard_Product_Guide_.

.. _QDMA_Hard_Product_Guide: https://docs.xilinx.com/viewer/book-attachment/bjqy7nDIaVBBMR9rpQMFyg/aru18LAWHcbJZJmSRvXf~A

.. _QDMA_Product_Guide: https://www.xilinx.com/support/documentation/ip_documentation/qdma/v4_0/pg302-qdma.pdf

QDMA Software Features
**********************
Expand All @@ -85,23 +107,23 @@ QDMA Software Features
QDMA software provides 2 different drivers. PF driver for Physical functions and and VF driver for Virtual Functions.
PF and VF drivers can be inserted in different modes.

- Polling Mode
In Poll Mode, Software polls for the write back completions (Status Descriptor Write Back)
- Polling Mode
In Poll Mode, Software polls for the write back completions (Status Descriptor Write Back)

- Direct Interrupt Mode
In Direct Interrupt mode, each queue is assigned to one of the available interrupt vectors in a round robin fashion to service the requests.
In Direct Interrupt mode, each queue is assigned to one of the available interrupt vectors in a round robin fashion to service the requests.
Interrupt is raised by the HW upon receiving the completions and software reads the completion status.

- Indirect Interrupt Mode
In Indirect Interrupt mode or Interrupt Aggregation mode, each vector has an associated Interrupt Aggregation Ring.
The QID and status of queues requiring service are written into the Interrupt Aggregation Ring.
When a PCIe MSI-X interrupt is received by the Host, the software reads the Interrupt Aggregation Ring to determine which queue needs service.
In Indirect Interrupt mode or Interrupt Aggregation mode, each vector has an associated Interrupt Aggregation Ring.
The QID and status of queues requiring service are written into the Interrupt Aggregation Ring.
When a PCIe MSI-X interrupt is received by the Host, the software reads the Interrupt Aggregation Ring to determine which queue needs service.
Mapping of queues to vectors is programmable

- Auto Mode
Auto mode is mix of Poll and Interrupt Aggregation mode. Driver polls for the write back status updates.
Interrupt aggregation is used for processing the completion ring.

- Allows only Privileged Physical Function to program the contexts and registers
- Dynamic queue configuration, refer to Interface file, libqdma_export.h (``struct qdma_queue_conf``) for configurable parameters
- Dynamic driver configuration, refer to Interface file, libqdma_export.h (``struct qdma_dev_conf``) for configurable parameters
Expand Down
6 changes: 3 additions & 3 deletions master/QDMA/linux-kernel/html/build.html
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'2023.1',
VERSION:'2023.2',
LANGUAGE:'None',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
Expand Down Expand Up @@ -316,7 +316,7 @@


<div class="version">
2023.1
2023.2
</div>


Expand Down Expand Up @@ -764,7 +764,7 @@ <h3>3. <strong>Dynamic Config Bar</strong><a class="headerlink" href="#dynamic-c
<div role="contentinfo" class="copyright">
<p class="footerinfo">
<span class="lastupdated">
Last updated on April 28, 2023.
Last updated on September 29, 2023.
</span>

</p>
Expand Down
6 changes: 3 additions & 3 deletions master/QDMA/linux-kernel/html/debugfs.html
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'2023.1',
VERSION:'2023.2',
LANGUAGE:'None',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
Expand Down Expand Up @@ -316,7 +316,7 @@


<div class="version">
2023.1
2023.2
</div>


Expand Down Expand Up @@ -579,7 +579,7 @@ <h1>QDMA Debug File System Support<a class="headerlink" href="#qdma-debug-file-s
<div role="contentinfo" class="copyright">
<p class="footerinfo">
<span class="lastupdated">
Last updated on April 28, 2023.
Last updated on September 29, 2023.
</span>

</p>
Expand Down
6 changes: 3 additions & 3 deletions master/QDMA/linux-kernel/html/devguide.html
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'2023.1',
VERSION:'2023.2',
LANGUAGE:'None',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
Expand Down Expand Up @@ -316,7 +316,7 @@


<div class="version">
2023.1
2023.2
</div>


Expand Down Expand Up @@ -466,7 +466,7 @@ <h1>Developers Guide<a class="headerlink" href="#developers-guide" title="Permal
<div role="contentinfo" class="copyright">
<p class="footerinfo">
<span class="lastupdated">
Last updated on April 28, 2023.
Last updated on September 29, 2023.
</span>

</p>
Expand Down
33 changes: 30 additions & 3 deletions master/QDMA/linux-kernel/html/dma-latency.html
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
<script type="text/javascript">
var DOCUMENTATION_OPTIONS = {
URL_ROOT:'./',
VERSION:'2023.1',
VERSION:'2023.2',
LANGUAGE:'None',
COLLAPSE_INDEX:false,
FILE_SUFFIX:'.html',
Expand Down Expand Up @@ -316,7 +316,7 @@


<div class="version">
2023.1
2023.2
</div>


Expand Down Expand Up @@ -438,6 +438,31 @@ <h1>DMA Latency Application (dma-latency)<a class="headerlink" href="#dma-latenc
-c (--config) config file that has configuration for IO
</pre></div>
</div>
<p>Example</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>[xilinx@]# dma-latency -c dmalat.conf
dma-ctl qdma06000 global_csr | grep Global| cut -d : -f 2 &gt; glbl_rng_sz
qdma06000-ST-0 H2C added.
Added 1 Queues.
1 Queues started, idx 0 ~ 0.
qdma06000-ST-0 C2H added.
Added 1 Queues.
1 Queues started, idx 0 ~ 0.
dmautils(1) threads
Stopped Queues 0 -&gt; 0.
Deleted Queues 0 -&gt; 0.
Stopped Queues 0 -&gt; 0.
Deleted Queues 0 -&gt; 0.
qdma06000:statistics
Total MM H2C packets processed = 0
Total MM C2H packets processed = 0
Total ST H2C packets processed = 2288284
Total ST C2H packets processed = 2288284
Min Ping Pong Latency = 0
Max Ping Pong Latency = 0
Avg Ping Pong Latency = 0
[xilinx@]#
</pre></div>
</div>
<p>dma-latency tool takes a configuration file as input. The configuration file format is as below.</p>
<div class="highlight-none"><div class="highlight"><pre><span></span>name=mm_1_1
mode=mm #mode
Expand All @@ -455,6 +480,8 @@ <h1>DMA Latency Application (dma-latency)<a class="headerlink" href="#dma-latenc
bidir_en=1
num_pkt=64
pkt_sz=64
pci_bus=06
pci_device=00
</pre></div>
</div>
<p><strong>Parameters</strong></p>
Expand Down Expand Up @@ -508,7 +535,7 @@ <h1>DMA Latency Application (dma-latency)<a class="headerlink" href="#dma-latenc
<div role="contentinfo" class="copyright">
<p class="footerinfo">
<span class="lastupdated">
Last updated on April 28, 2023.
Last updated on September 29, 2023.
</span>

</p>
Expand Down
Loading

0 comments on commit 2aa255b

Please sign in to comment.