This document reflects the v6.2.3 version of the source code.
|
The Developer Guide explains how to build and modify the source code of the DAQ40 software, refer to the User Guide to learn how to use this software. Both guides concern only the software, for the firmware please refer to the Firmware Guides. For the Experiment Control System components and software (implemented on top of the DAQ40 software) please refer to https://lhcb-ecs.web.cern.ch/.
Getting the source
The project source code is versioned using the CERN gitlab infrastructure, read access is given by default to all members of the LHCb gitlab group.
git clone -b nameofbranch https://gitlab.cern.ch/lhcb-daq40/lhcb-daq40-software.git
Two main branches are available to choose from:
- devel
-
This branch is intended for most development. Effort is taken to ensure that the head is always buildable but that is not guaranteed. If the current head revision fails to build from source, just use the most recent development tag (which is guaranteed to build). Development tags have a
-wipsuffix. The most recent development tag will be the last listed by:git tag | sort | grep "\-wip"
- master
-
This branch contains only tested releases that are meant for end users. This means that this branch contains only merge commits from
develand occasionally urgent bugfixes. The git head will already correspond to the most recent stable tag, but it can be also obtained at the end of this output:git tag | sort | grep -v "\-wip"
Building from source
The build system for this project is based on GNU Make.
A top-level Makefile is provided to build all subprojects at once. A top-level build cannot be an in-tree build, a dedicated build folder must be created first. For example:
mkdir build/ && cd "$_" make -f ../Makefile
To avoid repeating the -f flag in each make invocation, a ./make wrapper script is automatically created in the build folder as a shortcut, it can be invoked in the same way make would.
¶
In a top-level build, the following targets are available:
- default, install, uninstall, man, maninstall
-
Only available out-of-tree, each just makes the given target in all subprojects (subproject targets are documented below).
- clean
-
If in-tree, cleans each subproject, if out-of-tree, empties the build folder.
- doc
-
Only available in-tree, builds the documentation. ¶
Each subfolder containing a Makefile can be build individually by simply issuing make from there. Dependencies on targets built in other subfolders will be built automatically. This can be used during development to quickly iterate on a single subproject.
By default, a sub-project Makefile implements at least the following targets:
- default
-
Builds all library and executable targets.
- install
-
Copies installable files to their destination.
- uninstall
-
Removes files copied by both install and maninstall.
- man
-
Builds man pages.
- maninstall
-
Installs man pages under
$(PREFIX)/share/man. - clean
-
Cleans build directory and deletes final targets.
- depclean
-
Makes the clean target in every dependency of this subproject. ¶
Build dependencies
On CC7, the following packages are required to build the software: ¶
rpm-build glibc-headers glibc-devel git make gcc-c++ gcc flex boost-devel libpcap-devel dim dim-devel hwloc-devel libcurl-develOn CC7, the following packages are required to build the kernel drivers: ¶
rpm-build make gcc kernel kernel-devel dkms gitOn CC7, when using devtoolset-7, the following packages are required to build the software:
¶
rpm-build centos-release-scl git flex boost-devel libpcap-devel dim dim-devel hwloc-devel libcurl-develOn EL9, the following packages are required to build the software: ¶
rpm-build glibc-headers glibc-devel git make gcc-c++ gcc flex tcl boost-devel libpcap-devel dim dim-devel hwloc-devel libcurl-develDKMS is not included in the base distribution: ¶
RUN yum install -y epel-releaseOn EL9, the following packages are required to build the kernel drivers: ¶
rpm-build make gcc kernel kernel-devel dkms gitConfiguring the build
Several variables can be set in order to control what to build and what dependencies to enable, these can be set either in the environment, for example: ¶
ENABLE_XXX=false make # Set flag for single command, or
export ENABLE_XXX=false # export flag in environment and
make # reuse exported flags.Or by directly editing the file common/flags.mk.
¶
The following dependency flags are available:
- ENABLE_DIM
-
Whether to include support for DIM. Default: true.
- ENABLE_INFLUX
-
Whether to produce monitoring data (uses the InfluxDB Line Protocol format). Default: true
- ENABLE_MON
-
Whether to include the interface for DQMP (Data Quality Monitor and Presenter). Default: false.
- ENABLE_HWLOC
-
Whether to include support for the hwloc library (will try to schedule threads processing data from a PCIe40 board on a core closest to the corresponding PCIe root complex). Default: true.
- ENABLE_IB
-
Whether to include support for the InfiniBand verbs API. Default: false.
- ENABLE_DOCRA
-
Whether to generate the documentation (HTML and man pages). Requires docra and asciidoctor installed. Default: true. ¶
The following flags can be used to build only a subset of the targets:
- ENABLE_DAQ40
-
Build tools and libraries that are common to both AMC40 and PCIe40 setups. Default: true.
- ENABLE_AMC40
-
Build tools and libraries specific to the AMC40 board. Default: true.
- ENABLE_CCPC40
-
Build tools and libraries specific to the AMC40 CCPC module. Default: false.
- ENABLE_PCIE40
-
Build tools and libraries specific the the PCIe40 board. Default: true. ¶
Building in a docker container
In order to have a build environment compatible with the production environment where the AMC40 and PCIe40 boards are deployed, several Dockerfiles are provided that already include all the build dependencies. These are: ¶
- rpm/os/cc7/Dockerfile
-
CERN CentOS 7 with all dependencies. This is the most used environment. ¶
- rpm/os/cc7-dkms/Dockerfile
-
CERN CentOS 7 with kernel dependencies for building DKMS-enabled drivers. Only used to build the device and emulation drivers and to ensure the RPMs install cleanly after they’re build. ¶
- rpm/os/cc7-dt7/Dockerfile
-
CERN CentOS 7 with all dependencies and the
devtoolset-7software collection. This environment is used to ensure compatibility with more recent versions of GCC. ¶ - rpm/os/el9/Dockerfile
-
AlmaLinux 9 (RHEL9 compatible) with all dependencies. This is the most used environment. ¶
- rpm/os/el9-dkms/Dockerfile
-
Alma Linux 9 with kernel dependencies for building DKMS-enabled drivers. Only used to build the device and emulation drivers and to ensure the RPMs install cleanly after they’re build. ¶
- rpm/os/slc6/Dockerfile
-
Scientific Linux 6 with all dependencies plus the AMC40 LLI and 32-bit runtime libraries (required to build executables for the AMC40 CCPC). ¶
- rpm/os/slc6-dkms/Dockerfile
-
Scientific Linux 6 with dependencies for building DKMS-enabled drivers. Only provided for backwards compatibility. PCIe40 users are strongly reccommended to run CC7. ¶
The script at rpm/makecontainers.sh can be used to automatically build all these containers on your development machine.
¶
The script at rpm/makecontainers-gitlab.sh can be used to automatically push these containers to the CERN gitlab container registry. This will update the containers used to run the Continuous Integration jobs.
¶
Running the container
- cc7
-
The container is tagged as
lhcb/daq40-sw-rpm-cc7and can be run using./rpm/os/slc6/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-rpm-cc7 \
-v=${TOP}:${HOME}/rpmbuild/SOURCES/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el7:${HOME}/rpmbuild/RPMS- cc7-dkms
-
The container is tagged as
lhcb/daq40-sw-dkms-cc7and can be run using./rpm/os/cc7-dkms/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-dkms-cc7 \
-v=${TOP}:/usr/src/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el7:/rpm \
-u=root- cc7-dt7
-
The container is tagged as
lhcb/daq40-sw-rpm-cc7-dt7and can be run using./rpm/os/slc6/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-rpm-cc7-dt7 \
-v=${TOP}:${HOME}/rpmbuild/SOURCES/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el7:${HOME}/rpmbuild/RPMS- el9
-
The container is tagged as
lhcb/daq40-sw-rpm-el9and can be run using./rpm/os/slc6/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-rpm-el9 \
-v=${TOP}:${HOME}/rpmbuild/SOURCES/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el9:${HOME}/rpmbuild/RPMS- el9-dkms
-
The container is tagged as
lhcb/daq40-sw-dkms-el9and can be run using./rpm/os/el9-dkms/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-dkms-el9 \
-v=${TOP}:/root/rpmbuild/SOURCES/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el9:/root/rpmbuild/RPMS \
-u=root- slc6
-
The container is tagged as
lhcb/daq40-sw-rpm-slc6and can be run using./rpm/os/slc6/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-rpm-slc6 \
-v=${TOP}:${HOME}/rpmbuild/SOURCES/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el6:${HOME}/rpmbuild/RPMS- slc6-dkms
-
The container is tagged as
lhcb/daq40-sw-dkms-slc6and can be run using./rpm/os/slc6-dkms/run.shfrom the top-level folder. This is equivalent to (TOPis the top level of the git repository) ¶
${TOP}/common/docker/run.sh lhcb/daq40-sw-dkms-slc6 \
-v=${TOP}:/usr/src/lhcb-daq40-software \
-v=${TOP}/rpmbuild/el6:/rpm \
-u=rootInside any of these containers you can run rpmbuild or dkms.sh as required for each package. When using rpmbuild , the resulting RPM will be written under the rpmbuild/ subdirectory of the top-level source folder, which must be created if it does not exist.
Packaging
Using the provided spec files, RPM packages can be built by simply executing rpmbuild -ba file.spec. Regular packages assume that the source code top-level directory is ~/rpmbuild/SOURCES/lhcb-daq40-software while DKMS packages use /usr/src/lhcb-daq40-software (using Running the container these pre-requirements are automatically satisfied).
libdaq40
libdaq40_jtag
daq40_frgchecker
daq40_frgreader
daq40_frgreader_v0
daq40_frg2mdf
daq40_mdfchecker
daq40_mdfreader
daq40_mdfvalidator
daq40_jtag
daq40_ramdisk
daq40_txtparser
daq40_sysinfo
daq40_xmlgen
daq40_writespeedlibamc40
amc40_ccsetup
amc40_net_setup_pc
amc40_capchecker
amc40_frgchecker
amc40_frgreader
amc40_frgwriter
amc40_frg2mdf
amc40_mdfchecker
amc40_mdfreader
amc40_mdfvalidator
amc40_pgm
amc40_udpgen
amc40_sysinfo- rpm/lhcb-pcie40-driver.spec
-
This package provides the PCIe40 driver (after DKMS removed RPM integration). ¶ These include: ¶
jtagd
jtagconfig
quartus_pgm
quartus_pgmwlibpcie40_id
libpcie40_ecs
libpcie40_i2c
libpcie40_daq
libpcie40_hwloc
libpcie40_localeb
pcie40_id
pcie40_ecs
pcie40_i2c
pcie40_daq
pcie40_hwloc
pcie40_localeblibpcie40
pcie40_daqserver
pcie40_frgchecker
pcie40_frgreader
pcie40_frg2mdf
pcie40_mdfchecker
pcie40_mdfreader
pcie40_mdfvalidator
pcie40_localebserver
pcie40_mfpreader
pcie40_pgm
pcie40_pgmserver
pcie40_reload
pcie40_rfg
pcie40_simchecker
pcie40_simreader
pcie40_sim2frg
pcie40_sofusercode
pcie40_sysinfo
pcie40_systemd- rpm/lhcb-daq40-doc.spec
-
This package provides the HTML documentation for offline viewing. ¶
The RPMs produced from this repository have dependencies between each other. In order to always ensure compatibility between them the spec files enforces each package and its dependents to always be at the same version.
DKMS
Building DKMS packages is not as straightforward as simply launching rpmbuild, therefore helper scripts are provided, they can be found at rpm/os/{slc6|cc7}-dkms/dkms.sh and are used, for example, in the CI pipeline.
¶
The scripts take one parameter, which is the subfolder containing the kernel module source and the DKMS configuration.
¶
dkms.conf is effectively just a shell script and contains the following:
¶
DAQ40_VERSION=`/usr/src/lhcb-daq40-software/rpm/getversion.sh` (1)
DAQ40_RELEASE=`/usr/src/lhcb-daq40-software/rpm/getrelease.sh` (2)
export DAQ40_RELEASE (3)
DAQ40_VER_REL=${DAQ40_VERSION}-${DAQ40_RELEASE}
export DAQ40_VER_REL (4)
DKMS_DIRECTIVE=DAQ40_VER_REL=${DAQ40_VER_REL} (5)
PACKAGE_NAME=lhcb-pcie40-driver (6)
PACKAGE_VERSION=${DAQ40_VERSION}-${DAQ40_RELEASE} (7)
AUTOINSTALL=yes (8)
BUILT_MODULE_NAME[0]=lhcb_pcie40 (9)
DEST_MODULE_LOCATION[0]=/extra
BUILT_MODULE_NAME[1]=lhcb_pcie40_emu (10)
DEST_MODULE_LOCATION[1]=/extra
(11)| 1 | Detect version from git tag. |
| 2 | Detect release from git tag. |
| 3 | DAQ40_RELEASE is NOT used by DKMS directly but it is read by the .spec template |
| 4 | Representation of version and release used in packages and executable code |
| 5 | Explicitly inject variable in DKMS build environment |
| 6 | Required by DKMS. |
| 7 | Required by DKMS. We include the release part so that each package will be stored under unique paths. Without it two RPMs differing only in release will store their data under the same directories and when upgrading the DKMS tree of the new RPM will be deleted by the uninstall step of the previous RPM. |
| 8 | Ensure the driver is recompiled when a new kernel is installed |
| 9 | Name of the device driver. |
| 10 | Name of the emulation driver. |
| 11 | DEST_MODULE_LOCATION is ignored by the rpm but dkms add fails without it. |
The helper script proceeds as follows: ¶
source /usr/src/lhcb-daq40-software/$1/dkms.conf (1)
dkms add --verbose --rpm_safe_upgrade /usr/src/lhcb-daq40-software/$1 (2)
sed -i "s/DAQ40_VERSION=.*/DAQ40_VERSION=${DAQ40_VERSION}/" /usr/src/${PACKAGE_NAME}-${PACKAGE_VERSION}/dkms.conf (3)
sed -i "s/DAQ40_RELEASE=.*/DAQ40_RELEASE=${DAQ40_RELEASE}/" /usr/src/${PACKAGE_NAME}-${PACKAGE_VERSION}/dkms.conf (4)
find /usr/src/${PACKAGE_NAME}-${PACKAGE_VERSION} -mindepth 1 -type d | xargs rm -rf (5)
dkms build --verbose -k `ls /lib/modules` -m ${PACKAGE_NAME} -v ${PACKAGE_VERSION} (6)
ln -sf /var/lib/dkms/${PACKAGE_NAME}/${PACKAGE_VERSION}/ /var/lib/dkms/${PACKAGE_NAME}/${DAQ40_VERSION} (7)
dkms mkrpm --verbose -k `ls /lib/modules` -m ${PACKAGE_NAME} -v ${DAQ40_VERSION} (8)
rm /var/lib/dkms/${PACKAGE_NAME}/${DAQ40_VERSION}
mkdir -p /rpm/noarch
cp /var/lib/dkms/${PACKAGE_NAME}/${PACKAGE_VERSION}/rpm/*.noarch.rpm /rpm/noarch (9)| 1 | This is just a shell script and can be sourced as such. |
| 2 | This will copy the source code under /usr/src/<module-moduleversion> where DKMS expects it. |
| 3 | Remove reference to version detection command (which is not present on the host system when rebuilding the module). |
| 4 | Likewise, remove reference to release detection command and replace it with its value |
| 5 | Remove subdirectories (because in our case they contain things like historical driver performance logs which make no sense to distribute in the target machine) |
| 6 | Compile the module |
| 7 | Create symbolic link since mkrpm does not accept - characters in ${PACKAGE_VERSION} |
| 8 | Create rpm under /var/lib/dkms/… |
| 9 | Copy to usual place (note that DKMS packages are noarch since they still contain the source and will automatically recompile the pre-built binary for the target architecture and kernel) |
Versioning
This repository follows semantic versioning, meaning that versions are identified by a monotonically increasing tuple of three elements (major, minor, patch).
From the current git head, the scripts under rpm/getversion.sh and rpm/getrelease.sh are used to dynamically obtain the version and release identifiers. These identifiers are used, for example, to automatically generate RPM packages.
- version
-
If the git head is tagged, and the tag starts with
v, then the version identifier will be everything in the tag name after the initialv. This identifier must follow semantic versioning order (for example, the first minor release after v1.2.3 must be v1.3.0 ). ¶ - release
-
The release identifier is used, together with the version identifier, to define the package name used to publish a particular commit. Releases are identified by an integer, starting at
1for a tag release (as specified at https://fedoraproject.org/wiki/Packaging:Versioning) and automatically increased by one for any untagged commit on themasterbranch.-
Tagged commits are published to the
stableRPM repository. -
Untagged commits to the master branch are published to the
unstableRPM repository. -
Untagged commits in other branches can be published manually through GitLab CI by triggering the
publish-unstablejob. In this case the branch name and git hash are appended to the automatic release identifier. ¶
-
Releasing
To create an unstable release it is sufficient to tag the desired commit on the devel branch according to the format described above. In git, tags need to be pushed with a dedicated command:
git push --tags
On the other hand, commits for stable releases (on the master branch) are either one of two types:
-
Merge commits from the
develbranch -
Urgent bugfixes on top of such a merge commit that could not be readily merged from the
develbranch
In the first case, squashing and fast-forwarding commits should be avoided in order to keep track of the development leading to a particular release. The following command can be used to always create a merge commit:
git merge --no-ff devel
Continuous Integration
A GitLab Continuous Integration pipeline is implemented in order to automatically and reproducibly build and publish each software release to a dedicated RPM repository.
The pipeline implements the following stages:
-
build (on very commit)
-
package (only commits in
develandmasterbranches) -
install (as above)
-
publish (only semver tags by default, manual trigger available)
-
deploy (manual trigger only)
Each stage includes several parallel tasks. CI tasks can, to some extent, be tested locally, by installing gitlab-runner and docker and doing:
gitlab-runner exec docker <taskname>
The following describes each stage in detail. ¶
Build
The tasks in the build stage compile the entire source code for different operating systems and with different flag combinations. ¶
- make-cc7-dim
-
Build for CC7 with the following flags: ¶
ENABLE_DIM: 'true'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'- make-cc7-disableall
-
Build for CC7 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'ENABLE_DIM: 'true'
ENABLE_MON: 'true'
ENABLE_HWLOC: 'true'- make-cc7-hwloc
-
Build for CC7 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'true'- make-cc7-mon
-
Build for CC7 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'true'
ENABLE_HWLOC: 'false'ENABLE_DIM: 'true'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'- make-el9-disableall
-
Build for EL9 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'true'- make-el9-mon
-
Build for EL9 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'true'
ENABLE_HWLOC: 'false'- make-slc6-dim
-
Build for SLC6 with the following flags: ¶
ENABLE_DIM: 'true'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'- make-slc6-disableall
-
Build for SLC6 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'false'ENABLE_DIM: 'false'
ENABLE_MON: 'false'
ENABLE_HWLOC: 'true'- make-slc6-mon
-
Build for SLC6 with the following flags: ¶
ENABLE_DIM: 'false'
ENABLE_MON: 'true'
ENABLE_HWLOC: 'false'Package
The tasks in the package stage create the RPMs for all the supported operating systems. ¶
- dkms-cc7
-
Builds the
lhcb-pcie40-driverpackage for CC7. ¶ - dkms-el9
-
Builds the
lhcb-pcie40-driverpackage for el9. ¶ - dkms-slc6
-
Builds the
lhcb-pcie40-driverpackage for SLC6. ¶ - rpm-cc7-amc40
-
Builds the
lhcb-amc40-toolspackage for CC7. ¶ - rpm-cc7-daq40
-
Builds the
lhcb-daq40-commonandlhcb-daq40-winccpackages for CC7. ¶ - rpm-cc7-pcie40
-
Builds the
lhcb-pcie40-libs,lhcb-pcie40-toolsandlhcb-pcie40-winccpackages for CC7. ¶ - rpm-daq40-doc
-
Builds the
lhcb-daq40-docpackage. ¶ - rpm-el9-amc40
-
Builds the
lhcb-amc40-toolspackage for el9. ¶ - rpm-el9-daq40
-
Builds the
lhcb-daq40-commonandlhcb-daq40-winccpackages for EL9. ¶ - rpm-el9-pcie40
-
Builds the
lhcb-pcie40-libs,lhcb-pcie40-toolsandlhcb-pcie40-winccpackages for EL9. ¶ - rpm-slc6-amc40
-
Builds the
lhcb-amc40-toolspackage for SLC6. ¶ - rpm-slc6-ccpc40
-
Builds the
lhcb-ccpc40-toolspackage for SLC6. ¶ - rpm-slc6-daq40
-
Builds the
lhcb-daq40-commonpackage for SLC6. ¶ - rpm-slc6-pcie40
-
Builds the
lhcb-pcie40-libsandlhcb-pcie40-toolspackage for SLC6. ¶
Install
The tasks in the install stage ensure that the packages created in the previous stage install cleanly. ¶
Publish
The tasks in the publish stage take all the artifacts produced by the previous stages and make them available to other users and developers. ¶
- publish-branch
-
This task can be triggered by hand to publish an unstable release from any (non-master) branch. The branch name will automatically become part of the RPM release field. ¶
- publish-docs
-
This task copies the HTML documentation from the last commit on the
develbranch onto a dedicated folder. This ensures that the documentation at https://cern.ch/lhcb-online-soft/doc/daq/devel/ always reflects the current state of the development branch and that it can quickly be updated by a simple push, without having to tag a new release. ¶ - publish-unstable
-
This task only applies to unstable releases (See Versioning). First it updates the
unstablerepositories described in the User Guide. Then it copies the corresponding documentation under https://cern.ch/lhcb-online-soft/doc/daq/<tag> and updates https://cern.ch/lhcb-online-soft/doc/daq/latest-unstable/ to always point to the most recent development release. ¶
High-level code walkthrough
This section describes how different parts of the high-level (meaning C++) codebase interact together in order to implement a given piece of functionality.
Common code
In order to reuse as much code as possible between AMC40 and PCIe40 implementation, a common set of base classes is provided which are then specialized for the different readout boards.
Parsing configuration files
Configuration files follow an INI-like syntax. The parser is implemented in the [dcrIni_] class. This class is extended in [dcrdaq40_cfg_] to add all the configuration parameters supported by DAQ40 software tools and documented in the User Guide. A system typically uses only one global configuration file, daq40.cfg.
Starting from the firmware parameters read from a configuration file, daq40_cfg::detect_data_format is responsible for detecting the actual data format and chose the most appropriate function to use when parsing frontend data (either in binary or textual format).
¶
Parsing simulated frontend data
When running a firmware simulation, it is possible to generate text files containing a cycle-by-cycle representation of the data that was generated by the frontend generators on each optical link. The class daq40_txt_flex can be used to parse these text files. For each supported format, a scanner was implemented using GNU Flex. The following are available:
¶
- parse_ff_lsb
-
Scanner for FF data in LSB order (LSB is deprecated in the firmware and should not be used). ¶
- parse_ff_msb
-
Scanner for FF data in MSB order. ¶
- parse_fv_lsb
-
Scanner for FV data in LSB order (LSB is deprecated in the firmware and should not be used). ¶
- parse_fv_msb
-
Scanner for FV data in MSB order. ¶
- parse_muon_msb
-
Scanner for MUON data in MSB order. ¶
- parse_scifi_msb
-
Scanner for SciFi data in FF format.
- parse_scifi_msb
-
Scanner for SciFi channels in MSB order (SciFi data is aligned on 28-bit boundaries). ¶
- parse_velo_msb
-
Scanner for VELO front-end data, each line is 124 bits and contains 4 SuperPixelPackets (SPP). ¶
Parsing fragment data
A readout board receives data from several Front-End optical links. Front-End data corresponding to the same collision (meaning, data having the same BXID), is assembled by the readout board firmware into a fragment (fragments are in turn assembled by the event builder into complete events). So, a fragment contains collision data seen by an individual readout board, an event contains collision data from all LHCb subdetectors (and subdetector-like data sources). The DAQ40 software includes decoders able to locate within a single fragment the data originating from a given Front-End optical link, these are mostly used for validating the firmware. Two functions are available depending on the data ordering used in the firmware: [dcrdaq40_msb_parse_fragment_] for MSB data (preferred) and [dcrdaq40_lsb_parse_fragment_] for LSB data. Internally, the binary data generated by the firmware is densely packed and impossible to easily parse in software. Two utility classes are implemented in order to access individual bits and bit ranges within a fragment: [dcrreversed_bitbuffer_] for LSB data and [dcrbitbuffer_] for MSB data.
| These classes are only used to validate the default data processing done by the FPGA! In reality each subdetector should, inside the FPGA, rearchitect its data layout to emit fragments in a format that is practical to decode by software. ¶ |
Data buffering
Both in the case of an AMC40 board and a PCIe40 board (and in data acquisition in general), the process of receiving data from the board is decoupled from the process of consuming this data. In the DAQ40 software, this means that those two functions execute in independent threads. In order to synchronize data movement in a producer/consumer pair there exists a generic daq40_blockbuffer class.
¶
Consuming data streams
Data streams can be produced using either an AMC40 or a PCIe40 board. The DAQ protocols are radically different depending on the hardware used and no common classes exist to encapsulate data producers. On the opposite end, all data consumer threads do have to conform to a set of common interfaces, the first one being [dcrdaq40_consumer_] . Consumers that use the received data to create files on the local disk implement a specialized version of this interface, called [dcrdaq40_writer_] . When DIM is enabled, each consumer is associated with an instance of [dcrdaq40_consumer_dim_] (or [dcrdaq40_writer_dim_] depending on the case) in order to interface with the control system. ¶
Monitoring data streams
As a data stream is being consumed (either to be sent to the event builder network or to be stored locally), it can also, optionally, be sampled for the purpose of collecting real-time information about data quality. The generic interface between a data stream and the monitoring system is represented by [dcrdaq40_monitor_] . At this time, a single concrete implementation of this interface exists, [dcrdaq40_monitor_dqmp_] . ¶
Writing data files
Currently, the primary usage of the DAQ40 software is to store the data captured by a given readout board onto a local file for later analysis. The files currently generated have an .frg extension and contain the data fragments as they were captured from the readout board (see Frg data format). The common base class for .frg file writers is daq40_frgwriter .
¶
Frg data format
An .frg file starts with a header, defined by the struct [dcrdaq40_file_hdr_top_] , and continues with a list of data blocks. Each data block (often misleadingly referred to as a "MEP block") also starts with a header, defined by the struct [dcrdaq40_file_hdr_block_] and followed by a list of data fragments. In turn, each fragment also has a header, defined by the struct daq40_file_hdr_frg . Each fragment is associated with a temporally unique event id assigned by the TFC subsystem. The fragment header is followed by all the data, corresponding to that event id, received from all the optical input links associated with a given data stream (in the case of the AMC40, this means all the active fibers on the board, in the case of the PCIe40, this means the fibers going to PCIe link 0 or link 1).
¶
Reading data files
The DAQ40 software includes a command-line tool and a C++ library that can be used to read back the .frg data files produced using either AMC40 or PCIe40 setups. The main class implementing this functionality is daq40_frgreader .
¶
A typical example using this class to iterate a file would begin like this:
¶
daq40_frgreader rdr(argv[1]); (1)
if (!rdr.open()) { (2)
perror("Unable to open file");| 1 | Instantiate the reader |
| 2 | Open the file |
Then iterate: ¶
for (daq40_frgreader::block_iterator m = rdr.begin(); m != rdr.end(); ++m) { (1)
const daq40_file_hdr_block &block_hdr = m.hdr(); (2)
if (block_hdr.frags) {
for (daq40_frgreader::frg_iterator f = m.begin(); f != m.end(); ++f) { (3)
daq40_file_hdr_frg const *frg_hdr = f.hdr(); (4)| 1 | First using daq40_frgreader::block_iterator |
| 2 | To obtain a daq40_file_hdr_block |
| 3 | Then using daq40_frgreader::frg_iterator |
| 4 | To obtain a daq40_file_hdr_frg |
PCIe40-specific code
Configuration
The [dcrpcie40_cfg_] class specializes [dcrdaq40_cfg_] to implement parameters that are specific to the PCIe40. ¶
DMA streams
Data produced by a given DMA stream is in turn consumed by an instance of [dcrpcie40_frgwriter_] .
¶
When DIM support is enabled, each writer is associated with a pcie40_frgwriter_dim
DMA interface
A single PCIe40 board contains two PCIe interfaces. Each interface is represented by a [dcrpcie40_interface_dim_] instance. All PCIe40 interfaces within a single readout unit are enumerated inside one common [dcrpcie40_daqserver_dim_] instance. ¶
AMC40-specific code
Configuration
The [dcramc40_cfg_] class specializes [dcrdaq40_cfg_] to implement parameters that are specific to the AMC40. All parameters are described in the User Guide. ¶
Low-level code walkthrough
This section describes the low-level C API that high-level code uses to interface with the operating system (in particular with the readout board kernel drivers).
PCIe40-specific code
Low-level ECS driver API
In order to interact with the ECS interface of the PCIe40 driver, C libraries are provided (libpcie40_ecs.so and libpcie40_ecs.a).
To use these libraries, add #include <lhcb/pcie40/ecs.h> to your source code and -lpcie40_ecs to your linker flags.
This API is very simple, consisting only of a handful of functions to create I/O memory mappings and to access individual I/O registers, see PCIe40 driver ECS API.
¶
Low-level DAQ driver API
In order to interact with the DAQ interface of the PCIe40 driver, C libraries are provided (libpcie40_daq.so or libpcie40_daq.a).
To use these libraries, add #include <lhcb/pcie40/daq.h> to your source code and -lpcie40_daq to your linker flags.
The full API is listed at PCIe40 driver DAQ API. Functions starting in p40_ctrl_ access the stream controller on the FPGA (there is one such controller per PCIe link), while functions starting in p40_stream_ access an individual stream (there can be more streams instantiated on the FPGA inside a given controller).
The most important use of this API is to consume a data stream produced by the FPGA, the main steps in this case are:
The above diagram reflects how the library is used internally by [dcrpcie40_stream_dma_] , [dcrpcie40_blockbuffer_] and pcie40_frgwriter .
¶
Kernel-level code walkthrough
This section describes how kernel driver code is organized.
PCIe40-specific code
Device driver
Device initialization
When the kernel module is loaded, pcie40_init is immediately called. Its dual at unload time is pcie40_exit . The actual PCI device management happens in pcie40_probe and pcie40_remove .
¶
ECS interface
The ECS submodule implements the system calls behind the BAR0 and BAR2 device files. The submodule is initialized and uninitialized at the same time as the main module, through the pcie40_ecs_init and pcie40_ecs_exit functions. Likewise the ECS-specific probing logic is encapsulated in pcie40_ecs_probe and pcie40_ecs_remove .
¶
AMC40-specific code
There exists a kernel driver that is used to interface the AMC40 FPGA with the control system components running on the Credit-Card PC (CCPC), this is however a separate project maintained by CPPM (Centre de physique des particules de Marseille) and is not documented here, the source code is available in this repository.
High-level code cross-reference
daq40_blockbuffer
Generic implementation of thread-safe producer/consumer pair backed by an array of fixed-size blocks. ¶
This class is specialized depending on the board, and for a given board, also depending on the type of data it handles.
The AMC40 board has a single specialization for holding UDP packets, [dcramc40_blockbuffer_udp_] . The PCIe40 needs two: one ( [dcrpcie40_blockbuffer_nometa_] ) is used when metadata acceleration is disabled, in this case some "pseudo-metadata" is generated internally by the software, a second one ( pcie40_blockbuffer_meta
daq40_cfg::detect_data_format
Detects the LHCb firmware data format from the parameters under the key section of the configuration file.
¶
daq40_format daq40_cfg::detect_data_format(const char *key);Returns → daq40_format
If the given configuration key does not specify a size for the BXID or the info bits then no decoding is possible and the data will be interpreted as just a RAW stream of bits.
¶
Otherwise, if the configuration specifies a subdetector-specific format (as is the case for the UT and the SciFi), then that is returned.
¶
If not, one of the generic formats is returned. If fe_datalen_bits is 0, there is no length field, therefore all frames are of fixed size and FORMAT_FF is returned.
¶
Lastly, if fe_datalen_always is true, the length is present even when there is no data, so the header is fixed in size even if the data length varies. In this case FORMAT_FV is returned.
¶
There exists a definition for a FORMAT_VV option (variable header size, variable data size) but this is not used by any subdetector.
¶
daq40_consumer
Generic interface for a DAQ40 fragment stream consumer. This interface exposes methods to monitor the amount of data having been consumed during the current run, it is implemented by AMC40 and PCIe40 streams according to the different DAQ protocols. ¶
daq40_file_hdr_frg
On-disk representation of a data fragment header received from either an AMC40 or PCIe40 board (see daq40_frg_hdr ).
¶
typedef struct daq40_frg_hdr daq40_file_hdr_frg;daq40_file_hdr_top
Top-level header identifying a .frg file produced by amc40_frgwriter or pcie40_daqserver.
¶
Header for a file produced by amc40_frgwriterpcie40_daqserver
daq40_format
The following formats are possible: ¶
enum daq40_formatFORMAT_RAW_GBT,
FORMAT_RAW_WIDE,
FORMAT_RAW_GWT,
FORMAT_FF,
FORMAT_FV,
FORMAT_VV,
FORMAT_UT_4x3,
FORMAT_UT_2x3,
FORMAT_UT_2x4,
FORMAT_UT_2x5,
FORMAT_SCIFI_FF,
FORMAT_SCIFI_FV,
FORMAT_CALO,
FORMAT_RICH,
FORMAT_MUON,
FORMAT_VELO,
FORMAT_UNKNOWNdaq40_frg_hdr
Generic header for a data fragment produced by the TELL40 firmware data processing block. ¶
daq40_frg_hdr::BIT_STEP_RUN
BIT_STEP_RUN = 18As of april 2018, bit 18 determines whether we are in "step run" mode, in that case the gdl is restricted to bits [11..0] and bits [17..12] are used for the step number (only applies to old data format). ¶
daq40_frg_hdr::BIT_TRUNCATED
BIT_TRUNCATED = 19,As of october 2017, msb of the size determines whether the fragment is truncated (only applies to old data format). ¶
daq40_frgreader
File reader interface for .frg files produced by either AMC40 or PCIe40 setups. ¶
This class can be used to open an .frg file and iterate over its contents.
Two iterator classes are provided for this purpose. Since temporally consecutive fragments are aggregated into blocks, [dcrdaq40_frgreader_block_iterator_] iterates over the blocks themselves. Within a given block [dcrdaq40_frgreader_frg_iterator_] can be used to iterate from one fragment to the next. ¶
daq40_frgreader::block_iterator
Sequential iterator to access fragment blocks within an .frg file. ¶
daq40_frgreader::frg_iterator
Sequential iterator to access individual fragments within a fragment block. ¶
daq40_frgwriter
Base class for both AMC40 and PCIe40 .frg file writers, implements the daq40_writer interface.
¶
daq40_fsm::fsm_state
Encoding of FSM states. ¶
enum fsm_state {
FSM_ERROR = -100,
FSM_NOT_READY = 0,
FSM_READY = 100,
FSM_RUNNING = 200
};daq40_fsm_handler
Interface to handle FSM state transitions.
An instance can be passed to the daq40_fsm constructor to be registered as its FSM handler for all state transitions.
¶
daq40_lsb_parse_fragment
Parse binary data containing an assembled fragment (in LSB order) and optionally compare it against text files representing Front-End data. ¶
void daq40_lsb_parse_fragment(daq40_cfg &cfg, uint16_t frg_bxid, size_t frg_gdl, reversed_bitbuffer &buffer, daq40_txt_flex::map &txt, int &fail_counter);- cfg (I)
-
global configuration ¶
- frg_bxid (I)
-
BXID of fragment (compared against the BXID in each fiber, if not stripped) ¶
- buffer (IO)
-
reference to
reversed_bitbuffercontaining the LSB data payload (after the global fragment header, which is not included here) ¶ - txt (IO)
-
optional pointer to array of
daq40_txt_flexobjects (one for each optical Front-End link) ¶ - fail_counter (IO)
-
Error counter used by the caller to track validation state (if txt is given) ¶
daq40_monitor
Interface to monitor a given stream. ¶
This class is derived from daq40_fsm so that any implementation must also provide logic for the different possible FSM transitions. This allows the monitoring process to be synchronized (through the control system) with the lifecycle of the stream that is being monitored.
¶
daq40_monitor_dqmp
Implementation of daq40_monitor interface for DQMP.
¶
This class exposes a shared-memory interface between a given data stream (coming from either an AMC40 or a PCIe40 readout board) and the DQMP (Data Quality Monitor and Presenter) software. ¶
daq40_msb_parse_fragment
Parse binary data containing an assembled fragment (in MSB order) and optionally compare it against text files representing Front-End data. ¶
void daq40_msb_parse_fragment(daq40_cfg &cfg, uint16_t frg_bxid, uint8_t frg_type, size_t frg_gdl, bitbuffer &buffer, daq40_txt_flex::map &txt, int &fail_counter);daq40_msb_parse_fragment_scifi
void daq40_msb_parse_fragment_scifi(daq40_cfg &cfg, uint16_t frg_bxid, uint8_t frg_type, size_t frg_gdl, bitbuffer &buf, daq40_txt_flex::map &txt, int &fail_counter);daq40_msb_parse_fragment_velo
Parse binary fragments containing VELO SPPs and compare against velo SPPs from front-end input files. SPPs in front-end data are grouped by link and sorted in time. ¶
void daq40_msb_parse_fragment_velo(daq40_cfg &cfg, uint16_t frg_bxid, uint8_t frg_type, size_t frg_gdl, bitbuffer &buf, daq40_txt_velo::chipmap &txt, int &fail_counter);daq40_txt_flex::scan
Parse the input text according to the front-end format in the configuration. ¶
daq40_txt_flex::token_type daq40_txt_flex::scan();Returns → [dcrdaq40_txt_flex_token_type_] in particular [dcrdaq40_txt_flex_TOK_INVALID_] in case of error and daq40_txt_flex::TOK_EOT at the end of the text file.
¶
daq40_txt_flex::seek
Seek inside a text stream looking for a particular BXID. ¶
daq40_txt_flex::token_type daq40_txt_flex::seek(uint16_t bxid);Returns → The first [dcrdaq40_txt_flex_token_type_] for the collision with the given BXID, or [dcrdaq40_txt_flex_TOK_EOT_] if the file ended without finding it. ¶
daq40_txt_flex::skip
Skip a given number of BXIDs. ¶
daq40_txt_flex::token_type daq40_txt_flex::skip(size_t bxids);- bxids (I)
-
Number of collisions to skip in the data stream. ¶
Returns → The first daq40_txt_flex::token_type for the first collision after the skipped range.
¶
daq40_writer
Base class for all DAQ file writers. NOTE: Only the PCIe40 uses this atm, the AMC40 code must still be ported. ¶
This class extends [dcrdaq40_consumer_] adding methods to set where data is to be written and how much data should be written at most during a given run (so as to prevent a runaway process from completely filling the local storage). This class also inherits [dcrdaq40_fsm_] , this forces all implementers to implement FSM transitions in order to be driven by the control system. ¶
daq40_writer::set_mode
Set the writer mode and parameters ¶
bool set_mode(daq40_cfg::out_mode mode, const std::string &tcp_host, uint16_t tcp_port);daq40_writer::set_path
Set the full path of the output file ¶
bool set_path(const std::string &path);daq_mmap_common
static int daq_mmap_common(struct pcie40_dma_stream *stream, struct vm_area_struct *vma);
pcie40_blockbuffer_nometa
Specialization of [dcrpcie40_blockbuffer_] that contains [dcrpcie40_frg_block_] blocks. ¶
pcie40_daqserver_dim
DIM publisher for the top-level server FSM. ¶
This class enumerates all pcie40_interface_dim within a readout unit and publishes DIM commands and services that can be used to monitor and control them.
pcie40_frg_block
Pseudo-metadata block created internally by server when FPGA metadata is not available. ¶
pcie40_frg_hdr
Representation of a fragment header emitted by the PCIe40 board when metadata acceleration is disabled. ¶
struct __attribute__((__packed__)) pcie40_frg_hdrpcie40_interface_dim
DIM publisher for a given PCIe40 interface. Each PCIe40 board implements up to two PCIe interfaces. ¶
This class contains up to three data producers (one for the main data stream, one for the metadata and one for odin banks if implemented in the FPGA) and up to two data consumers (one for writing out .frg files for with the DAQ data and one, optional, to do the same for SODIN data).
Low-level C API cross-reference
p40_id_exists
Check whether there exists a device id endpoint with the given device number. ¶
int p40_id_exists(int dev);Returns → 0 if the endpoint exists and < 0 in case of error. ¶
p40_id_find
The name string is compared against the name and serial number of each interface. In addition, the following rules are applied:
-
A name string in the form XXX:0 or XXX_0 matches the first interface with name XXX and link id 0
-
A name string in the form XXX:1 or XXX_1 matches the first interface with name XXX and link id 1
-
A name string in the form XX-XX-XX-XX-XX-XX-XX-XX:0 matches the interface with the given FPGA serial and link id 0
-
A name string in the form XX-XX-XX-XX-XX-XX-XX-XX:1 matches the interface with the given FPGA serial and link id 1
-
A name string in the form bus:device.function matches the PCIe interface with the given address
-
A name string in the form 0xABCD matches the interface with the given source id ¶
Find lowest interface number with given name. ¶
int p40_id_find(const char *name);Returns → an interface number (>= 0) or < 0 if no such interface is present. ¶
p40_id_find_all
If a name is given, all matching variations including those with :0, _0, :1 and _1 suffixes will be returned.
¶
Find all interface numbers matching a given name. ¶
uint32_t p40_id_find_all(const char *name);- name (I)
-
optional board name, if empty or null all existing interfaces are returned ¶
Returns → A 32-bit mask with one bit set for each matched interface number. ¶
p40_id_get_fpga
Get serial number of FPGA. ¶
int64_t p40_id_get_fpga(int fd);See P40_ID_GET_FPGA .
¶
p40_id_get_leds
Get status of front panel LEDs for interface identification. ¶
See P40_ID_GET_LEDS .
¶
p40_id_get_link
Identify the PCIe link connecting to this controller. ¶
int64_t p40_id_get_link(int fd);See P40_ID_GET_LINK .
¶
p40_id_get_name
Get user-defined name for this data source. ¶
int p40_id_get_name(int fd, char *buffer, size_t size);- size (I)
-
Should be higher than 8, as the string consists of up to 8 ASCII bytes and a
\0terminator is always appended. ¶
See P40_ID_GET_NAME .
¶
p40_id_get_name_unique
Normalize the name returned by the interface to always end with the local link id. ¶
int p40_id_get_name_unique(int fd, char *buffer, size_t size);- size (I)
-
Buffer must hold at least 11 characters (8 for the name, 2 for the link suffix and a
\0terminator). ¶
p40_id_get_regmap
Get version of PCIe register map instantiated on the FPGA. ¶
int64_t p40_id_get_regmap(int fd);See P40_ID_GET_REGMAP .
¶
p40_id_get_rwtest
Read test register. ¶
int64_t p40_id_get_rwtest(int fd);See P40_ID_GET_RWTEST .
¶
p40_id_get_source
Get globally unique source number and format version for this data source. ¶
int p40_id_get_source(int fd);See P40_ID_GET_SOURCE for how to interpret the return value.
¶
p40_id_get_version
Get version of PCIe core instantiated on the FPGA. ¶
int64_t p40_id_get_version(int fd);See P40_ID_GET_VERSION .
¶
p40_id_open
Create a file descriptor to access the device id endpoint with the given device number. ¶
int p40_id_open(int dev);p40_id_set_leds
Set front panel LEDs for interface identification. ¶
int p40_id_set_leds(int fd, int leds);See P40_ID_SET_LEDS .
¶
p40_id_set_name
The following names are not allowed:
-
Longer than 8 characters
-
Starting with a digit
-
Containing colons or periods
-
Ending with either "_0" or "_1" ¶
Assign user-defined name to this data source. ¶
int p40_id_set_name(int fd, const char *buffer);- size (I)
-
Only the first 8 bytes of the string are stored. ¶
See P40_ID_SET_NAME .
¶
p40_id_set_rwtest
Write test register. ¶
int p40_id_set_rwtest(int fd, uint32_t val);See P40_ID_SET_RWTEST .
¶
p40_id_set_source
Set globally unique source number for this data source. ¶
int p40_id_set_source(int fd, uint16_t source_id);See P40_ID_SET_SOURCE .
¶
p40_ecs_bar_size
Query driver for size of a given PCI BAR. ¶
int p40_ecs_bar_size(int fd);- fd (I)
-
File descriptor returned by
p40_ecs_open. ¶
See P40_ECS_GET_BAR_SIZE .
¶
p40_ecs_close
Destroy file descriptor to an open ECS BAR. ¶
void p40_ecs_close(int fd, uint32_t *regs);- fd (I)
-
File descriptor returned by
p40_ecs_open. ¶ - regs (I)
-
Pointer populated by
p40_ecs_open. ¶
p40_ecs_open
Create file descriptor to access an ECS BAR. ¶
int p40_ecs_open(int dev, int bar, uint32_t **regs);- dev (I)
-
Device identifier corresponding to the given board. ¶
- bar (I)
-
PCI BAR to access, valid values are 0 (for the user registers), 2 (for the low-level registers) or the range 8-72 (for reverse BARs, if enabled in the FPGA). ¶
- regs (O)
-
Address of a valid uint32_t * pointer. It is set by the function upon return. ¶
p40_ecs_r32
Read register at given address. ¶
uint32_t p40_ecs_r32(uint32_t volatile *regs, uint32_t addr);- regs (I)
-
Pointer populated by
p40_ecs_open. ¶ - addr (I)
-
Address of register. ¶
p40_ecs_w32
Write register at given address. ¶
void p40_ecs_w32(uint32_t volatile *regs, uint32_t addr, uint32_t val);- regs (I)
-
Pointer populated by
p40_ecs_open. ¶ - addr (I)
-
Address of register. ¶
- val (I)
-
32-bit value to write. ¶
p40_ctrl_close
Destroy file descriptor returned by p40_ctrl_open .
¶
void p40_ctrl_close(int fd);See ctrl_release .
¶
p40_ctrl_exists
Check whether there exists a device controller with the given device number. ¶
int p40_ctrl_exists(int dev);Returns → 0 if the controller exists and < 0 in case of error. ¶
p40_ctrl_get_main_gen_fixed
Get fixed-pattern used by main data generator. ¶
int64_t p40_ctrl_get_main_gen_fixed(int fd);See P40_CTRL_GET_MAIN_GEN_FIXED .
¶
p40_ctrl_get_main_raw_mode
Get whether the main stream input is receiving raw data. ¶
int p40_ctrl_get_main_raw_mode(int fd);See P40_CTRL_GET_MAIN_RAW_MODE .
¶
p40_ctrl_get_meta_alignment
int32_t p40_ctrl_get_meta_alignment(int fd);See P40_CTRL_GET_META_ALIGNMENT .
¶
p40_ctrl_get_pcie_proto
Get PCIe protocol currently negotiated on the given link. ¶
int32_t p40_ctrl_get_pcie_proto(int fd);See P40_CTRL_GET_PCIE_PROTO .
¶
p40_ctrl_get_reset_counters
Read state of counters reset bit. ¶
int p40_ctrl_get_reset_counters(int fd);p40_ctrl_get_reset_default
Read state of default reset bit. ¶
int p40_ctrl_get_reset_default(int fd);p40_ctrl_get_rx_err_total_events
int64_t p40_ctrl_get_rx_err_total_events(int fd, P40_DAQ_STREAM stream_id, RawBankType type);p40_ctrl_get_rx_err_total_since_evid
int64_t p40_ctrl_get_rx_err_total_since_evid(int fd, P40_DAQ_STREAM stream_id, RawBankType type);p40_ctrl_get_rx_last_evid
int64_t p40_ctrl_get_rx_last_evid(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_GET_RX_LAST_EVID .
¶
p40_ctrl_get_spill_buf_size
int p40_ctrl_get_spill_buf_size(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_GET_SPILL_SIZE .
¶
p40_ctrl_get_spill_buf_used
int p40_ctrl_get_spill_buf_used(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_GET_SPILL_USED .
¶
p40_ctrl_get_status
Read error register. ¶
int p40_ctrl_get_status(int fd);See P40_CTRL_GET_STATUS
¶
v2276+
typedef enum {
P40_DAQ_CTRL_STATUS_BIT_MAIN_THR_READY, 0
P40_DAQ_CTRL_STATUS_BIT_MAIN_CDC_READY, 1
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_TXREADY, 2
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_SOPMISS, 3
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_SOPEXTR, 4
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_EOPEARL, 5
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_EOPLATE, 6
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_EVDJUMP, 7
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_EVIDDUP, 8
P40_DAQ_CTRL_STATUS_BIT_MAIN_MUX_READY, 9
P40_DAQ_CTRL_STATUS_BIT_MAIN_ORD_READY, 10
P40_DAQ_CTRL_STATUS_BIT_MAIN_FLU_READY, 11
P40_DAQ_CTRL_STATUS_BIT_MAIN_ERR_CDCJUMP 12
} P40_DAQ_CTRL_STATUS_BIT;p40_ctrl_get_stream_sel
P40_DAQ_STREAM p40_ctrl_get_stream_sel(int fd);See P40_CTRL_GET_STREAM_SEL .
¶
p40_ctrl_get_throt_last_cycles
int64_t p40_ctrl_get_throt_last_cycles(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_throt_last_from_evid
int64_t p40_ctrl_get_throt_last_from_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_throt_last_to_evid
int64_t p40_ctrl_get_throt_last_to_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_throt_total_events
int64_t p40_ctrl_get_throt_total_events(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_throt_total_since_evid
int64_t p40_ctrl_get_throt_total_since_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_trunc_last_cycles
int64_t p40_ctrl_get_trunc_last_cycles(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_trunc_last_from_evid
int64_t p40_ctrl_get_trunc_last_from_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_trunc_last_to_evid
int64_t p40_ctrl_get_trunc_last_to_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_trunc_thres
int p40_ctrl_get_trunc_thres(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_GET_TRUNC_THRES .
¶
p40_ctrl_get_trunc_total_events
int64_t p40_ctrl_get_trunc_total_events(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_trunc_total_since_evid
int64_t p40_ctrl_get_trunc_total_since_evid(int fd, P40_DAQ_STREAM stream_id);p40_ctrl_get_tx_last_evid
int64_t p40_ctrl_get_tx_last_evid(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_GET_TX_LAST_EVID .
¶
p40_ctrl_main_gen_disable
Disable the main data generator (resets the event id counter). ¶
int p40_ctrl_main_gen_disable(int fd);See P40_CTRL_SET_MAIN_GEN_CTL .
¶
p40_ctrl_main_gen_enable
Enable the main data generator. ¶
int p40_ctrl_main_gen_enable(int fd);See P40_CTRL_SET_MAIN_GEN_CTL .
¶
p40_ctrl_main_gen_enabled
Read whether the main stream data generator is enabled. ¶
int p40_ctrl_main_gen_enabled(int fd);p40_ctrl_main_gen_get_limit
Get rate limit of main generator. ¶
int p40_ctrl_main_gen_get_limit(int fd);p40_ctrl_main_gen_get_mode
Get main data generator setting. See P40_MAIN_GEN_MODE .
¶
P40_MAIN_GEN_MODE p40_ctrl_main_gen_get_mode(int fd);p40_ctrl_main_gen_running
Read whether the main stream data generator is running. ¶
int p40_ctrl_main_gen_running(int fd);p40_ctrl_main_gen_set_limit
Set rate limit of main generator. ¶
int p40_ctrl_main_gen_set_limit(int fd, int percent);- percent (I)
-
Value between 0 (no limit) and 100 (no data). ¶
p40_ctrl_main_gen_set_mode
Change main data generator setting. See P40_MAIN_GEN_MODE .
¶
int p40_ctrl_main_gen_set_mode(int fd, P40_MAIN_GEN_MODE);See P40_CTRL_SET_MAIN_GEN_CTL .
¶
p40_ctrl_main_gen_start
Start the main data generator. ¶
int p40_ctrl_main_gen_start(int fd);See P40_CTRL_SET_MAIN_GEN_CTL .
¶
p40_ctrl_main_gen_stop
Stop the main data generator. ¶
int p40_ctrl_main_gen_stop(int fd);See P40_CTRL_SET_MAIN_GEN_CTL .
¶
p40_ctrl_odin_gen_disable
Disable the odin data generator (resets the event id counter). ¶
int p40_ctrl_odin_gen_disable(int fd);See P40_CTRL_SET_ODIN_GEN_CTL .
¶
p40_ctrl_odin_gen_enable
Enable the odin data generator. ¶
int p40_ctrl_odin_gen_enable(int fd);See P40_CTRL_SET_ODIN_GEN_CTL .
¶
p40_ctrl_odin_gen_enabled
Read whether the odin stream data generator is enabled. ¶
int p40_ctrl_odin_gen_enabled(int fd);p40_ctrl_odin_gen_get_limit
Get rate limit of odin generator. ¶
int p40_ctrl_odin_gen_get_limit(int fd);p40_ctrl_odin_gen_running
Read whether the odin stream data generator is running. ¶
int p40_ctrl_odin_gen_running(int fd);p40_ctrl_odin_gen_set_limit
Set rate limit of odin generator. ¶
int p40_ctrl_odin_gen_set_limit(int fd, int percent);- percent (I)
-
Value between 0 (no limit) and 100 (no data). ¶
p40_ctrl_odin_gen_start
Start the odin data generator. ¶
int p40_ctrl_odin_gen_start(int fd);See P40_CTRL_SET_ODIN_GEN_CTL .
¶
p40_ctrl_odin_gen_stop
Stop the odin data generator. ¶
int p40_ctrl_odin_gen_stop(int fd);See P40_CTRL_SET_ODIN_GEN_CTL .
¶
p40_ctrl_open
Create a file descriptor to access the device controller with the given device number. ¶
int p40_ctrl_open(int dev);p40_ctrl_reset_counters
Reset monitoring counters on the controller. ¶
int p40_ctrl_reset_counters(int fd);See P40_CTRL_SET_RESET .
¶
p40_ctrl_reset_default
Reset controller to its default power-on state. ¶
int p40_ctrl_reset_default(int fd);See P40_CTRL_SET_RESET .
¶
p40_ctrl_reset_flush
Request the controller to flush the DMA buffers. ¶
int p40_ctrl_reset_flush(int fd);See P40_CTRL_SET_RESET .
¶
p40_ctrl_reset_logic
Reset the logic on the controller. ¶
int p40_ctrl_reset_logic(int fd);See P40_CTRL_SET_RESET .
¶
p40_ctrl_set_main_gen_fixed
Set fixed-pattern used by main data generator. ¶
int p40_ctrl_set_main_gen_fixed(int fd, uint32_t pattern);See P40_CTRL_SET_MAIN_GEN_FIXED .
¶
p40_ctrl_set_msi_bytes
int p40_ctrl_set_msi_bytes(int fd, uint32_t bytes);See P40_CTRL_SET_MSI_BYTES .
¶
p40_ctrl_set_stream_sel
int p40_ctrl_set_stream_sel(int fd, P40_DAQ_STREAM stream_id);See P40_CTRL_SET_STREAM_SEL .
¶
p40_ctrl_set_trunc_thres
int p40_ctrl_set_trunc_thres(int fd, P40_DAQ_STREAM stream_id, int thres);See P40_CTRL_SET_TRUNC_THRES .
¶
p40_stream_close
Destroy file descriptor created by p40_stream_open .
¶
void p40_stream_close(int fd, void *buffer);See p40driver`dma_release` .
¶
p40_stream_consume_host_bytes
TODO ¶
int p40_stream_consume_host_bytes(int fd, uint32_t bytes);This would only work if the ctrl is in MSI_MAIN or MSI_META mode! ¶
p40_stream_disable
Disable the stream. ¶
int p40_stream_disable(int fd);See P40_STREAM_SET_ENABLE .
¶
p40_stream_disable_bit
Disable individual sub-streams. ¶
int p40_stream_disable_mask(int fd, int mask);p40_stream_enable_bit
Enable individual sub-streams. ¶
int p40_stream_enable_mask(int fd, int mask);p40_stream_enabled
Get whether the stream is enabled. ¶
int p40_stream_enabled(int fd);See P40_STREAM_GET_ENABLE .
¶
p40_stream_exists
Check whether the given controller implements a DMA stream. ¶
int p40_stream_exists(int dev, P40_DAQ_STREAM stream_id);Returns → 0 if the stream exists and < 0 in case of error. ¶
p40_stream_free_host_buf_bytes
Free bytes from circular buffer in host memory. ¶
ssize_t p40_stream_free_host_buf_bytes(int fd, size_t bytes);p40_stream_get_dmabuf
Allocate a dmabuf-compatible file descriptor. ¶
int p40_stream_get_dmabuf(int fd);See P40_STREAM_GET_DMABUF .
¶
p40_stream_get_fpga_buf_bytes
Get size of DMA buffer on FPGA. ¶
int p40_stream_get_fpga_buf_bytes(int fd);p40_stream_get_fpga_buf_bytes_used
Get number of bytes currently used from the DMA buffer on the FPGA. ¶
int p40_stream_get_fpga_buf_bytes_used(int fd);p40_stream_get_fpga_buf_desc_bytes
Get size of each DMA descriptor region on FPGA. ¶
int p40_stream_get_fpga_buf_desc_bytes(int fd);p40_stream_get_fpga_buf_desc_fill_bytes
Get number of bytes used in DMA descriptor currently being filled. ¶
int p40_stream_get_fpga_buf_desc_fill_bytes(int fd);p40_stream_get_fpga_buf_descs
Get number of DMA descriptor regions on FPGA. ¶
int p40_stream_get_fpga_buf_descs(int fd);p40_stream_get_fpga_buf_descs_busy
Get bimask of DMA descriptors in BUSY state. ¶
int64_t p40_stream_get_fpga_buf_descs_busy(int fd);p40_stream_get_fpga_buf_descs_fill
Get bitmask of DMA descriptors in FILL state. ¶
int64_t p40_stream_get_fpga_buf_descs_fill(int fd);p40_stream_get_host_buf_bytes
Get size of DMA buffer in host memory. ¶
int64_t p40_stream_get_host_buf_bytes(int fd);p40_stream_get_host_buf_bytes_used
Get number of bytes currently used from DMA buffer in host memory. ¶
int64_t p40_stream_get_host_buf_bytes_used(int fd);p40_stream_get_host_buf_read_off
Get read offset within DMA buffer in host memory. ¶
int64_t p40_stream_get_host_buf_read_off(int fd);p40_stream_get_host_buf_write_off
Get write offset within DMA buffer in host memory. ¶
int64_t p40_stream_get_host_buf_write_off(int fd);p40_stream_get_locker
Return PID of the process currently locking the stream, if any, or 0 if none. ¶
pid_t p40_stream_get_locker(int fd);See also P40_STREAM_GET_LOCKER .
¶
p40_stream_get_meta_packing
int32_t p40_stream_get_meta_packing(int fd);See P40_STREAM_GET_META_PACKING .
¶
p40_stream_get_reset_default
Read state of default reset bit. ¶
int p40_stream_get_reset_default(int fd);p40_stream_get_status
Get stream status flags. ¶
int64_t p40_stream_get_status(int fd, int sel);See P40_STREAM_GET_STATUS .
¶
See P40_STREAM_GET_RESET .
¶
v2276+
typedef enum {
P40_DAQ_STREAM_STATUS_BIT_SNK_READY,
P40_DAQ_STREAM_STATUS_BIT_HOST_MAP_BOOTSTRAP,
P40_DAQ_STREAM_STATUS_BIT_DMA_WR_RX_READY,
P40_DAQ_STREAM_STATUS_BIT_SOP_SEEN,
P40_DAQ_STREAM_STATUS_BIT_EOP_SEEN,
P40_DAQ_STREAM_STATUS_BIT_EOP_SEND,
P40_DAQ_STREAM_STATUS_BIT_EOP_SENT,
P40_DAQ_STREAM_STATUS_BIT_EOP_WAIT,
P40_DAQ_STREAM_STATUS_BIT_EOP_ACK,
P40_DAQ_STREAM_STATUS_BIT_EOP_ACKED,
P40_DAQ_STREAM_STATUS_BIT_EOH_SEEN,
P40_DAQ_STREAM_STATUS_BIT_SNK_PAD_ALIGNING,
P40_DAQ_STREAM_STATUS_BIT_SNK_PAD_ALIGNED,
P40_DAQ_STREAM_STATUS_BIT_HOST_BUF_READY
} P40_DAQ_STREAM_STATUS_BIT;p40_stream_id_to_meta_mask
Derive metadata bit position corresponding to the given stream. ¶
int p40_stream_id_to_meta_mask(int dev, P40_DAQ_STREAM stream_id);p40_stream_id_to_name
Convert numerical stream identifier to string. See P40_DAQ_STREAM .
¶
const char *p40_stream_id_to_name(P40_DAQ_STREAM stream_id);p40_stream_lock
Lock a stream to the current process until the process terminates or the lock is explicitly released by p40_stream_unlock . Only the process holding the lock, if any, is allowed to move the read pointer for this stream.
¶
int p40_stream_lock(int fd);See also P40_STREAM_LOCK .
¶
p40_stream_map
Map DMA circular buffer in memory. ¶
void *p40_stream_map(int fd);This function automatically does double-buffering of the circular buffer in order to avoid complicated wrap-around logic in the user application.
See also p40driver`dma_mmap` .
¶
p40_stream_name_to_id
Convert stream name to numerical id. See P40_DAQ_STREAM .
¶
P40_DAQ_STREAM p40_stream_name_to_id(const char *name);p40_stream_open
Create file descriptor to control a DMA stream on given interface. See P40_DAQ_STREAM .
¶
int p40_stream_open(int dev, P40_DAQ_STREAM stream_id);See p40driver`dma_open` .
¶
p40_stream_poll
Query the stream for data. ¶
ssize_t p40_stream_poll(int fd, loff_t *pread_off, int msecs);This is a low-level function, it just returns how many bytes are available to read.
If pread_off is not null, it must point to the base of the buffer as initialized
by p40_stream_map , then, on return, it will be adjusted to point to the new data.
Returns 0 on timeout, < 0 on error.
See also p40driver`dma_poll` .
¶
p40_stream_reset_default
Reset stream to its default power-on state. ¶
int p40_stream_reset_default(int fd);See P40_STREAM_SET_RESET .
¶
p40_stream_reset_flush
Request the stream to flush its DMA buffers. ¶
int p40_stream_reset_flush(int fd);See P40_STREAM_SET_RESET .
¶
p40_stream_reset_logic
Reset the logic on the stream. ¶
int p40_stream_reset_logic(int fd);See P40_STREAM_SET_RESET .
¶
p40_stream_set_meta_packing
int p40_stream_set_meta_packing(int fd, int32_t mpf);See P40_STREAM_SET_META_PACKING .
¶
p40_stream_unlock
Unlock a stream previously locked by the current process using p40_stream_lock .
¶
int p40_stream_unlock(int fd);See also P40_STREAM_UNLOCK .
¶
p40_stream_unmap
Remove memory mapping created by p40_stream_map .
¶
void p40_stream_unmap(int fd, void *buffer);P40_DAQ_STREAM
typedef enum {
P40_DAQ_STREAM_NULL = -1,
P40_DAQ_STREAM_MAIN = 0,
P40_DAQ_STREAM_META,
P40_DAQ_STREAM_ODIN0,
P40_DAQ_STREAM_ODIN1,
P40_DAQ_STREAM_ODIN2,
P40_DAQ_STREAM_ODIN3,
P40_DAQ_STREAM_ODIN4
} P40_DAQ_STREAM;P40_MAIN_GEN_MODE
Main stream data generator mode. ¶
typedef enum {
P40_MAIN_GEN_MODE_ERROR = -1,
P40_MAIN_GEN_MODE_FRAGS,
P40_MAIN_GEN_MODE_FIXED
} P40_MAIN_GEN_MODE;Kernel-level code cross-reference
daq_release
static int daq_release(struct inode *inode, struct file *filp);Since this function is called when the last handle to a given stream is closed, the stream lock is also released here, if held by the current process. ¶
dma_stream_configure
static int dma_stream_configure(int ifc_num, struct pcie40_dma_stream *stream, size_t map_base, size_t map_max_entries, size_t buf_requested_bytes, int numa_node);pcie40_dma_stream
Kernelspace representation of a DMA stream between the PCIe40 FPGA and upstream memory. ¶
pcie40_ecs_init
Register ECS device class. ¶
int pcie40_ecs_init(void);This functions registers a dedicated device class used to create ECS device files. ¶
dma_map_alloc
static int dma_map_alloc(struct pci_dev *pci_dev, struct pcie40_dma_map *map, int numa_node);dma_map_free
static int dma_map_free(struct pci_dev *pci_dev, struct pcie40_dma_map *map, int reuse);dma_map_read_entry_base
static inline dma_addr_t dma_map_read_entry_base(struct pcie40_dma_map *map, int i);dma_map_read_entry_size
static inline size_t dma_map_read_entry_size(struct pcie40_dma_map *map, int i);pcie40_device_ids
static const struct pci_device_id pcie40_device_ids[] = {
{ PCI_DEVICE(0x10DC, 0xCE40), }, PCIe40 (Arria10)
{ PCI_DEVICE(0x10DC, 0xCE80), }, PCIe400 (Agilex-F)
{ PCI_DEVICE(0x10DC, 0xCECD), }, PCIe400 (Agilex-I)
{ 0, }
};
MODULE_DEVICE_TABLE(pci, pcie40_device_ids);pcie40_ecs_probe
Initialize ECS BARs and create device files. ¶
int pcie40_ecs_probe(struct pci_dev *dev, const struct pci_device_id *id);This function allocates a pcie40_ecs_state instance to keep ECS-specific state.
¶
It then requests exclusive access to the ECS BARs
¶
and allocates a range of minor numbers for its character devices.
¶
One such device is created for BAR0
¶
and a second one for BAR2.
¶
pcie40_exit
Unregister PCIe driver and uninitialize subdrivers. ¶
static void __exit pcie40_exit(void)pcie40_init
Initialize subdrivers and register PCIe driver with kernel. ¶
static int __init pcie40_init(void);The first module to be initialized is the ECS, using pcie40_ecs_init .
¶
Followed by the DAQ, using pcie40_daq_init .
¶
The driver is registered with the kernel using pci_register_driver, its argument also contains the PCI device ids that correspond to the PCIe40 firmware (see pcie40_device_ids ).
¶
pcie40_probe
Scan PCI bus to detect PCIe40 board. ¶
static int pcie40_probe(struct pci_dev *dev, const struct pci_device_id *id);This function allocates a pcie40_state instance used to track the state of this device within the kernel.
¶
The state is initialized with the position and size of all PCI BARs.
¶
BAR1 is always mapped inside the kernel as it’s used directly by DAQ interface.
¶
Using this mapping, the driver ensures that PCIe registers on the FPGA can be accessed.
¶
Then it reads the pcie.dma_ctrl.link_id register to identify which PCIe link from the FPGA is being probed.
¶
Using this information, a unique interface identifier is allocated to the PCIe link.
¶
Interfaces with PCIe link 0 get an even interface id.
¶
Interfaces with PCIe link 1 get an odd interface id.
¶
The driver always allocates the lowest available interface id.
¶
Finally it calls the probing logic of the subdrivers via pcie40_ecs_probe and pcie40_daq_probe .
¶
After initializing the subdrivers, this function always returns success, this is to ensure that pcie40_remove is always called also in case only some subdrivers are loaded.
¶
pcie40_state_release
Remove PCIe40 interface from kernel (once its reference count is zero). ¶
First the submodules are uninitialized using pcie40_daq_remove and pcie40_ecs_remove .
¶
BAR1 is unmapped using iounmap.
¶
Finally the PCI device is disabled
¶
and the pcie40_state memory is freed.
¶
daq_emu_thread
Thread loop generating emulated board data. ¶
static int daq_emu_thread(void *data)- data (I)
-
Opaque pointer to interface state ¶
One instance of this thread is spawned for each emulated interface. ¶ The thread loops until the module is unloaded.
while (!kthread_should_stop()) { ... }¶ Every loop iteration reads the emulated control registers
mmr_reset = pcie40_read32_ctrl(state->common, P40_DMA_CTRL_OFF_RESET);
mmr_main_gen_ctl = pcie40_read32_ctrl(state->common, P40_DMA_CTRL_OFF_MAIN_GEN_CTL);
mmr_main_gen_fixed = pcie40_read32_ctrl(state->common, P40_DMA_CTRL_OFF_MAIN_GEN_FIXED);
mmr_main_raw_mode = pcie40_read32_ctrl(state->common, P40_DMA_CTRL_OFF_MAIN_RAW_MODE);
mmr_main_enable = pcie40_read32_stream(&state->main_stream, P40_DMA_DAQ_STREAM_OFF_ENABLE);
mmr_meta_enable = pcie40_read32_stream(&state->meta_stream, P40_DMA_DAQ_STREAM_OFF_ENABLE);
mpf = pcie40_read32_stream(&state->main_stream, P40_DMA_DAQ_STREAM_OFF_META_PACKING);Then uses these values to drive its behaviour for the rest of the current emulation cycle. ¶
dma_stream_emu_write
Emulate a dma write into a given stream. ¶
static ssize_t dma_stream_emu_write(
struct pcie40_dma_stream *stream, int *map_idx, uint32_t *buf_off, const void *from, size_t bytes);The write is started only if sufficient space is available. ¶ If not, a negative value is immediately returned. Its magnitude is the number of bytes missing. ¶ A write is split across multiple DMA buffers if the current one does not have sufficient space available. ¶
Appendix A: PCIe40 backpressure behaviour
Data fragments produced in the FPGA by an onboard TELL40 instance reach the corresponding PCIe output stage via a clock-crossing FIFO. The write clock domain is defined by the specific TELL40 data processing implementation. The read clock domain is the 220MHz used internally by the DMA controller.
The pcie.dma_ctrl.trunc_thres register that controls the truncation threshold operates on the fill level of this FIFO. Its value is 32 upon power-on reset (or default reset) but it can be configured via the trunc_threshold setting in the configuration file (up to 64).
Depending on the fill level, three scenarios are possible:
- FIFO is full
-
The Avalon
snk_readysignal from the controller is deasserted to report backpressure. Since no further data can be transmitted, the current fragment is throttled (its type is changed toDaqErrorFragmentThrottledand its size is cut at the size transmitted so far). Data words from the TELL40 are discarded so long as backpressure persists, but the last transmitted and the last seen event IDs are recorded in the DMA controller. When space becomes available, the controller generates zero-sizeDaqErrorFragmentThrottledfragments for all missing event IDs. The ready signal is reasserted when the transmitted event ID sequence has caught up and normal data transmission can resume. - FIFO is above threshold
-
The Avalon
snk_readysignal from the controller remains asserted, but an additionalsnk_truncsignal is also asserted. The latter instructs the TELL40 that, although space is still available, backpressure is imminent and the next fragment should be truncated to reduce the likelyhood of throttling. Truncated fragments have a size of 0 and a type ofDaqErrorFragmentTruncated. Data words from the TELL40 pass through the FIFO unmodified, throttle engages automatically if space is exhausted. - FIFO is below threshold
-
Data words from the TELL40 pass through the FIFO unmodified, truncation engages automatically if the threshold is crossed and throttle engages automatically if space is exhausted.
Appendix B: PCIe40 DMA data format
Depending on the FPGA configuration, the board can produce data according to three different data formats.
Main data only (no metadata)
When raw mode is off and the metadata stream is not enabled, the FPGA will emit readout data as a continuous stream of fragments aligned to 32-byte (256-bits) boundaries. Each fragment is laid out in memory as follows, starting with the event ID:
The class pcie40_frg_hdr is provided for convenience in order to access the different fields preceding the payload.
¶
Optimized (with metadata acceleration)
The previous data format favours simplicity over performance, when the optional metadata stream is enabled in the firmware, several adjustments are made to the output data format to increase its efficiency. These are:
-
The event ID is omitted from the fragment header
-
Fragments are aligned on 8 byte boundaries, instead of 32.
This results in the following fragment format:
A global fragment header can be inspected using an instance of the daq40_frg_hdr class.
¶
Appendix C: DIM interface
{consumer}/blocks
SVC(I:2) A 64-bit integer (represented as two 32-bit values).
¶
Contains the number of blocks consumed during the last run. ¶
{consumer}/bytes
SVC(I:2) A 64-bit integer (represented as two 32-bit values).
¶
Contains the number of bytes consumed during the last run. ¶
{consumer}/evid
SVC(I:2) A 64-bit integer (represented as two 32-bit values).
¶
Contains the unique EventID of the last fragment that was consumed. ¶
{consumer}/frags
SVC(I:2) A 64-bit integer (represented as two 32-bit values).
¶
Contains the number of fragments consumed during the last run. ¶
{frgwriter}/set_subdet_info
CMD(I) Sets an arbitrary subdetector-defined 16bit value, this quantity will be stored in a dedicated field in the .frg file header.
¶
{fsm}/action
CMD(C) Character string indicating an FSM transition to execute.
¶
The following commands are accepted:
-
"configure"
-
"start"
-
"stop"
-
"reset" ¶
{daqserver}/build
SVC(C) Software build version and release identifiers, separated by a dash sign.
¶
Appendix E: PCIe40 driver IOCTL interface
P40_CTRL_GET_MAIN_GEN_CTL
See pcie.dma_ctrl.main_gen_ctl .
¶
#define P40_CTRL_GET_MAIN_GEN_CTL _IOR('c', 4, uint64_t)#define P40_CTRL_GET_ODIN_GEN_CTL _IOR('c', 11, uint64_t)P40_CTRL_GET_MAIN_MSI_CYCLES
uint64_t)//+P40_CTRL_GET_META_MSI_BYTES
#define P40_CTRL_GET_MSI_CYCLES _IOR('c', 18, uint64_t)#define P40_CTRL_GET_META_MSI_BYTES _IOR('c', 19,uint64_t)//+P40_CTRL_SET_META_MSI_BYTES
#define P40_CTRL_SET_META_MSI_BYTES _IOW('c', 20,uint64_t)//+P40_CTRL_GET_META_MSI_CYCLES
#define P40_CTRL_GET_META_MSI_CYCLES _IOR('c', 21,uint64_t)//+P40_CTRL_GET_META_MSI_BLOCKS
#define P40_CTRL_GET_META_MSI_BLOCKS _IOR('c', 22,uint64_t)//+P40_CTRL_SET_META_MSI_BLOCKS
#define P40_CTRL_SET_META_MSI_BLOCKS _IOW('c', 23,P40_CTRL_GET_MAIN_RAW_MODE
See pcie.dma_ctrl.main_raw_mode .
¶
uint64_t)//+P40_CTRL_GET_META_PACKING
#define P40_CTRL_GET_MAIN_RAW_MODE _IOR('c', 8, uint64_t)#define P40_CTRL_GET_META_PACKING _IOR('c', 9,uint64_t)//+P40_CTRL_SET_META_PACKING
#define P40_CTRL_SET_META_PACKING _IOW('c', 10,P40_CTRL_GET_PCIE_PROTO
Reports the result of the PCIe link negotiation. The returned value is 32 bits.
-
bits[25..20] PCIe link width reported by the operating system.
-
bits[19..16] PCIe link gen reported by the operating system.
-
bits[9..4] PCIe link width reported by the FPGA (see
pcie.dma_ctrl.pcie_lanes). -
bits[3..0] PCIe link gen reported by the FPGA (see
pcie.dma_ctrl.pcie_gen). ¶
#define P40_CTRL_GET_PCIE_PROTO _IOR('c', 15, uint64_t)P40_CTRL_GET_RX_ERR_TOTAL_SINCE_EVID
#define P40_CTRL_GET_RX_ERR_TOTAL_SINCE_EVID _IOR('c', 44, uint64_t)P40_CTRL_GET_RX_LAST_EVID
#define P40_CTRL_GET_RX_LAST_EVID _IOR('c', 41, uint64_t)#define P40_CTRL_GET_TX_LAST_EVID _IOR('c', 42, uint64_t)P40_CTRL_GET_SPILL_SIZE
See regmap`pcie.dma_ctrl.spill_buf_size`
¶
#define P40_CTRL_GET_SPILL_SIZE _IOR('c', 24, uint64_t)P40_CTRL_GET_SPILL_USED
See regmap`pcie.dma_ctrl.spill_buf_used`
¶
#define P40_CTRL_GET_SPILL_USED _IOR('c', 25, uint64_t)P40_CTRL_GET_STREAM_SEL
See regmap`pcie.dma_ctrl.spill_buf_sel`
¶
#define P40_CTRL_GET_STREAM_SEL _IOR('c', 39, uint64_t)P40_CTRL_GET_THROT_TOTAL_SINCE_EVID
#define P40_CTRL_GET_THROT_TOTAL_SINCE_EVID _IOR('c', 34, uint64_t)P40_CTRL_GET_TRUNC_TOTAL_SINCE_EVID
#define P40_CTRL_GET_TRUNC_TOTAL_SINCE_EVID _IOR('c', 29, uint64_t)P40_CTRL_SET_MAIN_GEN_CTL
See pcie.dma_ctrl.main_gen_ctl .
¶
#define P40_CTRL_SET_MAIN_GEN_CTL _IOW('c', 5, uint64_t)P40_CTRL_SET_ODIN_GEN_CTL
See pcie.dma_ctrl.odin_gen_ctl .
¶
#define P40_CTRL_SET_ODIN_GEN_CTL _IOW('c', 12, uint64_t)#define P40_ODIN_GEN_BIT_ENABLE (0)
#define P40_ODIN_GEN_BIT_RUNNING (1)
#define P40_ODIN_GEN_BIT_LIMIT (2)
#define P40_CTRL_GET_MSI_MODE _IOR('c', 13,uint64_t)//+P40_CTRL_SET_MSI_MODE
#define P40_CTRL_SET_MSI_MODE _IOW('c', 14,P40_CTRL_SET_STREAM_SEL
See regmap`pcie.dma_ctrl.spill_buf_sel`
¶
#define P40_CTRL_SET_STREAM_SEL _IOW('c', 40, uint64_t)P40_ID_GET_LINK
The value returned by this IOCTL encodes both the PCI topological address and the FPGA link number, as follows:
-
bits[31..24]: PCI bus number
-
bits[23..16]: PCI slot number
-
bits[15..8]: PCI function number
-
bits[7..0]: Local PCI link number (see
pcie.dma_ctrl.link_id) ¶
#define P40_ID_GET_LINK _IOR('i', 6, uint64_t)P40_ID_GET_REGMAP
The returned value is 64 bits.
-
bits[63..32]: register map version expected by the driver
-
bits[31..0]: value of the FPGA register
pcie.dma_ctrl.regmap¶
#define P40_ID_GET_REGMAP _IOR('i', 3, uint64_t)P40_ID_GET_SOURCE
See pcie.dma_ctrl.link_id
The value returned by this IOCTL encodes both the source identifier and its data format version, as follows:
-
bits[23..16]: source version
-
bits[15..11]: source subsystem number
-
bits[10..0]: source number within the subsystem ¶
#define P40_ID_GET_SOURCE _IOR('i', 11, uint64_t)P40_ID_GET_VERSION
The returned value is 64 bits.
-
bits[48..32]: DMA core version expected by the driver
-
bits[31..0]: value of the FPGA register
pcie.dma_ctrl.version¶
#define P40_ID_GET_VERSION _IOR('i', 4, uint64_t)P40_ID_SET_SOURCE
Sets the source identifier for a given data source using the lowest 16 bits of the input argument, other bits are ignored.
The value is stored in the SourceID portion of the pcie.dma_ctrl.link_id FPGA register.
¶
#define P40_ID_SET_SOURCE _IOW('i', 12, uint64_t)P40_STREAM_FREE_HOST_BUF_BYTES
#define P40_STREAM_FREE_HOST_BUF_BYTES _IOWR('m', 4, uint64_t)If the stream is locked, only the owner process is allowed to change the read pointer, EPERM is returned otherwise.
¶
Note that value is internally rounded up to its nearest multiple of 4.
¶
P40_STREAM_GET_FPGA_BUF_DESCS_BUSY
#define P40_STREAM_GET_FPGA_BUF_DESCS_BUSY _IOR('s', 11, uint64_t)P40_STREAM_GET_FPGA_BUF_DESCS_FILL
#define P40_STREAM_GET_FPGA_BUF_DESCS_FILL _IOR('s', 9, uint64_t)P40_STREAM_GET_FPGA_BUF_DESC_BYTES
#define P40_STREAM_GET_FPGA_BUF_DESC_BYTES _IOR('s', 8, uint64_t)P40_STREAM_GET_FPGA_BUF_DESC_FILL_BYTES
#define P40_STREAM_GET_FPGA_BUF_DESC_FILL_BYTES _IOR('s', 10, uint64_t)P40_STREAM_GET_HOST_BUF_BYTES
for huge buffer allocation on demand, size is zero before allocation ¶
#define P40_STREAM_GET_HOST_BUF_BYTES _IOR('m', 1, uint64_t)P40_STREAM_GET_HOST_BUF_BYTES_USED
uint64_t)//+P40_STREAM_GET_HOST_MSI_COUNT
#define P40_STREAM_GET_HOST_BUF_BYTES_USED _IOR('m', 2, uint64_t)#define P40_STREAM_GET_HOST_MSI_COUNT _IOR('m', 3,P40_STREAM_GET_META_PACKING
See regmap`pcie.dma_stream.meta_packing` .
¶
#define P40_STREAM_GET_META_PACKING _IOR('s', 14, uint64_t)P40_STREAM_GET_STATUS
See regmap`pcie.dma_daq_stream.ready`
¶
#define P40_STREAM_GET_STATUS _IOR('s', 3, uint64_t)P40_STREAM_LOCK
#define P40_STREAM_LOCK _IO('m', 6)The stream can be locked only if no other process currently holds the lock, otherwise EBUSY is returned.
¶
The same process can lock the same stream multiple times. In this case all subsequent calls will also succeed, but since locks are not reference counted the first unlock operation will immediately unlock the stream regardless of how many times it was locked.
¶
P40_STREAM_SET_STATUS_SEL
See regmap`pcie.dma_daq_stream.ready`
¶
#define P40_STREAM_SET_STATUS_SEL _IOW('s', 16, uint64_t)P40_STREAM_UNLOCK
#define P40_STREAM_UNLOCK _IO('m', 7)The stream can only be unlocked by the process currently holiding the lock, otherwise EPERM is returned.
¶
Appendix F: PCIe40 PCIe register map
pcie_regmap_version
version = 0x20181106Current version of the register map. This value must be the same between the driver and the firmware. ¶
pcie_core_version
version = 0x0602Expected version of the FPGA PCIe core. This value must be the same between the driver and the firmware. ¶
pcie_dma_daq_odin0_buf_qsys_base
base = 0x300000QSys base address of first odin FPGA memory buffer.
¶
pcie_dma_daq_odin0_stream_qsys_base
base = 0x3000QSys base address of first odin stream controller.
¶
pcie_dma_daq_odin1_buf_qsys_base
base = 0x400000QSys base address of second odin FPGA memory buffer.
¶
pcie_dma_daq_odin1_stream_qsys_base
base = 0x4000QSys base address of second odin stream controller.
¶
pcie_dma_daq_odin2_buf_qsys_base
base = 0x500000QSys base address of third odin FPGA memory buffer.
¶
pcie_dma_daq_odin2_stream_qsys_base
base = 0x5000QSys base address of third odin stream controller.
¶
pcie_dma_daq_odin3_buf_qsys_base
base = 0x600000QSys base address of fourth odin FPGA memory buffer.
¶
pcie_dma_daq_odin3_stream_qsys_base
base = 0x6000QSys base address of fourth odin stream controller.
¶
pcie_dma_daq_odin4_buf_qsys_base
base = 0x700000QSys base address of fifth odin FPGA memory buffer.
¶
pcie_dma_daq_odin4_stream_qsys_base
base = 0x7000QSys base address of fifth odin stream controller.
¶
pcie.dma_ctrl.err_total_events
off_rx_err_total_events = 0x68(RO) Events lost because of malformed input data. ¶
pcie.dma_ctrl.err_total_since_evid
off_rx_err_total_since_evid_lo = 0x6C
off_rx_err_total_since_evid_hi = 0x70(RO) EVID when an input format error was first detected since the last counter reset. ¶
pcie.dma_ctrl.leds_id
off_leds_id = 0x18(RW) Control frontplate identification LEDs according to the following format:
-
bit 8: override (if this bit is zero the led colors remain those set by the LLI)
-
bits[6..4]: binary RGB value for second LED
-
bits[2..0]: binary RGB value for first LED ¶
pcie.dma_ctrl.link_id
off_link_id = 0x14(RW) Link identifier according to the following format:
-
bits[31..24]: source version (read-only)
-
bits[23..8]: global link identifier (SourceID)
-
bits[23..19]: subsystem number
-
bits[18..8]: source number within the subsystem
-
-
bits[7..0]: local link identifier (read-only)
The value of the local link identifier is hardcoded in the FPGA and currently can only be either 0 (for the primary PCIe link in a given board) or 1 (for the secondary). ¶
pcie.dma_ctrl.main_gen_ctl
off_main_gen_ctl = 0x2C(RW) Main data generator control, the following bits are defined:
-
bits[30..24]: limiter% (0 = no limit, 100 = no data)
-
bit 3: set rate limiter
-
bit 2: generate fixed pattern/pseudo fragments
-
bit 1: start/stop generator
-
bit 0: enable/disable generator ¶
pcie.dma_ctrl.main_gen_fixed
off_main_gen_fixed = 0x30(RW) 32-bit pattern used when generating data in fixed-pattern mode. ¶
pcie.dma_ctrl.main_raw_mode
off_main_raw_mode = 0x34(RO) Indicates whether the input to the main stream is in raw (1) or normal (0) format.
¶
pcie.dma_ctrl.name
off_name_lo = 0x1C
off_name_hi = 0x20(RW) Globally unique board name within the system, it is a 8-byte ASCII-encoded string.
Both interfaces on one board can share the same value for this field. To uniquely reference a given interface when calling p40_id_find , append either _0 or :0 to the name for the first interface and either _1 or :1 for the second.
¶
|
pcie.dma_ctrl.odin_gen_ctl
off_odin_gen_ctl = 0x4C(RW) Odin data generator control, the following bits are defined:
-
bits[30..24]: limiter% (0 = no limit, 100 = no data)
-
bit 2: set rate limiter
-
bit 1: start/stop generator
-
bit 0: enable/disable generator ¶
pcie.dma_ctrl.pcie_gen
off_pcie_gen = 0x80(RO) PCI-express protocol version, depending on this value, the serial link speed will be as follows:
-
2.5 Gb/s
-
5.0 Gb/s
-
8.0 Gb/s ¶
pcie.dma_ctrl.pcie_ltssm
off_pcie_ltssm = 0x7C(RW) PCI-express link training state machine log. Each read returns a word in the form:
-
bits[31..24]: entry position in log
-
bits[23..8]: state transition length (in 10ns cycles)
-
bits[7..0]: LTSSM state
Write 0 to move the read position to the beginning of the log (subsequent reads will return subsequent positions). Write 0xFFFFFFFF to clear the log. ¶
pcie.dma_ctrl.regmap
off_regmap = 0x04(RO) Version of the register map implemented by the FPGA, it must match pcie_regmap_version .
¶
pcie.dma_ctrl.reset
off_reset = 0x24(RW) Reset register for the controller. The reset that is performed depends on which bit is written to it:
-
bit 0: reset everything to its power-on default state
-
bit 1: reset only logic processes and buffers, do not revert user-controlled registers
-
bit 2: only flush the buffers
-
bit 3: only reset counters ¶
pcie.dma_ctrl.rwtest
off_rwtest = 0x00(RW) Test register, can be used to check that reads and writes are correctly handled by the Avalon bus on the FPGA. ¶
pcie.dma_ctrl.rx_last_evid
off_rx_last_evid_lo = 0x94
off_rx_last_evid_hi = 0x98(RO) EVID of last valid event received. ¶
pcie.dma_ctrl.throt_last_cycles
off_throt_last_cycles = 0xD8(RO) Duration, in consecutive cycles, of last time the input ready signal was deasserted. ¶
pcie.dma_ctrl.throt_last_from_evid
off_throt_last_from_evid_lo = 0xDC
off_throt_last_from_evid_hi = 0xE0(RO) EVID when the input ready signal was last deasserted. ¶
pcie.dma_ctrl.throt_last_to_evid
off_throt_last_to_evid_lo = 0xE4
off_throt_last_to_evid_hi = 0xE8(RO) EVID when the input ready signal was last asserted. ¶
pcie.dma_ctrl.throt_total_events
off_throt_total_events = 0xCC(RO) Events lost to throttle since the last counter reset. ¶
pcie.dma_ctrl.throt_total_since_evid
off_throt_total_since_evid_lo = 0xD0
off_throt_total_since_evid_hi = 0xD4(RO) EVID when the input ready signal was first deasserted since the last counter reset. ¶
pcie.dma_ctrl.trunc_last_cycles
off_trunc_last_cycles = 0xB8(RO) Duration, in consecutive cycles, of last time the truncation signal was asserted. ¶
pcie.dma_ctrl.trunc_last_from_evid
off_trunc_last_from_evid_lo = 0xBC
off_trunc_last_from_evid_hi = 0xC0(RO) EVID when the truncation signal was last asserted. ¶
pcie.dma_ctrl.trunc_last_to_evid
off_trunc_last_to_evid_lo = 0xC4
off_trunc_last_to_evid_hi = 0xC8(RO) EVID when the truncation signal was last deasserted. ¶
pcie.dma_ctrl.trunc_thres
off_trunc_thres = 0xA8(RW) Spill buffer threshold level, in words, above which the truncation signal is triggered. ¶
pcie.dma_ctrl.trunc_total_events
off_trunc_total_events = 0xAC(RO) Events lost to truncation since the last counter reset. ¶
pcie.dma_ctrl.trunc_total_since_evid
off_trunc_total_since_evid_lo = 0xB0
off_trunc_total_since_evid_hi = 0xB4(RO) EVID when the truncation signal was first asserted since the last counter reset. ¶
pcie.dma_ctrl.tx_last_evid
off_tx_last_evid_lo = 0xEC
off_tx_last_evid_hi = 0xF0(RO) EVID of last valid event transmitted. ¶
pcie.dma_ctrl.version
off_version = 0x08(RO) Version of the DMA module on the FPGA. It must match pcie_core_version .
¶
pcie.dma_daq_stream.enable
off_enable = 0x00(RW) Controls which DMA channels are enabled (1) or not (0). ¶
pcie.dma_daq_stream.fpga_buf_bytes
off_fpga_buf_bytes = 0x0C(RO) Size of the DMA buffer on the FPGA (in bytes) ¶
pcie.dma_daq_stream.fpga_buf_desc_bytes
off_fpga_buf_desc_bytes = 0x14(RO) Size of each descriptor region inside the DMA buffer on the FPGA (in bytes) ¶
pcie.dma_daq_stream.fpga_buf_desc_fill_bytes
off_fpga_buf_desc_fill_bytes = 0x20(RO) Bytes used in the description region currently being filled. ¶
pcie.dma_daq_stream.fpga_buf_descs
off_fpga_buf_descs = 0x10(RO) Number of descriptor regions inside the DMA buffer on the FPGA ¶
pcie.dma_daq_stream.fpga_buf_descs_busy
off_fpga_buf_descs_busy_lo = 0x24
off_fpga_buf_descs_busy_hi = 0x28(RO) Descriptor state bitmask, if a bit is 1 the corresponding descriptor is in the BUSY state. ¶
pcie.dma_daq_stream.fpga_buf_descs_fill
off_fpga_buf_descs_fill_lo = 0x18
off_fpga_buf_descs_fill_hi = 0x1C(RO) Descriptor state bitmask, if a bit is 1 the corresponding descriptor is in the FILL state (at most one descriptor can be in this state at any moment). ¶
pcie.dma_daq_stream.host_buf_read_off
off_host_buf_read_off_lo = 0x30
off_host_buf_read_off_hi = 0x40(RW) Current DMA read offset inside the DMA buffer on the host. ¶
pcie.dma_daq_stream.host_buf_write_off
off_host_buf_write_off_lo = 0x2C
off_host_buf_write_off_hi = 0x3C(RO) Current DMA write offset inside the DMA buffer on the host. ¶
pcie.dma_daq_stream.host_map_entries
off_host_map_entries = 0x34(RW) Number of entries inside the memory map representing the DMA host buffer. ¶
pcie.dma_daq_stream.host_map_pages
off_host_map_pages = 0x38(RO) Number of pages inside the memory map representing the DMA host buffer. ¶
Appendix G: BAR0 PCIe register map
| This list is partial, refer to the control system datapoints for the full register map. |
bar0.top.board_id
board_id = 0xFFCUnique board identifier, on a PCIe40 this is derived from the lowest 32 bits of the FPGA serial number. ¶
bar0.top.build_id
build_id = 0x400-0x9FFContains an autogenerated string formatted according to Build ID ROM format. ¶
bar0.top.ext_clock_40mhz
ext_clock_40mhz = 0x310Clock configuration register, the following values are possible:
-
internal
-
external
-
tfc
-
custom ¶
bar0.top.fw_config_enb
fw_config_enb = 0x0BCIncludes several fields:
-
bits[0]: FW_with_FE
-
bits[4]: FW_with_PRBS
-
bits[8]: sol40_with_hardcoded_sbtype
-
bits[9]: tfc_sfp_rx_ready_0[3]
-
bits[10]: tfc_sfp_rx_ready_1[3]
-
bits[15..12]: SCA_VERSION
-
bits[21..16]: SCA_MAX_PER_LINK
-
bits[22]: tfc_sfp_rx_ready_0[1]
-
bits[23]: tfc_sfp_rx_ready_1[1]
-
bits[24]: SOL40_FW_with_SODIN
-
bits[25]: tfc_sfp_rx_ready_0[0]
-
bits[26]: tfc_sfp_tx_ready_0
-
bits[27]: tfc_sfp_rx_ready_1[0]
-
bits[28]: tfc_sfp_tx_ready_1
-
bits[29]: tfc_sfp_rx_ready_0[2]
-
bits[30]: tfc_sfp_rx_ready_1[2]
-
bits[31]: link_ordered ¶
Appendix H: BAR2 PCIe register map
| This list is partial, refer to the low-level interface for the full register map. |
bar2.lli.status
status = 0x120After the FPGA is programmed, this register is 0, after the PLLs and clocks have been configured correctly, this register reads 1. ¶
bar2.lli_0.ext_clock_select
ext_clock_select = 0x10140Clock tree selector, set bit 2 to enable the LHC clock input. https://gitlab.cern.ch/lhcb-daq40/lhcb-daq40-software/blob/v6.2.3/common/bar2_regmap.cfg#L19[¶]
Appendix J: Build ID ROM format
The ROM contents are generated at compile-time and consist of a list of strings, each terminated by a semicolon.
2019-01-27T08:15:46; (1) admin@lbminidaq2-17.dyndns.cern.ch; (2) 16.1.0 196 10/24/2016; (3) :master:7be53db:v20190110-1; (4) lli-amc40:master:2fc7f22:v6.0.0; (5) lli-simulation:master:c3b617a:v6.0.0; lli-gbt:master:2681257:v6.0.1; tell40:master:536a2d0:v6.0.0; ... sodin:master:32d4b1e:v6.0.1; sol40:master:b156a3f:v6.0.1-1-gb156a3f; (5) sol40-sc:master:db57d78:v6.0.0-5-gdb57d78;
| 1 | Timestamp of compilation |
| 2 | Username and host used for compilation |
| 3 | Quartus version used for compilation |
| 4 | Git upstream repository information |
| 5 | Git submodule repository information (for each submodule) |
Each line containing git repository information is itself a colon-separated list of fields, in order:
-
Submodule name (empty for upstream repository)
-
Branch name used for compilation
-
SHA of commit used for compilation
-
Tag name (or the output of
git-describeif no tag matches the commit exactly)
Appendix K: Testing JCOP components
The fwTap JCOP component can be used to produce test coverage reports from CTRL scripts, for consumption by a test harness. The output format follows TAP specification the that is supported by tools and libraries in many languages (TAP stands for "Test Anything Protocol"). This implementation is mostly a straight C-to-CTRL conversion of the libtap project hosted at http://www.shlomifish.org/open-source/projects/libtap/.
This component provides a library that other JCOP components can use to run unit tests and report their success, or failure. The next section describes the library API, while the last section of this appendix shows several use-case examples. ¶
fwTap library cross reference
diag
Print a diagnostic message (use instead of printf/fprintf).
diag ensures that the output will not be considered to be a test
result by the TAP test harness. It will append '\n' for you.
¶
unsigned diag(string fmt, ...);- fmt (I)
-
the format of the printf-style message ¶
except
Note that a test raised an exception
In addition to pass and fail , a test can fail by throwing an exception, this can be reported by catching the exception passing it as a first argument to this function.
¶
void except(dyn_errClass exc, string fmt, ...);exit_status
The value that main should return.
For maximum compatability your test program should return a particular exit code (ie. 0 if all tests were run, and every test which was expected to succeed succeeded). ¶
int exit_status();fail
Note that a test failed
For complicated code paths, it can be easiest to simply call pass in one
branch and fail in another.
¶
void fail(string fmt, ...);- fmt (I)
-
the printf-style name of the test ¶
ok
Conditional test with a name.
If the expression is true, the test passes. The name of the test will be the filename, line number, and the printf-style string. This can be clearer than simply the expression itself. ¶
bool ok(bool e, string fmt, ...);pass
Note that a test passed.
For complicated code paths, it can be easiest to simply call pass in one
branch and fail in another.
¶
void pass(string fmt, ...);- fmt (I)
-
the printf-style name of the test ¶
plan_no_plan
I have no idea how many tests I’m going to run.
In some situations you may not know how many tests you will be running, or
you are developing your test program, and do not want to update the
plan_tests call every time you make a change. For those situations use
plan_no_plan instead of plan_tests . It indicates to the test harness
that an indeterminate number of tests will be run.
Remember, if you fail to plan, you plan to fail. ¶
int plan_no_plan(string name);- name (I)
-
name for this test plan ¶
plan_skip_all
Indicate that you will skip all tests.
If your test program detects at run time that some required functionality
is missing (for example, it relies on a database connection which is not
present, or a particular configuration option that has not been included
in the running kernel) use plan_skip_all instead of plan_tests .
¶
int plan_skip_all(string name, string reason);plan_tests
Announce the number of tests you plan to run.
This should be the first call in your test program: it allows tracing ¶
int plan_tests(string name, unsigned tests);skip
Print a diagnostic message (use instead of printf/fprintf).
Sometimes tests cannot be run because the test system lacks some feature: you should explicitly document that you’re skipping tests using skip().
From the Test::More documentation:
If it’s something the user might not be able to do, use SKIP. This includes optional modules that aren’t installed, running under an OS that doesn’t have some feature (like fork() or symlinks), or maybe you need an Internet connection and one isn’t available. ¶
int skip(unsigned n, string fmt, ...);todo_start
Mark tests that you expect to fail.
It’s extremely useful to write tests before you implement the matching fix
or features: surround these tests by todo_start / todo_end . These tests
will still be run, but with additional output that indicates that they are
expected to fail.
This way, should a test start to succeed unexpectedly, tools like prove(1) will indicate this and you can move the test out of the todo block. This is much more useful than simply commenting out (or '#if 0') the tests.
From the Test::More documentation:
If it’s something the programmer hasn’t done yet, use TODO. This is for any code you haven’t written yet, or bugs you have yet to fix, but want to put tests in your testing script (always a good idea). ¶
void todo_start(string fmt, ...);- fmt (I)
-
the reason they currently fail ¶
fwTap library examples
diagdiag("Now running complex tests");
try {
x = do_something();
if (!checkable(x) || check_value(x))
pass("do_something() returned a valid value");
else
fail("do_something() returned an invalid value");
} catch {
except(getLastException(), "do_something() raised an exception");
}
ok(init_subsystem() == 1); ok(init_subsystem() == 0, "Second initialization should fail");
x = do_something();
if (!checkable(x) || check_value(x))
pass("do_something() returned a valid value");
else
fail("do_something() returned an invalid value");
¶
.Using plan_no_plan
plan_no_plan("myComponent");
while (random() % 2)
ok(some_test());
exit(exit_status());
¶
.Using plan_skip_all
if (!have_some_feature) {
plan_skip_all("myComponent", "Need some_feature support");
exit(exit_status());
}
plan_tests("myComponent", 13);
¶
.Using plan_tests
plan_tests("mycomponent", 13);
if(HAVE_SOME_FEATURE) ok(test_some_feature()); else skip(1, "Don't have SOME_FEATURE");
¶
.Using todo_start
todo_start("dwim() not returning true yet");
ok(dwim(), "Did what the user wanted");
todo_end();