Petalinux is a modified version of the [Yocto Project](https://www.yoctoproject.org/). Petalinux was created and is maintained by Xilinx for their hardware platforms. [Here](https://support.xilinx.com/s/article/968413?language=en_US) is a good article giving a rough overview of Petalinux.
This project uses Petalinux primarily because it comes with USB/Serial/Ethernet drivers so we don't have to develop and maintain them ourselves. Not only that, but having Petalinux run on the CyDAQ makes it extremely configurable in the future.
All the necessary files are located in the [petalinux](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux) directory at the root of the project. That directory then splits into three subdirectories that operate as follows:
-[hw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/hw) - The hardware project created in Vivado, which exports a `.xsa` file and `.bit` file that is used by petalinux tools and Vitis.
-[os](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/os) - The petalinux OS project. All petalinux tools commands are ran in this directory.
-[sw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw) - Any applications we develop to run on petalinux go here, and are designed to be opened in Vitis.
## Diagram
The diagram gives a top level view of the entire firmware. It is split into two main parts, which represent a single core running their respective application (Petalinux and baremetal). The two CPUs share multiple memory regions to send messages and sample data, and samples are then written onto the filesystem in Peatlinux. Petalinux communicates with the host PC over USB via an emulated serial and ethernet connection (ACM and ECM Gadgets).
Each aspect of this diagram is explained further in this wiki.
It's highly recommended to use a Linux-based desktop development environment when configuring/building petalinux, as that is what we found had the least amount of issues. Petalinux tools also only runs on Linux machines, and it's clear that Xilinx prefers a Linux development environment for their software tools.
Check out our [Petalinux Ubuntu Development Environment](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Ubuntu-Development-Environment) guide for specific steps to set that up!
# Hardware - Vivado
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vivado on your Ubuntu development environment.
Check out our [Petalinux Hardware Design](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Hardware-Design) page for all the documentation and setup process.
# OS - Petalinux
Building petalinux requires petalinux tools, which can only run on Linux. Check out our [Petalinux Tools Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Tools-Install) for a guide on how to set that up in the Ubuntu development environment.
The main purpose of Petalinux is to allow for fast and stable communication between the CyDAQ and host pc over USB.
## USB kernel modules
We are using two USB gadgets to allow communication over USB.
- Abstract Control Model (ACM) USB Gadget: Simulates a serial port on the host PC side.
- Ethernet Control Model (ECM) USB Gadget: Simulates an ethernet connection on the host PC side. Each CyDAQ is configured to have the static ip `169.254.7.2`. In other words, when the CyDAQ is plugged into a PC, you can ping, ssh, etc to that ip. Read about [How to use automatic TCP/IP addressing without a DHCP server](https://learn.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-automatic-tcpip-addressing-without-a-dh) for Windows.
Each USB gadget shows up in windows as a separate device. The ACM gadget is a COM port and the ECM as a network interface. Both should auto configure, so the CyDAQ should be plug and play!
On the petalinux side, the serial gadget shows up as a readable/writable file in `/dev/ttyGS0` and the ethernet gadget as a network interface named `usb0`
The configuration of the two Gadgets is done using ConfigFS. The script that runs on petalinux startup that does the filesystem-based configuration is located [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
The correct kernel modules must also be enabled for them to work properly. This can be done with the following (they are already be enabled in this repo).
```bash
petalinux-config -c kernel
```
A configuration screen should open when the command completes.
Make sure the following are selected under `Device Drivers/USB Support/USB Gadget Support`
*`USB Peripheral Controller/<*> Xilinx USB Driver`
*`<M> USB Gadget functions configurable through configfs`
*`[*] Abstract Control Model (CDC ACM)`
*`[*] Ethernet Control Model (CDC ECM)`
*`[*] RNDIS`
## Device Tree
The device tree is used to define hardware (including memory allocation) in Petalinux. The main Device tree files are generated during the `petalinux-config --get-hw-description` command. We can modify the device tree by the device tree addon file which is located at [petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi), and any changes to it require a rebuild of petalinux to take effect.
### Reserved Memory
The reserved-memory node outlines parts of memory that we set aside so Petalinux doesn't use. They contain the following entries:
- vdev0vring0, vdev0vring1, vdev0buffer - Required for RPMsg protocol
- rproc_0_reserved - Required for remoteproc to launch the baremetal app
- shared_memory - The baremetal app writes sample data to this memory region, and Petalinux reads from it when obtaining sample data
## Baremetal App
One of the downsides of using Petalinux is that it adds a lot of overhead to certain hardware operations. The best way to get around this issue is to run an application directly on one of the two CPU cores. This is referred to as a baremetal app.
The baremetal app for this project is [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw/platform/CyDAQ_sampling). It is launched from Petalinux using the [remoteproc](https://www.kernel.org/doc/html/latest/staging/remoteproc.html) framework. When this occurs, both CPUs are then running separately, and need to use shared memory to communicate with each other. We use a library called [openAMP](https://www.openampproject.org/) and [rpmsg](https://www.kernel.org/doc/html/latest/staging/rpmsg.html) to do this. There is also an additional memory region carved out specifically for storing sample data, which the baremetal writes to whenever the ADC triggers an interrupt. Once sampling is stopped, Petalinux reads from that same shared memory region and sends the sample data back to the host PC.
# SW - Vitis
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vitis on your Ubuntu development environment.
The software directory `<project root>/petalinux/sw/platform` contains the multiple software applications and the platforms associated with them.
- CyDAQ_platform: A linux platform project that points to the necessary petalinux SDK files generated by petalinux tools.
- CyDAQ_comm / CyDAQ_comm_system: The application that runs on petalinux which handles 1) communication with the host PC over serial and 2) communication to the sampling application running as standalone on the other CPU core.
- CyDAQ_standalone: A baremetal platform project that is designed to run on the `ps7_cortexa9_1` CPU core.
- CyDAQ_sampling / CyDAQ_sampling_system: The baremetal application that runs on the standalone platform. It listens for configuration requests from petalinux, configures the hardware (input select, sample rate, etc) accordingly, and communicates any information back to petalinux.
## Asymmetric Multi Processing (AMP)
We are using the OpenAMP library to communicate between Petalinux and baremetal applications. The [Libmetal and OpenAMP User Guide](https://docs.xilinx.com/v/u/2020.1-English/ug1186-zynq-openamp-gsg) is the official documentation for petalinux.
> Warning: The documentation linked above isn't the best. It is mainly written for the ZyboMP platform, which is slightly different then the Zybo Z7 board we are using.
OpenAMP is compatable with linux remoteproc, rpmsg, and VirtIO components, which we use to send commands and data from petalinux -> baremetal and baremetal -> petalinux.
The necessary functions for the comm and sampling applications are located in ```rpc.c``` and ```rpc.h``` of each app.
# Bootscript
When Petalinux boots, we need it to automatically configure the USB gadgets, give itself a static IP, enable certain hardware features, and launch the comm app (which itself launches the sampling app on the other CPU core). This is all done in a bash script that runs on boot. Keep in mind that the last few commands in the script (mainly ```./CyDAQ_comm.elf &```) might be commented out. This is because the development process is a lot easier when the app doesn't auto start. Regardless, those lines must not be commented out when building petalinux to distribute for labs!
The script can be found [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
The bootscript requires CyDAQ_comm.elf and CyDAQ_sampling.elf to be in the same directory as it for petalinux to build. These are generated by Vitis. If you are having trouble building Petalinux because these files are missing, you can simply create empty files in their place:
> Note: A version of petalinux built with these empty files will not auto start the comm and sampling applications, because they don't exist! It's only useful for getting a working version of Petalinux so you can build the SDK for Vitis.
More information about building (and copying) those .elf files are in the [Petalinux Repo Setup](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Firmware-Repo-Setup) docs.
The bootscript was created with the following Petalinux tools commands:
The [bootscript.bb](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/bootscript.bb) file is also worth taking note of, as it defines where the bootscript and .elf files get placed on the Petalinux filesystem when it's built.
# External Resources
Check out our [Useful Resources](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Useful-Resources) page for links to official documentation and other guides we used to develop this project.
Petalinux is a modified version of the [Yocto Project](https://www.yoctoproject.org/). Petalinux was created and is maintained by Xilinx for their hardware platforms. [Here](https://support.xilinx.com/s/article/968413?language=en_US) is a good article giving a rough overview of Petalinux.
This project uses Petalinux primarily because it comes with USB/Serial/Ethernet drivers so we don't have to develop and maintain them ourselves. Not only that, but having Petalinux run on the CyDAQ makes it extremely configurable in the future.
All the necessary files are located in the [petalinux](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux) directory at the root of the project. That directory then splits into three subdirectories that operate as follows:
-[hw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/hw) - The hardware project created in Vivado, which exports a `.xsa` file and `.bit` file that is used by petalinux tools and Vitis.
-[os](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/os) - The petalinux OS project. All petalinux tools commands are ran in this directory.
-[sw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw) - Any applications we develop to run on petalinux go here, and are designed to be opened in Vitis.
## Diagram
The diagram gives a top level view of the entire firmware. It is split into two main parts, which represent a single core running their respective application (Petalinux and baremetal). The two CPUs share multiple memory regions to send messages and sample data, and samples are then written onto the filesystem in Peatlinux. Petalinux communicates with the host PC over USB via an emulated serial and ethernet connection (ACM and ECM Gadgets).
Each aspect of this diagram is explained further in this wiki.
It's highly recommended to use a Linux-based desktop development environment when configuring/building petalinux, as that is what we found had the least amount of issues. Petalinux tools also only runs on Linux machines, and it's clear that Xilinx prefers a Linux development environment for their software tools.
Check out our [Petalinux Ubuntu Development Environment](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Ubuntu-Development-Environment) guide for specific steps to set that up!
# Hardware - Vivado
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vivado on your Ubuntu development environment.
Check out our [Petalinux Hardware Design](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Hardware-Design) page for all the documentation and setup process.
# OS - Petalinux
Building petalinux requires petalinux tools, which can only run on Linux. Check out our [Petalinux Tools Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Tools-Install) for a guide on how to set that up in the Ubuntu development environment.
The main purpose of Petalinux is to allow for fast and stable communication between the CyDAQ and host pc over USB.
## USB kernel modules
We are using two USB gadgets to allow communication over USB.
- Abstract Control Model (ACM) USB Gadget: Simulates a serial port on the host PC side.
- Ethernet Control Model (ECM) USB Gadget: Simulates an ethernet connection on the host PC side. Each CyDAQ is configured to have the static ip `169.254.7.2`. In other words, when the CyDAQ is plugged into a PC, you can ping, ssh, etc to that ip. Read about [How to use automatic TCP/IP addressing without a DHCP server](https://learn.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-automatic-tcpip-addressing-without-a-dh) for Windows.
Each USB gadget shows up in windows as a separate device. The ACM gadget is a COM port and the ECM as a network interface. Both should auto configure, so the CyDAQ should be plug and play!
On the petalinux side, the serial gadget shows up as a readable/writable file in `/dev/ttyGS0` and the ethernet gadget as a network interface named `usb0`
The configuration of the two Gadgets is done using ConfigFS. The script that runs on petalinux startup that does the filesystem-based configuration is located [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
The correct kernel modules must also be enabled for them to work properly. This can be done with the following (they are already be enabled in this repo).
```bash
petalinux-config -c kernel
```
A configuration screen should open when the command completes.
Make sure the following are selected under `Device Drivers/USB Support/USB Gadget Support`
*`USB Peripheral Controller/<*> Xilinx USB Driver`
*`<M> USB Gadget functions configurable through configfs`
*`[*] Abstract Control Model (CDC ACM)`
*`[*] Ethernet Control Model (CDC ECM)`
*`[*] RNDIS`
## Device Tree
The device tree is used to define hardware (including memory allocation) in Petalinux. The main Device tree files are generated during the `petalinux-config --get-hw-description` command. We can modify the device tree by the device tree addon file which is located at [petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi), and any changes to it require a rebuild of petalinux to take effect.
### Reserved Memory
The reserved-memory node outlines parts of memory that we set aside so Petalinux doesn't use. They contain the following entries:
- vdev0vring0, vdev0vring1, vdev0buffer - Required for RPMsg protocol
- rproc_0_reserved - Required for remoteproc to launch the baremetal app
- shared_memory - The baremetal app writes sample data to this memory region, and Petalinux reads from it when obtaining sample data
## Baremetal App
One of the downsides of using Petalinux is that it adds a lot of overhead to certain hardware operations. The best way to get around this issue is to run an application directly on one of the two CPU cores. This is referred to as a baremetal app.
The baremetal app for this project is [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw/platform/CyDAQ_sampling). It is launched from Petalinux using the [remoteproc](https://www.kernel.org/doc/html/latest/staging/remoteproc.html) framework. When this occurs, both CPUs are then running separately, and need to use shared memory to communicate with each other. We use a library called [openAMP](https://www.openampproject.org/) and [rpmsg](https://www.kernel.org/doc/html/latest/staging/rpmsg.html) to do this. There is also an additional memory region carved out specifically for storing sample data, which the baremetal writes to whenever the ADC triggers an interrupt. Once sampling is stopped, Petalinux reads from that same shared memory region and sends the sample data back to the host PC.
# SW - Vitis
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vitis on your Ubuntu development environment.
The software directory `<project root>/petalinux/sw/platform` contains the multiple software applications and the platforms associated with them.
- CyDAQ_platform: A linux platform project that points to the necessary petalinux SDK files generated by petalinux tools.
- CyDAQ_comm / CyDAQ_comm_system: The application that runs on petalinux which handles 1) communication with the host PC over serial and 2) communication to the sampling application running as standalone on the other CPU core.
- CyDAQ_standalone: A baremetal platform project that is designed to run on the `ps7_cortexa9_1` CPU core.
- CyDAQ_sampling / CyDAQ_sampling_system: The baremetal application that runs on the standalone platform. It listens for configuration requests from petalinux, configures the hardware (input select, sample rate, etc) accordingly, and communicates any information back to petalinux.
## Asymmetric Multi Processing (AMP)
We are using the OpenAMP library to communicate between Petalinux and baremetal applications. The [Libmetal and OpenAMP User Guide](https://docs.xilinx.com/v/u/2020.1-English/ug1186-zynq-openamp-gsg) is the official documentation for petalinux.
> Warning: The documentation linked above isn't the best. It is mainly written for the ZyboMP platform, which is slightly different then the Zybo Z7 board we are using.
OpenAMP is compatable with linux remoteproc, rpmsg, and VirtIO components, which we use to send commands and data from petalinux -> baremetal and baremetal -> petalinux.
The necessary functions for the comm and sampling applications are located in ```rpc.c``` and ```rpc.h``` of each app.
# Bootscript
When Petalinux boots, we need it to automatically configure the USB gadgets, give itself a static IP, enable certain hardware features, and launch the comm app (which itself launches the sampling app on the other CPU core). This is all done in a bash script that runs on boot. Keep in mind that the last few commands in the script (mainly ```./CyDAQ_comm.elf &```) might be commented out. This is because the development process is a lot easier when the app doesn't auto start. Regardless, those lines must not be commented out when building petalinux to distribute for labs!
The script can be found [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
The bootscript requires CyDAQ_comm.elf and CyDAQ_sampling.elf to be in the same directory as it for petalinux to build. These are generated by Vitis. If you are having trouble building Petalinux because these files are missing, you can simply create empty files in their place:
> Note: A version of petalinux built with these empty files will not auto start the comm and sampling applications, because they don't exist! It's only useful for getting a working version of Petalinux so you can build the SDK for Vitis.
More information about building (and copying) those .elf files are in the [Petalinux Repo Setup](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Firmware-Repo-Setup) docs.
The bootscript was created with the following Petalinux tools commands:
The [bootscript.bb](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/bootscript.bb) file is also worth taking note of, as it defines where the bootscript and .elf files get placed on the Petalinux filesystem when it's built.
# External Resources
Check out our [Useful Resources](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Useful-Resources) page for links to official documentation and other guides we used to develop this project.
