Petalinux is a modified version of the [Yocto Project](https://www.yoctoproject.org/). Petalinux was created and is maintained by Xilinx for their hardware platforms. [Here](https://support.xilinx.com/s/article/968413?language=en_US) is a good article giving a rough overview of Petalinux.
# Overview
This project uses Petalinux primarily because it comes with USB/Serial/Ethernet drivers so we don't have to develop and maintain them ourselves. Not only that, but having Petalinux run on the CyDAQ makes it extremely configurable in the future.
Petalinux is a modified version of the [Yocto Project](https://www.yoctoproject.org/). Petalinux was created and is maintained by Xilinx for their hardware platforms. [Here](https://support.xilinx.com/s/article/968413?language=en_US) is a good article giving a rough overview of Petalinux.
All the necessary files are located in the [petalinux](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux) directory at the root of the project. That directory then splits into three subdirectories that operate as follows:
This project uses Petalinux primarily because it comes with USB/Serial/Ethernet drivers so we don't have to develop and maintain them ourselves. Not only that, but having Petalinux run on the CyDAQ makes it extremely configurable in the future.
-[hw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/hw) - The hardware project created in Vivado, which exports a `.xsa` file and `.bit` file that is used by petalinux tools and Vitis.
-[os](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/os) - The petalinux OS project. All petalinux tools commands are ran in this directory.
All the necessary files are located in the [petalinux](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux) directory at the root of the project. That directory then splits into three subdirectories that operate as follows:
-[sw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw) - Any applications we develop to run on petalinux go here, and are designed to be opened in Vitis.
-[hw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/hw) - The hardware project created in Vivado, which exports a `.xsa` file and `.bit` file that is used by petalinux tools and Vitis.
## Diagram
-[os](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/os) - The petalinux OS project. All petalinux tools commands are ran in this directory.
The diagram gives a top level view of the entire firmware. It is split into two main parts, which represent a single core running their respective application (Petalinux and baremetal). The two CPUs share multiple memory regions to send messages and sample data, and samples are then written onto the filesystem in Peatlinux. Petalinux communicates with the host PC over USB via an emulated serial and ethernet connection (ACM and ECM Gadgets).
-[sw](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw) - Any applications we develop to run on petalinux go here, and are designed to be opened in Vitis.
Each aspect of this diagram is explained further in this wiki.
## Diagram
The diagram gives a top level view of the entire firmware. It is split into two main parts, which represent a single core running their respective application (Petalinux and baremetal). The two CPUs share multiple memory regions to send messages and sample data, and samples are then written onto the filesystem in Peatlinux. Petalinux communicates with the host PC over USB via an emulated serial and ethernet connection (ACM and ECM Gadgets).
It's highly recommended to use a Linux-based desktop development environment when configuring/building petalinux, as that is what we found had the least amount of issues. Petalinux tools also only runs on Linux machines, and it's clear that Xilinx prefers a Linux development environment for their software tools.
# Development environment
Check out our [Petalinux Ubuntu Development Environment](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Ubuntu-Development-Environment) guide for specific steps to set that up!
It's highly recommended to use a Linux-based desktop development environment when configuring/building petalinux, as that is what we found had the least amount of issues. Petalinux tools also only runs on Linux machines, and it's clear that Xilinx prefers a Linux development environment for their software tools.
# Hardware - Vivado
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vivado on your Ubuntu development environment.
Check out our [Petalinux Ubuntu Development Environment](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Ubuntu-Development-Environment) guide for specific steps to set that up!
Check out our [Petalinux Hardware Design](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Hardware-Design) page for all the documentation and setup process.
# Hardware - Vivado
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vivado on your Ubuntu development environment.
# OS - Petalinux
Check out our [Petalinux Hardware Design](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Hardware-Design) page for all the documentation and setup process.
Building petalinux requires petalinux tools, which can only run on Linux. Check out our [Petalinux Tools Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Tools-Install) for a guide on how to set that up in the Ubuntu development environment.
# OS - Petalinux
The main purpose of Petalinux is to allow for fast and stable communication between the CyDAQ and host pc over USB.
Building petalinux requires petalinux tools, which can only run on Linux. Check out our [Petalinux Tools Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Petalinux-Tools-Install) for a guide on how to set that up in the Ubuntu development environment.
## USB kernel modules
We are using two USB gadgets to allow communication over USB.
The main purpose of Petalinux is to allow for fast and stable communication between the CyDAQ and host pc over USB.
- Abstract Control Model (ACM) USB Gadget: Simulates a serial port on the host PC side.
- Ethernet Control Model (ECM) USB Gadget: Simulates an ethernet connection on the host PC side. Each CyDAQ is configured to have the static ip `169.254.7.2`. In other words, when the CyDAQ is plugged into a PC, you can ping, ssh, etc to that ip. Read about [How to use automatic TCP/IP addressing without a DHCP server](https://learn.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-automatic-tcpip-addressing-without-a-dh) for Windows.
## USB kernel modules
We are using two USB gadgets to allow communication over USB.
Each USB gadget shows up in windows as a separate device. The ACM gadget is a COM port and the ECM as a network interface. Both should auto configure, so the CyDAQ should be plug and play!
- Abstract Control Model (ACM) USB Gadget: Simulates a serial port on the host PC side.
- Ethernet Control Model (ECM) USB Gadget: Simulates an ethernet connection on the host PC side. Each CyDAQ is configured to have the static ip `169.254.7.2`. In other words, when the CyDAQ is plugged into a PC, you can ping, ssh, etc to that ip. Read about [How to use automatic TCP/IP addressing without a DHCP server](https://learn.microsoft.com/en-us/windows-server/troubleshoot/how-to-use-automatic-tcpip-addressing-without-a-dh) for Windows.
On the petalinux side, the serial gadget shows up as a readable/writable file in `/dev/ttyGS0` and the ethernet gadget as a network interface named `usb0`
Each USB gadget shows up in windows as a separate device. The ACM gadget is a COM port and the ECM as a network interface. Both should auto configure, so the CyDAQ should be plug and play!
The configuration of the two Gadgets is done using ConfigFS. The script that runs on petalinux startup that does the filesystem-based configuration is located [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
On the petalinux side, the serial gadget shows up as a readable/writable file in `/dev/ttyGS0` and the ethernet gadget as a network interface named `usb0`
The correct kernel modules must also be enabled for them to work properly. This can be done with the following (they are already be enabled in this repo).
The configuration of the two Gadgets is done using ConfigFS. The script that runs on petalinux startup that does the filesystem-based configuration is located [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
```bash
petalinux-config -c kernel
The correct kernel modules must also be enabled for them to work properly. This can be done with the following (they are already be enabled in this repo).
```
A configuration screen should open when the command completes.
```bash
Make sure the following are selected under `Device Drivers/USB Support/USB Gadget Support`
petalinux-config -c kernel
*`USB Peripheral Controller/<*> Xilinx USB Driver`
```
*`<M> USB Gadget functions configurable through configfs`
A configuration screen should open when the command completes.
*`[*] Abstract Control Model (CDC ACM)`
Make sure the following are selected under `Device Drivers/USB Support/USB Gadget Support`
*`[*] Ethernet Control Model (CDC ECM)`
*`USB Peripheral Controller/<*> Xilinx USB Driver`
*`[*] RNDIS`
*`<M> USB Gadget functions configurable through configfs`
*`[*] Abstract Control Model (CDC ACM)`
## Device Tree
*`[*] Ethernet Control Model (CDC ECM)`
The device tree is used to define hardware (including memory allocation) in Petalinux. The main Device tree files are generated during the `petalinux-config --get-hw-description` command. We can modify the device tree by the device tree addon file which is located at [petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi), and any changes to it require a rebuild of petalinux to take effect.
*`[*] RNDIS`
### Reserved Memory
## Device Tree
The reserved-memory node outlines parts of memory that we set aside so Petalinux doesn't use. They contain the following entries:
The device tree is used to define hardware (including memory allocation) in Petalinux. The main Device tree files are generated during the `petalinux-config --get-hw-description` command. We can modify the device tree by the device tree addon file which is located at [petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-bsp/device-tree/files/system-user.dtsi), and any changes to it require a rebuild of petalinux to take effect.
- vdev0vring0, vdev0vring1, vdev0buffer - Required for RPMsg protocol
- rproc_0_reserved - Required for remoteproc to launch the baremetal app
### Reserved Memory
- shared_memory - The baremetal app writes sample data to this memory region, and Petalinux reads from it when obtaining sample data
The reserved-memory node outlines parts of memory that we set aside so Petalinux doesn't use. They contain the following entries:
- vdev0vring0, vdev0vring1, vdev0buffer - Required for RPMsg protocol
## Baremetal App
- rproc_0_reserved - Required for remoteproc to launch the baremetal app
One of the downsides of using Petalinux is that it adds a lot of overhead to certain hardware operations. The best way to get around this issue is to run an application directly on one of the two CPU cores. This is referred to as a baremetal app.
- shared_memory - The baremetal app writes sample data to this memory region, and Petalinux reads from it when obtaining sample data
The baremetal app for this project is [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw/platform/CyDAQ_sampling). It is launched from Petalinux using the [remoteproc](https://www.kernel.org/doc/html/latest/staging/remoteproc.html) framework. When this occurs, both CPUs are then running separately, and need to use shared memory to communicate with each other. We use a library called [openAMP](https://www.openampproject.org/) and [rpmsg](https://www.kernel.org/doc/html/latest/staging/rpmsg.html) to do this. There is also an additional memory region carved out specifically for storing sample data, which the baremetal writes to whenever the ADC triggers an interrupt. Once sampling is stopped, Petalinux reads from that same shared memory region and sends the sample data back to the host PC.
## Baremetal App
One of the downsides of using Petalinux is that it adds a lot of overhead to certain hardware operations. The best way to get around this issue is to run an application directly on one of the two CPU cores. This is referred to as a baremetal app.
# SW - Vitis
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vitis on your Ubuntu development environment.
The baremetal app for this project is [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/tree/master/petalinux/sw/platform/CyDAQ_sampling). It is launched from Petalinux using the [remoteproc](https://www.kernel.org/doc/html/latest/staging/remoteproc.html) framework. When this occurs, both CPUs are then running separately, and need to use shared memory to communicate with each other. We use a library called [openAMP](https://www.openampproject.org/) and [rpmsg](https://www.kernel.org/doc/html/latest/staging/rpmsg.html) to do this. There is also an additional memory region carved out specifically for storing sample data, which the baremetal writes to whenever the ADC triggers an interrupt. Once sampling is stopped, Petalinux reads from that same shared memory region and sends the sample data back to the host PC.
The software directory `<project root>/petalinux/sw/platform` contains the multiple software applications and the platforms associated with them.
# SW - Vitis
Check out our [Vivado/Vitis Install](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Vivado-and-Vitis-Install) guide to learn how to install Vitis on your Ubuntu development environment.
- CyDAQ_platform: A linux platform project that points to the necessary petalinux SDK files generated by petalinux tools.
The software directory `<project root>/petalinux/sw/platform` contains the multiple software applications and the platforms associated with them.
- CyDAQ_comm / CyDAQ_comm_system: The application that runs on petalinux which handles 1) communication with the host PC over serial and 2) communication to the sampling application running as standalone on the other CPU core.
- CyDAQ_platform: A linux platform project that points to the necessary petalinux SDK files generated by petalinux tools.
- CyDAQ_standalone: A baremetal platform project that is designed to run on the `ps7_cortexa9_1` CPU core.
- CyDAQ_comm / CyDAQ_comm_system: The application that runs on petalinux which handles 1) communication with the host PC over serial and 2) communication to the sampling application running as standalone on the other CPU core.
- CyDAQ_sampling / CyDAQ_sampling_system: The baremetal application that runs on the standalone platform. It listens for configuration requests from petalinux, configures the hardware (input select, sample rate, etc) accordingly, and communicates any information back to petalinux.
- CyDAQ_standalone: A baremetal platform project that is designed to run on the `ps7_cortexa9_1` CPU core.
## Asymmetric Multi Processing (AMP)
- CyDAQ_sampling / CyDAQ_sampling_system: The baremetal application that runs on the standalone platform. It listens for configuration requests from petalinux, configures the hardware (input select, sample rate, etc) accordingly, and communicates any information back to petalinux.
We are using the OpenAMP library to communicate between Petalinux and baremetal applications. The [Libmetal and OpenAMP User Guide](https://docs.xilinx.com/v/u/2020.1-English/ug1186-zynq-openamp-gsg) is the official documentation for petalinux.
> Warning: The documentation linked above isn't the best. It is mainly written for the ZyboMP platform, which is slightly different then the Zybo Z7 board we are using.
## Asymmetric Multi Processing (AMP)
OpenAMP is compatable with linux remoteproc, rpmsg, and VirtIO components, which we use to send commands and data from petalinux -> baremetal and baremetal -> petalinux.
We are using the OpenAMP library to communicate between Petalinux and baremetal applications. The [Libmetal and OpenAMP User Guide](https://docs.xilinx.com/v/u/2020.1-English/ug1186-zynq-openamp-gsg) is the official documentation for petalinux.
> Warning: The documentation linked above isn't the best. It is mainly written for the ZyboMP platform, which is slightly different then the Zybo Z7 board we are using.
The necessary functions for the comm and sampling applications are located in ```rpc.c``` and ```rpc.h``` of each app.
OpenAMP is compatable with linux remoteproc, rpmsg, and VirtIO components, which we use to send commands and data from petalinux -> baremetal and baremetal -> petalinux.
# Bootscript
When Petalinux boots, we need it to automatically configure the USB gadgets, give itself a static IP, enable certain hardware features, and launch the comm app (which itself launches the sampling app on the other CPU core). This is all done in a bash script that runs on boot. Keep in mind that the last few commands in the script (mainly ```./CyDAQ_comm.elf &```) might be commented out. This is because the development process is a lot easier when the app doesn't auto start. Regardless, those lines must not be commented out when building petalinux to distribute for labs!
The necessary functions for the comm and sampling applications are located in ```rpc.c``` and ```rpc.h``` of each app.
The script can be found [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
# Bootscript
When Petalinux boots, we need it to automatically configure the USB gadgets, give itself a static IP, enable certain hardware features, and launch the comm app (which itself launches the sampling app on the other CPU core). This is all done in a bash script that runs on boot. Keep in mind that the last few commands in the script (mainly ```./CyDAQ_comm.elf &```) might be commented out. This is because the development process is a lot easier when the app doesn't auto start. Regardless, those lines must not be commented out when building petalinux to distribute for labs!
The bootscript requires CyDAQ_comm.elf and CyDAQ_sampling.elf to be in the same directory as it for petalinux to build. These are generated by Vitis. If you are having trouble building Petalinux because these files are missing, you can simply create empty files in their place:
```bash
The script can be found [here](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/files/bootscript).
The bootscript requires CyDAQ_comm.elf and CyDAQ_sampling.elf to be in the same directory as it for petalinux to build. These are generated by Vitis. If you are having trouble building Petalinux because these files are missing, you can simply create empty files in their place:
> Note: A version of petalinux built with these empty files will not auto start the comm and sampling applications, because they don't exist! It's only useful for getting a working version of Petalinux so you can build the SDK for Vitis.
More information about building (and copying) those .elf files are in the [Petalinux Repo Setup](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Firmware-Repo-Setup) docs.
```
> Note: A version of petalinux built with these empty files will not auto start the comm and sampling applications, because they don't exist! It's only useful for getting a working version of Petalinux so you can build the SDK for Vitis.
The bootscript was created with the following Petalinux tools commands:
More information about building (and copying) those .elf files are in the [Petalinux Repo Setup](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Firmware-Repo-Setup) docs.
The [bootscript.bb](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/bootscript.bb) file is also worth taking note of, as it defines where the bootscript and .elf files get placed on the Petalinux filesystem when it's built.
```bash
petalinux-build -c kernel
# External Resources
```
Check out our [Useful Resources](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Useful-Resources) page for links to official documentation and other guides we used to develop this project.
The [bootscript.bb](https://git.ece.iastate.edu/sd/sdmay23-47/-/blob/master/petalinux/os/project-spec/meta-user/recipes-apps/bootscript/bootscript.bb) file is also worth taking note of, as it defines where the bootscript and .elf files get placed on the Petalinux filesystem when it's built.
\ No newline at end of file
# External Resources
Check out our [Useful Resources](https://git.ece.iastate.edu/sd/sdmay23-47/-/wikis/Useful-Resources) page for links to official documentation and other guides we used to develop this project.
