Virtual Camera
|
|
The Matab Virtual Camera topology is shown below. There are many components associated with the topology and each component has its own code base. This landing page will provide you with everything you need to get up and running with any Xilinx Zynq SoC board in no time.
Below are the links to the Mathworks File Exchange code bases that are required for the Virtual Camera. Depending on your specific use case you may not need all of the code bases. A brief description of each code base is given below and detailed tutorial videos we be available soon.
Bare Bones Virtual Camera Capability
Matlab Virtual Camera
Enables Matlab to communicate via WiFi to the Matlab Zynq Server running on the Zynq SoC board. [Runs on host PC]
Matlab Zynq Server
Communicates with the Matlab Virtual Camera code base listed above. [Runs on Zynq SoC]
Virtual Image Generation
Mathworks Unity Link
Enables Matlab to talk to the Unity graphics engine. Unity can create the virtual frames for the virtual camera. This is not needed if one already has image frames to send across to the hardware board.
Mathworks Blender Link
Enables Matlab to talk to the Blender graphics engine. Blender can create the virtual frames for the virtual camera. This is not needed if one already has image frames to send across to the hardware board.
FPGA Processing
Update Device Tree
In order to access the FPGA frame buffers in linux, the device tree must be updated. This code base gives detailed instructions of how to perform this operation.
Virtual Camera Vivado Project
Vivado VDMA project that implements the topology above with the exception that the image processing core is just a feed through. Compiling this project will generate a system.bit file which must be transferred to the Zynq SoC. It is also necessary to download the User Space VDMA driver to access the FPGA image frames from linux.
User Space VDMA Driver
This driver allows one to access the FPGA image frames without the need to recompile the kernel or create any kernel driver. This 'driver' is in user space and accesses the FPGA buffers via the UIO framework.
Simulink HDL Generation and Targeting
Zynq Vivado Reference Design for Simulink
This reference design incorporates the Virtual Camera Vivado project from above however the Image Processing Core section is not a feed through but rather a cut out for where the Simulink generated HDL IP core should automatically be placed.
Sobel Filter for HDL Coder
This code base implements a Sobel filter in Simulink that is capable of being converted into HDL and run on any Zynq board via a Virtual Camera.
Frame-Based Sobel Filter for HDL Coder
This code base implements the same Sobel filter from above however it is created via a frame-based approach using some of the newest features of the Mathworks tools. This makes the design much easier as one does not have to worry about kernel buffering and control signal synchronization.
Quickest Way to Get up and Running
Board Setup
Matlab Virtual Camera
Enables Matlab to communicate via WiFi to the Matlab Zynq Server running on the Zynq SoC board. [Runs on host PC]
Matlab Zynq Server
Communicates with the Matlab Virtual Camera code base listed above. [Runs on Zynq SoC]
Virtual Image Generation
Mathworks Unity Link
Enables Matlab to talk to the Unity graphics engine. Unity can create the virtual frames for the virtual camera. This is not needed if one already has image frames to send across to the hardware board.
Mathworks Blender Link
Enables Matlab to talk to the Blender graphics engine. Blender can create the virtual frames for the virtual camera. This is not needed if one already has image frames to send across to the hardware board.
FPGA Processing
Update Device Tree
In order to access the FPGA frame buffers in linux, the device tree must be updated. This code base gives detailed instructions of how to perform this operation.
Virtual Camera Vivado Project
Vivado VDMA project that implements the topology above with the exception that the image processing core is just a feed through. Compiling this project will generate a system.bit file which must be transferred to the Zynq SoC. It is also necessary to download the User Space VDMA driver to access the FPGA image frames from linux.
User Space VDMA Driver
This driver allows one to access the FPGA image frames without the need to recompile the kernel or create any kernel driver. This 'driver' is in user space and accesses the FPGA buffers via the UIO framework.
Simulink HDL Generation and Targeting
Zynq Vivado Reference Design for Simulink
This reference design incorporates the Virtual Camera Vivado project from above however the Image Processing Core section is not a feed through but rather a cut out for where the Simulink generated HDL IP core should automatically be placed.
Sobel Filter for HDL Coder
This code base implements a Sobel filter in Simulink that is capable of being converted into HDL and run on any Zynq board via a Virtual Camera.
Frame-Based Sobel Filter for HDL Coder
This code base implements the same Sobel filter from above however it is created via a frame-based approach using some of the newest features of the Mathworks tools. This makes the design much easier as one does not have to worry about kernel buffering and control signal synchronization.
Quickest Way to Get up and Running
Board Setup
- You will need some type of filesystem running on your zynq board. I recommend using the PYNQ framework or a version of Ubuntu. This will provide you with utilities like apt-get to download software tools such as python, samba, etc. See the video below on installing Ubuntu on the Kria.
- Log onto your board. If you have a monitor and keyboard connected to your board, just power on and log in. If not, you can connect to your board via a serial terminal or SSH. I recommend using Tera-Term or MobaXterm to connect.
- Determine the IP address of the board by typing ifconfig. This IP address will be used in the next step.
- Download Matlab Virtual Camera and Matlab Zynq Server code bases. If you are using a zynq7000, copy that directory to your board. If you are using a zynq ultrascale, copy the kv260 directory to your board. As of now, the zynq7000 code uses python2 and the ultrascale code uses python3
- Open Matlab, navigate to the Matlab Virtual Camera code that was downloaded and to the proper type of board.
- Modify the IP address located in the helloWorld.m file to match that of your board.
- Execute the helloWorld.py on the board and helloWorld.m on the host PC. You should see a loopback performed. Congrats you now have your board connected to your host PC correctly.
- Run the matlabStereoServerProcessorOnly.py file on the zynq target.
- Run the virtualCameraProcessorOnly.m file in Matlab.
- You should see the sailboat images being transferred and when returned they should have an edge detection operation performed on them. Congratulations you have just done some remote image processing on the zynq target.
Video Tutorials
|
Links
Kria KV260 Landing Page Ubuntu Download Link FTDI Com Port Driver Download Commands in Tutorial sudo apt update sudo apt install samba sudo vi /etc/samba/smb.conf [kria] path = /home/ubuntu guest ok = yes browsable = yes create mask = 0775 directory mask = 0775 read only = no sudo smbpasswd -a ubuntu service smbd start Links
Commands in Tutorial sudo pip install opencv-python Links
Commands in Tutorial cd C:\Xilinx\Vitis\2024.1\bin xsct.bat cd c:/users/daniel/Downloads/test/kv260 createdts -hw kv260vdma.xsa -zocl -out . -platform-name mydevice -git-branch xlnx_rel_v2024.1 -overlay -compile sudo dtc -I dts -O dtb -o kv260vdma.dtbo kv260vdma.dtsi Links
|
Links
Links
Links
Links
|
Links
Links
Commands in Tutorial cat /proc/meminfo cat /proc/iomem dmesg | grep cma Links
Commands in Tutorial sudo dtc -I dts -O dtb -o simulink.dtbo simulink.dtsi hdlsetuptoolpath('ToolName','Xilinx Vivado','ToolPath', 'C:\Xilinx\Vivado\2024.1\bin\vivado.bat') |