The high level software works by:

1. Recognizing a person
2. Placing a bounding box around them
3. Commanding lower level drivers to move the stepper motors such that the person would be in the center of the frame.
4. Sending images and data to the server side using the python socket interface.

The image recognition is done using a pre-trained neutral network called YOLO (can be found at https://pjreddie.com/darknet/yolo/).

The conversion from pixels from the bounding box to stepper commands is given by approximating the distance from the camera to the target by assuming average person height and width (1.5' wide and 5'8" tall).

After measuring given box width in pixel from a series of given distances we were able to find the following relationship (figured left). With this information we are able to interpolate a distance from the target given the width of the bounding box (given by the object recognition portion of software).

Using this distance away and the targets displacement from the center of frame we can calculate the approximate angle needed which can easily be converted to steps (200 steps = 360 degrees).

Linux drivers were written to allow us to interface with our custom PCB's stepper controllers and UART pins. These drivers are specific to the De1-SoC and the Verilog written on its FPGA portion.