aerial surveillance drone

Aerial Surveillance Drone

The Aerial Surveillance Drone is a quadrotor controlled by a mobile application

able to autonomously identify and follow suspicious targets while transmitting

pictures and video taken by an onboard camera over the inte rnet to social media.

Colin Donahue

Samantha Kenyon

Jason Lowden

Benjamin Wheeler

February 14, 2013

Aerial Surveillance Drone 2

Table of Contents I. Overview ............................................................................................................................................... 3

A. Needs Statement .............................................................................................................................. 3

B. Objective Statement ......................................................................................................................... 3

C. Achievement of Objectives ............................................................................................................... 3

D. High-Level Diagram ........................................................................................................................... 4

II. Requirements Specification .................................................................................................................. 4

A. Marketing Requirements .................................................................................................................. 4

B. Engineering Specifications ................................................................................................................ 5

C. Analysis of Engineering Specification................................................................................................ 5

III. Concept Selection ................................................................................................................................. 6

A. Existing Systems ................................................................................................................................ 6

B. Concepts Selected ............................................................................................................................. 7

IV. Design .................................................................................................................................................. 10

A. Overall System ................................................................................................................................ 10

B. Autonomous Threat Detection ....................................................................................................... 10

C. Circuits and Assemblies................................................................................................................... 12

D. User Interface and Controls ............................................................................................................ 13

E. Engineering Standards .................................................................................................................... 16

F. Multidisciplinary Aspects ................................................................................................................ 17

G. Background ..................................................................................................................................... 17

H. Outside Contributors ...................................................................................................................... 17

V. Other Constraints and Considerations ................................................................................................ 17

A. Extensibility ..................................................................................................................................... 17

B. Manufacturability ........................................................................................................................... 17

C. Reliability ......................................................................................................................................... 18

D. Ethical Issues ................................................................................................................................... 18

E. Health and Safety Issues ................................................................................................................. 18

F. Societal Context .............................................................................................................................. 18

G. Sustainability ................................................................................................................................... 18

VI. Cost Estimates ..................................................................................................................................... 18

VII. Testing Strategy .................................................................................................................................. 19

A. Wireless Communications .............................................................................................................. 19

B. Threat Detection ............................................................................................................................. 19

C. List of Tests ..................................................................................................................................... 19

i. Unit Test Cases for Components ................................................................................................. 19

ii. Acceptance Tests ........................................................................................................................ 28

VIII. Risks .................................................................................................................................................... 32

A. List of Risks ...................................................................................................................................... 32

B. Analysis and Alleviation of Risks ..................................................................................................... 32

IX. Milestone Chart .................................................................................................................................. 33


I. Overview

A. Needs Statement

Unmanned aerial vehicles have been used by the military for surveillance purposes since World War II.

However, decades later, civilians interested in the security of their homes and businesses are forced to

set up networks of security cameras in order to monitor these areas for intruders. This method of

surveillance can easily become very expensive as the number of cameras needed increases, and great

care must be taken to ensure that the cameras are placed in a way that provides sufficient coverage.

Any intruders identified by the network of cameras can be difficult to track as the suspect leaves the

range of vision of one camera and enters that of another. Additionally, such systems are often

controlled in a single location, which can be very inconvenient. Even if it were possible for a civilian to

acquire an unmanned aerial vehicle to monitor an area, such devices are far outside the price range of

the average consumer, especially when size is taken into consideration. There is a need for a low-cost,

man-portable unmanned aerial vehicle that can be controlled from a portable control center and is

capable of monitoring a specified area, identifying and following potential threats, and sending data

back to the control center.

B. Objective Statement

The goal is to design and implement a digital system that has the ability to survey an area, identify a

suspicious target, and follow that target for a specified amount of time. The system will be controlled by

a portable device, and real-time updates, including status messages, pictures, and video, regarding the

security of its area of interest will be reported in a way that will allow its controller to access the

information from any location.

C. Achievement of Objectives

The overall system is a quadrotor controlled by a mobile device. The mobile device connects to a

wireless access point created by the quadrotor. The quadrotor itself has four motors, one on each arm

of the device. The quadrotor also has an accelerometer used for stabilization and control. Two

ultrasonic rangefinders are on the bottom of the quadrotor to determine altitude. A camera sensor is

also on the quadrotor, which will send data back to the mobile device so that image processing can be

done. Data from the accelerometers and ultrasonic rangefinders are also sent back to the mobile

device. The quadrotor can be controlled either by a human user or can do autonomous threat detection.

If the system is operating autonomously, data from the camera and other sensors will be used to

determine the next course of action. The mobile device will be running an application that achieves

these objectives.


D. High-Level Diagram

II. Requirements Specification

A. Marketing Requirements

1. Easily Portable — Must be able to transport easily to allow for flying in new areas.

2. Low Weight — Cannot be too heavy in order to allow for easy transport.

3. Low Cost — Since this system is for personal use and security it must be reasonable price for

the average person.

4. Easy to Use — It must be easy to use and control for an average person with any range of

expertise.

5. Safe operation — It must operate safely without any chance of harm to the user or other

bystanders.

6. Accurate Control — It must move accurately based on the controls given by the user and

therefore allow the user full control over the device.

7. Intuitive Feedback — It must provide some sort of feedback to the user to be used to

monitor the environment for personal security.

8. Ability to Function Outdoors — It must be able to fly in different weather conditions where

the wind speed is below 10 miles per hour and there is little to no precipitation.

9. Enough Power to Sustain Flight — It must be able to remain in flight for a significant period

of time.

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=-52FL4KgtK9JvM&tbnid=fUjfUs-8BP1oWM:&ved=0CAUQjRw&url=http://www.mentalmunition.com/2012/07/maker-of-ar-drone-invests-in-gis-is.html&ei=H4caUbKfFIbp0gGHhoG4Dw&bvm=bv.42261806,d.dmQ&psig=AFQjCNFCR-_c046dKLpDfguc_Cg18a8BrA&ust=1360778917027422


10. Area of interest — The system must remain in a certain area to prevent loss.

11. Maintainable System — It must be easy to fix and maintain to increase lifespan.

B. Engineering Specifications

Number Specification Associated Marketing

Requirements

1 The total weight of the system will not exceed 5 pounds. 1, 2, 5

2 The system will not leave a specified area of interest. 4, 5, 10, 11

3 The cost of the system shall not exceed $500. 3

4 The system will be constructed with components that can function properly in temperatures ranging from 0°C to 40°C.

8

5 The electronic components of the system will be enclosed in

order to prevent damage from light precipitation. 8

6 The system will be controlled by a portable device with switch-

like controls for power and data transmission settings. 1, 4, 7, 9, 11

7 The system will operate on full charge for at least 8 minutes. 9, 11

8 The system will detect potential threats from a distance of

between 3 and 20 feet. 6, 7

9 The system will not exceed 2 feet in width or length. 11

C. Analysis of Engineering Specification

Number Justification

1 The system needs to be easily portable. Anything heavier than this amount could pose difficulties for when moving the system. A system that is too heavy will require more powerful motors and will have reduced flight time. Keeping the weight of the system low will allow it to operate longer and be more maneuverable.

2 They system needs to remain in the specified area even if the potential threat detected leaves that area. This ensures that it is always controllable and never lost.

3 The cost must remain affordable for an average person, and presently the cost of systems such as this exceeds $500. Android is becoming a popular platform for consumer devices. Since this project is designed for that platform, a number of consumers would already own a device that is capable for running the quadrotor. Therefore, the only cost would be for the quadrotor.

4 The system must be able to function properly both indoors and outdoors in different climates including both warm and cold.

5 The system must be able to sustain a light amount of precipitation in the case of rain or snow.

6 The system needs to be easy to use and controllable requiring a remote control device that is


intuitive.

7 The system must be able to remain in flight long enough to be controlled to get to a location or to identify potential threats. Many systems of this type remain in flight for 5 minutes.

8 The system needs to identify a potential threat and allow the user to decide how to proceed to enforce their own security.

9 The system needs to be easy to transport and low in weight so it should not have too large of an area. This is the average size of some existing systems.

III. Concept Selection

A. Existing Systems Existing systems for aerial surveillance are widely used in the military and are starting to be used for

geographic surveying. Large military drones cannot be compared to this project since they operate on a

much larger scale. They are capable of staying in flight for hours or days and have ranges of thousands of

miles. They also carry heavy payloads used for combat. These systems tend to be fixed wing aircraft.

Aerial drones are also useful in places unsafe for humans. Drones can be flown directly in hurricanes or

dangerous weather systems to collect data that would otherwise be difficult to collect. Similarly, less

robust drones can be used to monitor areas or map terrain. Digital cameras by themselves are immobile,

but after being attached to a drone they can be used to monitor the health of an oil pipeline. Google

used a quadrotor to augment the mapping already done by its cars on the road.

This project is based around a comparatively low cost surveillance drone that can be controlled remotely

and can fly autonomously. A consequence of keeping the drone low cost is that it cannot be too

powerful or have a long flight time. For these reasons it is not very useful in military applications and is

unable to go into dangerous conditions. This project is focused on navigating in much smaller spaces and

tracking specific targets rather than mapping an entire area. Some open source initiatives such as

ArduCopter already have enabled a large number of features such as GPS waypoints but do not have any

image processing.

Engineers at the Illinois Institute of Technology’s Robotics Lab have developed the HyTAQ (Hybrid

Terrestrial and Aerial Quadrotor). As its name implies, it can travel on land as well as in the air. This is

possible because the quadrotor is enclosed in a cylindrical cage, which acts as a protective shield in the

air and a wheel and shock absorber on the ground. The ability to travel on the ground as well as in the

air results in substantial increases in travel distance and travel time. This design meets the need for the

ability to function in different environments and satisfies the specification that all electronic

components must be enclosed. However, the construction of such a cage would be difficult because the

system needs to be low cost and easy to maintain. Also, the last specification states that the system

must not exceed 1.5 feet in width or length, which means enclosing the quadrotor in a cage like the

HyTAQ would probably result in too large of a system.


Some autonomous aerial vehicles are equipped with additional cameras that are dedicated solely to

scanning the surrounding area for potential obstacles. This type of solution relies on computer vision

algorithms to detect objects. Additional code then decides whether objects detected by the computer

vision code are potential threats to the vehicle and whether the vehicle should actively attempt to avoid

them. In theory, this is a good option because the system will already be using a camera to detect

threats, so very little code modification would be necessary in order to incorporate the detection of

obstacles and the steps necessary to avoid detected objects. However, because this option requires

multiple cameras in addition to the camera already being used to detect and track targets, the cameras

used would have to be fairly inexpensive. As a result, the video quality of these cameras could

potentially be subpar, which would make it more difficult to detect objects using computer vision

algorithms.

There are two different ways to surveil with a quadrotor system. There are systems that hold external

cameras. The Draganflyer X4 has a module in which one of three styles of camera can be placed so that

video and photos can be captured during flight. These cameras are self-powered and self-sustained. In

this system the camera is completely separate from the quadrotor, and this does not allow for streaming

back to the controller. Another way that this is done is using a camera that is integrated with the

system. The Parrot AR Drone 2.0 is a quadrotor with a built in 1280 x 720 front facing camera that

interfaces back to the controller. This camera is integrated into the quadrotor using some form of power

from within the device.

A threat detection system was developed by DARPA, and it is called the Cognitive Technology Threat

Warning System. It uses a 120-megapixel camera, EEG reader and various processing algorithms to

determine threats. The system was able to detect 91% of threats using a combination of image

processing and the human mind’s innate ability to find threats. The false alarm rate was also very low,

and the system was able to determine that a flying bird is not a threat, while a flying helicopter most

likely is.

B. Concepts Selected There are three main types of aerial vehicles that can be used for surveillance. These options are fixed

wing airplanes, helicopters and quadrotors. Fixed wing airplanes are most useful for carrying heavy

payloads while traveling long distances. Helicopters have shorter ranges but more maneuverability due

to their ability to hover in place. However, they lack the stability of a quadrotor. Quadrotors originally

were considered much more difficult to fly than helicopters due to their increased number of propellers

and complex control systems. The advent of electronic control systems and high quality sensors have

made quadrotor control easier, and stable quadrotors can now be bought as toys. It is for this reason

that a quadrotor was chosen for the final vehicle.

Quadrotors also have some kinetic and design advantages over traditional helicopters. They do not need

to vary the pitch angle of their blades, which makes mechanical construction simpler. They also need

reduced blade sizes in comparison to helicopters, which consumes less energy. Helicopters also

generally need an extra tail rotor, which contributes nothing to the overall system lift. Quadrotor

navigation can be accomplished with fixed rotors quite easily by simply varying motor power opposing


rotors and making the quadrotor roll. More advanced aerial maneuvers such as flips are also easy to do

in a quadrotor but will not be utilized in this project.

Since this drone will be carrying a payload and monitoring an area, precise control of a camera is very

important. Quadrotors are quite stable when hovering and moving, which reduces blur in images. This

feature is helpful when doing image processing since there will be less noise. It is also easy to mount a

camera on a quadrotor’s frame without disrupting its center of gravity too much. The four rotors of a

quadrotor also allow easier compensation for a disturbance in the system’s physical characteristics.

There are many options when choosing a quadrotor. The specifications state that the quadrotor should

be able to fly for 8 minutes on a single charge and it should be able to detect and follow threats

autonomously. Both of these requirements need to be considered when choosing a quadrotor. Our

initial choice was the Blade mQX shown below.

The problem with this selection is that this quadrotor is small and does not have many built in features.

For this quadrotor to be able to fly autonomously it needs added hardware. With hardware added, this

small quadrotor is unable to maintain flight for sufficient time. This is due to power management and

weight. This quadrotor is just too small to be effective for our project.

From there we selected the Parrot AR drone. This quadrotor is larger than the previous one, and it also

has many more features. It does not require that we add hardware to it, and it also has an extensive API

that can be used for control.

The specifications state that the system should be safe and able to function in different environments,

including both indoors and outdoors. In order to meet these specifications, it is imperative that the

system is able to both observe its surroundings to determine whether there are any obstacles in its path

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=SMMkWjpJJsFxEM&tbnid=3-dYWRAaNtegcM:&ved=0CAUQjRw&url=http://www.horizonhobby.com/products/blade-mqx-bnf-BLH7580&ei=7IQaUcvKIcbJ0QHe3oHIBA&bvm=bv.42261806,d.dmQ&psig=AFQjCNEosuqU2XNrGrHL7ohrnJ0n-HJ6-Q&ust=1360778857207690

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=2wbVdU62_bW8cM&tbnid=6xV71tLvWW4aQM:&ved=0CAUQjRw&url=http://www.blisteredthumbs.net/2011/06/parrot-ar-drone-e3-interview/&ei=cYUaUe-NB-yw0QH4loCYCQ&bvm=bv.42261806,d.dmQ&psig=AFQjCNFCR-_c046dKLpDfguc_Cg18a8BrA&ust=1360778917027422

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=SMMkWjpJJsFxEM&tbnid=3-dYWRAaNtegcM:&ved=0CAUQjRw&url=http://www.horizonhobby.com/products/blade-mqx-bnf-BLH7580&ei=7IQaUcvKIcbJ0QHe3oHIBA&bvm=bv.42261806,d.dmQ&psig=AFQjCNEosuqU2XNrGrHL7ohrnJ0n-HJ6-Q&ust=1360778857207690

http://www.google.com/url?sa=i&rct=j&q=&esrc=s&frm=1&source=images&cd=&cad=rja&docid=2wbVdU62_bW8cM&tbnid=6xV71tLvWW4aQM:&ved=0CAUQjRw&url=http://www.blisteredthumbs.net/2011/06/parrot-ar-drone-e3-interview/&ei=cYUaUe-NB-yw0QH4loCYCQ&bvm=bv.42261806,d.dmQ&psig=AFQjCNFCR-_c046dKLpDfguc_Cg18a8BrA&ust=1360778917027422


that needed to be avoided and know its orientation in three dimensional space so that it can

successfully maintain flight for as long as possible.

In order to do threat detection, a system on the scale of DARPA’s with an EEG reader and laptop doing

heavy processing is not possible. Instead the detection has to be relatively simple and use only images

as input. The image processing will not all be built from the ground up, and OpenCV will be used to do

initial work. The algorithms available in OpenCV provide a good starting point for threat detection, and

there is also a version of OpenCV that runs on Android platforms.

The method of determining threats will be implemented in three ways. The methods are File Feature

Detect, Dynamic Feature Detect and Fast Object Detect. The first method takes a file as input and is

meant to be used when a user knows exactly what kind of threat he/she is looking for. The second

method lets the user tap an onscreen object and therefore requires the user to be actively looking at the

video stream. This method is meant to be used when the user does not know what the threat will be

ahead of time but knows the threat when he/she sees it. The final method looks for fast moving objects

in a scene and then tracks the fastest one. In all detection methods the system will attempt to follow

the threat that was detected.

Communications to and from the quadrotor are the final design concept to consider. With the Parrot AR

Drone, a Wi-Fi communication link is automatically established by the quadrotor. Using the existing UDP

and TCP data ports, the control data, configuration information, navigation data, and video streams are

sent to the receiver. The difficulty that is encountered with this quadrotor is the ability to transmit the

images that are obtained to the internet in a timely manner. On most mobile devices, there is a single

Wi-Fi adapter that can be used; since it is already used for quadrotor communication, it cannot be used

for the transmission of data. Therefore, two other options exist. The first option is to use the cellular

network if the device has one. The use of this network could be relatively difficult, and it is not

guaranteed to be present on all mobile devices. The second option is the use of Bluetooth to transmit

back to a base station. Bluetooth adapters are widely available on mobile devices, making this an ideal

option for the transmission.

On the mobile application, there can be an asynchronous task that transmits the data to the base

station. The base station will be responsible for reading the information sent over the communication

channel whenever it is sent. Once the information is received, it can be uploaded to social networking

websites using existing APIs.

While this approach does not allow the system to publish the images immediately, it does allow the

system to respond in a reasonable amount of time. Without this notion of communication, the system

would have two options. The first option is to wait until the quadrotor lands and the mobile device is

able to reconnect to Wi-Fi internet. The other option is to switch the network in which the adapter is

connected. While this may appear as a good solution, it would be difficult maintain adequate control of

the quadrotor due to the delays associated with authentication and connection. The intermediate hop

requires more components, but it allows the images to be displayed with less delay.


IV. Design

A. Overall System The following table and diagram show the overall system design that will be used. In a quick overview,

the quadrotor is responsible for obtaining the data and the mobile application displays that information

to the user. At the same time, the user is able to provide some control information to the quadrotor in

both manual and autonomous modes of operation.

Level 0 Functional Decomposition

Module Quadrotor

Inputs Control Signals: Used for moving the quadrotor; these can be generated by user intervention or from the autonomous control mechanism Portable Power Source: Used to operate the on-board electronic components for data acquisition and control Voltage: 11.1 V

Outputs Video Feed: Output of the on-board camera Output Resolution: 640x480 Pictures to Social Networking: Images of what the quadrotor is currently viewing Output Resolution: At least 1 megapixel

Functionality The quadrotor will have the ability to be controlled from manual user input or through autonomous control by following a designated target. The video feed will be sent whenever the feature is enabled; images will be placed on social media as the quadrotor travels from origin to destination.

B. Autonomous Threat Detection There are two main components in the mobile application. One is the user interface and controls, which

is described in the next subsection. The second component consists of the algorithms to be used for

autonomous control. There are three distinct ways threats will be detected and followed. The overall

operation of the component is shown in the figure below.


In the File Feature Detect algorithm the user will be able to choose a picture file on the device that

represents the threat he/she wants to detect. The matching between the file and still images from the

video stream will be accomplished using the Fast Approximate Nearest Neighbors with Automatic

Algorithm Configuration method proposed by Muja and Lowe. This method matches points in a

reference set (user selected file) to points in a much larger dataset (image from video stream). The

algorithm does not search for exact matches since they will in general be impossible to find, but instead

attempts to find the closest approximations to the target. The method is well suited to the quadrotor

system because it is fast and was designed to work with Euclidean vector spaces in mind. An example of

our preliminary implementation of this is shown below.

The Dynamic Feature Detect algorithm is similar to File Feature Detect in that the user has control over

what is detected as a threat. In this case the user will be actively watching the video stream from the

quadrotor and tap an object on screen. The system will attempt to continue focusing on that object and

following it. This algorithm will be implemented using blob detection and color recognition. Blob

detection finds places in a scene where there are clumps of pixels with similar properties. Blob

detection algorithms generally use combinations of detecting edges and finding local minima and


maxima. If a user taps an object that is part of a well-defined blob then the algorithm will attempt to

keep track of that blob and follow it. If the user does not tap on a blob then the algorithm will fail and

no object will be tracked.

The Fast Object Detect algorithm differs from the previous two in that it requires no user input. Instead

the algorithm will track blobs in the scene and determine how fast they are moving. This will require

movement data about the quadrotor so that the absolute speed can be determined rather than the

relative speed. Objects moving above a certain speed will be classified as threats. Blobs will be detected

the same way they were in the Dynamic Feature Detect algorithm except there is additional complexity

involved with tracking all blobs in the scene. A size threshold will be made so that only blobs above a

certain size will be continually tracked.

C. Circuits and Assemblies

No additional circuits or assemblies will be added to the device. The internal block diagram of the

relevant components of the quadrotor is shown below.


D. User Interface and Controls

The user interface in this system is the mobile application. The mobile application, implemented using

Android, will allow users to configure the threat detection functionality in an initial settings pane before

transitioning to the main screen of the application. The main screen will display the output of the

camera in the background with sensor data on the sides. In manual control mode, the application will

convert touch events into commands to send to the quadrotor by interfacing with the Parrot AR Drone

API. In autonomous mode, this functionality is disabled. In either mode, a method for switching modes

will be provided.

The following figure shows the initial startup screen when the application is launched on the mobile

device. Its simplicity allows the user to determine the mode of operation with ease.

The next figure shows the options that will be available if the user chooses to detect threats

autonomously.


As this panel is displayed, there are a number of options that allow the user to determine how to follow

a target autonomously. The flowchart below shows the sequence of events that must occur before the

quadrotor is able to track an object or begin autonomous threat detection.

Following the main setup screen, the application will display the current mode of control. In this mode,

the user has the option to change modes by pressing the button. For added support, the sensor values

and the camera feed(s) will be displayed to view the current area.


According to the main display panel, the following flow chart is provided to describe the mode of

operation. This is a particular user scenario that changes the direction and method of control before

landing the quadrotor.


E. Engineering Standards In this design, the following standards will be used. In addition to the standards, the location and

description of its application are also provided.

Engineering Standards Location in Design

Bluetooth Used for communication from mobile application to desktop application.

WiFi Used for communication between quadrotor and mobile application.

Android SDK v15 Used to implement mobile application.

Java SDK v7 Used to implement desktop application.

Open CV 2.4.3 Used for object detection and tracking


F. Multidisciplinary Aspects 1. Software Engineering: Development of a desktop and mobile application to be used for manual

control of the quadrotor, monitoring the surrounding area when the quadrotor is in

autonomous flight mode, and sending images to social media.

2. Computer Science: Development of algorithms for controlling the quadrotor when it is in

manual flight mode and for detecting potential threats to follow when it is in autonomous

mode.

G. Background 1. Digital Control Systems: Modeling and design using a variety of approaches, including the use of

Matlab and Simulink for simulation

2. Modeling of Real-Time Systems: UML modeling of various real-time and implementation in an

object-oriented environment

3. Software Engineering: Basis for the design, test, and implementation of software systems

4. Applied Programming: Learned basic C concepts and low-level programming

H. Outside Contributors Dr. A. Mondragon: Quadrotor Supplier

Jeffrey Paul Myers: Flight Control Consultant

V. Other Constraints and Considerations

A. Extensibility While this quadrotor will be controlled by the provided microcontroller, the communication mechanism

will be implemented for a variety of user interfaces. The quadrotor control system can be extended to

other projects if a greater amount of control is required for some other applications. Once the

communication is completed, this software/hardware component can be used with other projects that

have a similar interface. For the interfaces that are device specific, such as to the Apple iPhone or

Windows Phone 8, the application could be recycled to match other requirements.

B. Manufacturability The selection of components for this system could reduce the general manufacturability of the system.

Each quadrotor has a unique control system and different manufacturers may use different interfaces

for control. It may be difficult to implement the custom control interface for this system that could apply

to all quadrotors. The camera for video feed could easily be incorporated into other designs because the

communication will be separate from the control system. In this case, the additional equipment could be

manufactured for the integration into other flying systems.


C. Reliability The reliability of this system is based on its ability to last a significant amount of time with little damage

as a result of its operation. With the appropriate safety measures, this system can be reliable in the

sense that it is able to survive a drop or fall. The second aspect of reliability is the software component.

If there is a target, then it should be followed until the boundary conditions of the system are met or the

target is changed. If the same target is selected multiple times, the software should be reliable in that

the target will have the same outcome every time.

D. Ethical Issues Ethical issues are associated with the selection of potential targets and tracking. Whenever a target is

selected, it should be out of evidence of suspicion or consent for demonstration. When it comes to

tracking individuals, it will be unethical to follow a target without some informed consent or knowledge

that tracking may occur in a specified area.

E. Health and Safety Issues In general, flying devices pose a potential risk because parts could fall from the device or the device

itself could fall. Safety mechanisms can be used to mitigate the potential damage caused to person and

property, but the overall risk cannot be eliminated. The two greatest safety issues are the falling device

and the propellers. Even though the device will not exceed 1 pound, a falling object has the potential to

cause injury. The propellers are a safety issue for low-flying objects. With the current quadrotor, there is

a Styrofoam guard around the propellers that protects them from running into objects.

F. Societal Context Based on current laws, there is a certain level of privacy that every individual comes to expect. Since this

system has the ability to fly to many locations, privacy must be maintained, and it should not be used in

a home. If the wrong people acquire this device or are able to control the system, then privacy may be

over-stepped, resulting in illegal behavior. With proper marketing, the general public could see this

system as a new means to share information via the internet, rather than a privacy invasion device.

G. Sustainability Even though some of the device components are made of plastic and can be recycled if broken or no

longer needed, some components must be disposed properly. If rechargeable battery packs are used,

proper disposal is required. Additionally, the microcontroller components must be placed in the proper

form of recycling or have another application for reuse. By selecting off-the-shelf components, recycling

could mean the integration into another system.

VI. Cost Estimates

Component Description Cost Our Cost Availability

Quadrotor $299.99 $0.00 Already Acquired

Dr. Mondragon


Android Device $199.00 $0.00 Already Owned

Total Cost $498.99 $0.00

VII. Testing Strategy

A. Wireless Communications Wireless communication consists of two areas of testing. The first area of testing is the communication

of data between the quadrotor and mobile application. The second area is the communication of images

from the mobile application to the base station. The quadrotor communication is the higher of the two

in terms of priority, as adequate control must be effectively maintained at all times. A number of simple

unit tests will be used to verify the functionality at each stage. The test plan in section C describes the

tests that will be performed.

B. Threat Detection The threat detection module can be testing independently of the whole system. Just using a normal PC

webcam to test the threat detection algorithms gives a good baseline for how well they will perform on

the actual quadrotor. The two main metrics to test are false positives and percent of threats detected.

It is desirable to keep the number of false positives low and detect a high percentage of threats.

Depending on which threat detection algorithm is being tested, different threats will be displayed to a

camera, and the performance of the system will be evaluated.

Once the algorithms work well on a stationary camera, they will be evaluated on a moving camera. This

testing does not necessarily have to be done on the quadrotor; the camera can also be moved around

manually by a person. The next step is to ensure the algorithms can run on a microcontroller or a

mobile device. If those devices are not fast enough, the algorithms will need to be simplified until they

run at a reasonable speed.

Once threat detection is working well enough, the quadrotor will be allowed to follow the threat it

detects. Testing this is more qualitative and requires that the quadrotor stays close enough to threat so

that the threat cannot escape.

C. List of Tests

i. Unit Test Cases for Components

1. Bluetooth Communication from Application to Base Station

Test Number: 1.1 Date Performed:

Description: Establishment of a Bluetooth communication channel between the mobile device and an Internet connected base station

Team Member:

Jason

Inputs Expected Output Observed Output


Mobile Device Initiation

Communication channel is successfully opened


Description: Communicate data over the Bluetooth channel that can be interpreted by the application for social networking

Team Member:

Jason


Send text/image from mobile device to desktop application

Received data is the same the transmitted data in a relatively fast response time

2. Display of Camera Feeds on the Application


Description: Display camera feed on mobile application

Team Member:

Jason


Camera feed from the quadrotor

Real time video for immediate display to the user


Description: Allowing switching between camera views

Team Member:

Jason


Press the button to switch between forward and bottom cameras

Video feed changes to display the appropriate view


3. Configuration of Input Parameters for Flight


Description: Allow configurations of the mobile application through a panel interface

Team Member:

Jason


Maximum tilt angle

Quadrotor tilt angle changes to reflect the new value

Changed maximum altitude

Altitude changes based on the new value

4. Obtain Sensor Data for Application Display


Description: Display the information sent from the quadrotor about its position back to the quadrotor

Team Member:

Jason


Obtain pitch, roll, yaw

View real-time values about position

Obtain altitude Display current altitude from the ground

5. Interaction with Social Media


2/22

Description: Verify that text can be successfully be transmitted to a social network and used to post an update

Team Member:

Ben


Update text Credentials Update verification



2/22

Description: Verify that text and images can be successfully be transmitted to a social network and used to post an update

Team Member:

Ben


Update text and image

Credentials Update verification

6. Manual Quadrotor Control


3/8

Description: Verify that the mobile application can detect and handle touch events and motion events

Team Member:

Ben, Jason


Touch events Motion events Circles drawn around events to show they are being handled


3/15

Description: Translate touch events and motion events into commands to send to the quadrotor

Team Member:

Ben, Jason


Touch events Motion events Quadrotor commands

7. Autonomous Quadrotor Control


4/12

Description: Verify that the mobile app can interpret data from the quadrotor's sensors

Team Member:

Ben, Jason


Sensor data Display data on screen



4/19

Description: Translate quadrotor sensor data into commands to send back to the quadrotor

Team Member:

Ben, Jason


Sensor data Quadrotor commands

8. Switching Between Methods of Control


4/26

Description: Verify that the mobile application can switch from manual control to autonomous control

Team Member:

Ben, Jason


Request to switch modes

Manual controls are disabled

Autonomous controls are enabled


4/26

Description: Verify that the mobile application can switch from autonomous control to manual control

Team Member:

Ben, Jason


Request to switch modes

Autonomous controls are disabled

Manual controls are enabled

9. Target Selection


3/22

Description: Verify that the mobile application can select between different threat detection algorithms

Team Member:

Ben, Jason


User selection The correct threat detection algorithm is used


10 File Feature Detection


3/8

Description: Verify that a stationary threat can be detected using file feature detection.

Team Member:

Sam


Image with a stationary threat

Feature to detect

Red box around threat


3/8

Description: Verify that a moving threat can be detected using file feature detection.

Team Member:

Sam


Series of images with a moving threat

Feature to detect

Data output that shows how the threat moved

11. Dynamic Feature Detection


3/22

Description: Verify that a stationary threat can be detected using dynamic feature detection.

Team Member:

Colin, Sam


Image with a stationary threat

Location of Feature to detect

Red box around threat



3/22

Description: Verify that a moving threat can be detected using dynamic feature detection.

Team Member:

Colin, Sam


Series of images with a moving threat

Location of Feature to detect


12. Tracking


3/15

Description: Verify that a moving threat can be detected using feature detection while the camera sensor itself is moving.

Team Member:

Colin


Series of images taken from a moving camera of a moving threat

Feature to detect



3/15

Description: Verify that a fast moving threat can be detected.

Team Member:

Colin


Series of images with a threat moving above a certain threshold




3/25

Description: Verify that a fast moving threat can be detected while the camera sensor itself is moving.

Team Member:

Colin


Series of images with a threat moving above a certain threshold

Sensor data showing how the quadrotor is moving


13 Quadrotor Controls


4/19

Description: Verify that correct control commands are sent to the quadrotor depending on the location of the threat.

Team Member:

Sam, Colin


Threat location in image

Sensor data showing how quadrotor is moving

Commands that will direct quadrotor to the threat location

14 Mobile Image Processing


3/25

Description: Verify that image processing can be executed on a mobile device with acceptable latency.

Team Member:

Sam, Colin


File Feature Detect algorithm

The expected output running below a certain maximum latency

Dynamic Feature Detect Algorithm

The expected output running below a certain maximum


latency

Fast Object Detect Algorithm

The expected output running below a certain maximum latency

15 Mobile Application to Quadrotor Communication


Description: Establishment of a communication channel between the mobile device and the quadrotor.

Team Member:

Jason


Mobile Device Initiation

Communication channel is successfully opened


Description: Complete quadrotor control from mobile application.

Team Member:

Jason


Takeoff Quadrotor hovers at 1m above ground

Landing Quadrotor gracefully lands

Emergency Landing

Quadrotor cuts all power to motors; landing will not be graceful

Change Altitude Quadrotor increases or decreases its height

Move left/right Quadrotor changes position

Move forward/backward

Quadrotor changes position


Rotation Quadrotor rotates to look at new position

ii. Acceptance Tests

Test Number: 1 Date Performed:

4/26

Description: Check the engineering requirement: The total weight of the system will not exceed 5 pounds.

Team Member:

Colin


The complete system

Scale The total weight will not exceed 5 pounds

Test Number: 2 Date Performed: 4/26

Description: Check the engineering requirement: The system will not leave a specified area of interest.

Team Member: Sam


Use the system manually and with all threat detection modes.

The system should not follow invalid threats.

The user can intervene at any time to ensure the system does not leave a certain area.


4/26

Description: Check the engineering requirement: The cost of the system shall not exceed $500.

Team Member:

Jason


The cost of all system components

The cost of all components in the system is less than or equal to $500



4/26

Description: Check the engineering requirement: The system will be constructed with components that can function properly in temperatures ranging from 0°C to 40°C.

Team Member:

Ben


All components of the system

Varying ambient temperature The system is functional for all temperatures


4/26

Description: Check the engineering requirement: The electronic components of the system will be enclosed in order to prevent damage from light precipitation.

Team Member:

Colin



Upon visual inspection, all electronic components are not exposed


Test Number: 6 Date Performed: 4/26

Description: Check the engineering requirement: The system will be controlled by a portable device with switch-like controls for power and data transmission settings.

Team Member: Sam



The mobile application used to control the device will have a switch for power settings

The mobile application used to control the device will have a switch for data transmission settings


4/26

Description: Check the engineering requirement: The system will operate on full charge for at least 8 minutes.

Team Member:

Jason


All components of the system powered on and in use

The system will remain functional for at least 8 minutes


4/26

Description: Check the engineering requirement: The system will detect potential threats from a distance of between 3 and 20 feet.

Team Member:

Ben



Threats ranging from distances of 3 to 20 feet

The system is able to detect all threats



4/26

Description: Check the engineering requirement: The system will not exceed 2 feet in width or length.

Team Member:

Colin



Ruler The components of the system will not exceed 2 feet in width or length


VIII. Risks

A. List of Risks

1. Target Identification

2. Running out of battery

3. Keeping system in constrained area

4. Inside vs. outside operation

5. Loss of communication

B. Analysis and Alleviation of Risks

Risk Number

Analysis Solution

1

Identifying incorrect targets will cause the system to track the wrong targets. Not being able to identify targets makes the system largely useless from an autonomous standpoint.

When the system is unable to recognize the correct target, a user will be able to switch the system to manual control.

2 If the system runs out of battery while in flight its fall could damage the payload and motors.

The system will have padding underneath it so that if it falls straight down nothing on the main chassis of the system will be damaged. The battery level will also be monitored to ensure the user can take action if the charge level is low.

3

When in autonomous flight mode the system is able to go wherever its programming allows, which may not always correspond to a safe location.

The system will be able to switch to manual control mode at any time so the user can stop it from going too far away. A kill switch will also be implemented so that the system can be stopped at any time.

4

In order to detect threats autonomously, the Aerial Surveillance Drone needs to be aware of its surroundings so that it can land properly and avoid obstacles in its path.

The bottom of the system has ultrasonic range

sensors so it can do controlled landings based

on its altitude. If the system hits an obstacle

and it reaches an unsafe tilt level then it will

immediately turn off its rotors.

5 If communication with the system is lost, it has to know how to navigate by itself.

The system will attempt to land safely if

communication with the control device is lost.


IX. Milestone Chart

Name and Description Scheduled Completion Date

Team Member Responsible

Current Status

User Interface/Application 4/26 Ben, Jason Incomplete

Control UI 3/15 Ben, Jason Incomplete

Streaming Pane 3/15 Ben, Jason Incomplete

Sensors Pane 4/12 Ben, Jason Incomplete

Status Pane 4/12 Ben, Jason Incomplete

Configuration Pane 4/19 Ben, Jason Incomplete

Autonomous Mode 4/19 Ben, Jason Incomplete

Switch between Control Modes 4/26 Ben, Jason Incomplete

Interact with Social Media 4/26 Ben Incomplete

Tracking 3/29 Colin, Samantha Incomplete

OpenCV File Feature detection 3/8 Samantha Incomplete

OpenCV High Speed detection 3/15 Colin Incomplete

OpenCV Dynamic Feature detection 3/22 Colin, Samantha Incomplete

Tracking while in motion 3/22 Colin, Samantha Incomplete

Tracking on low-perf device 3/29 Colin, Samantha Incomplete

Communication 4/12 Jason, Ben Incomplete

Between mobile and desktop apps 3/8 Jason, Ben Incomplete

Send meaningful data to desktop 3/15 Jason, Ben Incomplete

Display camera feed on mobile app 4/12 Jason, Ben Incomplete

Display sensor data on mobile app 4/12 Jason, Ben Incomplete

aerial surveillance drone

Documents