|
|
|
|
@ -0,0 +1,203 @@ |
|
|
|
|
<!DOCTYPE html> |
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang=""> |
|
|
|
|
<head> |
|
|
|
|
<meta charset="utf-8" /> |
|
|
|
|
<meta name="generator" content="pandoc" /> |
|
|
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" /> |
|
|
|
|
<title>LoRaWAN Signal Mapping - User Manual</title> |
|
|
|
|
<style type="text/css"> |
|
|
|
|
code{white-space: pre-wrap;} |
|
|
|
|
span.smallcaps{font-variant: small-caps;} |
|
|
|
|
span.underline{text-decoration: underline;} |
|
|
|
|
div.column{display: inline-block; vertical-align: top; width: 50%;} |
|
|
|
|
</style> |
|
|
|
|
<style type="text/css"> |
|
|
|
|
a.sourceLine { display: inline-block; line-height: 1.25; } |
|
|
|
|
a.sourceLine { pointer-events: none; color: inherit; text-decoration: inherit; } |
|
|
|
|
a.sourceLine:empty { height: 1.2em; } |
|
|
|
|
.sourceCode { overflow: visible; } |
|
|
|
|
code.sourceCode { white-space: pre; position: relative; } |
|
|
|
|
div.sourceCode { margin: 1em 0; } |
|
|
|
|
pre.sourceCode { margin: 0; } |
|
|
|
|
@media screen { |
|
|
|
|
div.sourceCode { overflow: auto; } |
|
|
|
|
} |
|
|
|
|
@media print { |
|
|
|
|
code.sourceCode { white-space: pre-wrap; } |
|
|
|
|
a.sourceLine { text-indent: -1em; padding-left: 1em; } |
|
|
|
|
} |
|
|
|
|
pre.numberSource a.sourceLine |
|
|
|
|
{ position: relative; left: -4em; } |
|
|
|
|
pre.numberSource a.sourceLine::before |
|
|
|
|
{ content: attr(title); |
|
|
|
|
position: relative; left: -1em; text-align: right; vertical-align: baseline; |
|
|
|
|
border: none; pointer-events: all; display: inline-block; |
|
|
|
|
-webkit-touch-callout: none; -webkit-user-select: none; |
|
|
|
|
-khtml-user-select: none; -moz-user-select: none; |
|
|
|
|
-ms-user-select: none; user-select: none; |
|
|
|
|
padding: 0 4px; width: 4em; |
|
|
|
|
color: #aaaaaa; |
|
|
|
|
} |
|
|
|
|
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; } |
|
|
|
|
div.sourceCode |
|
|
|
|
{ } |
|
|
|
|
@media screen { |
|
|
|
|
a.sourceLine::before { text-decoration: underline; } |
|
|
|
|
} |
|
|
|
|
code span.al { color: #ff0000; font-weight: bold; } /* Alert */ |
|
|
|
|
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */ |
|
|
|
|
code span.at { color: #7d9029; } /* Attribute */ |
|
|
|
|
code span.bn { color: #40a070; } /* BaseN */ |
|
|
|
|
code span.bu { } /* BuiltIn */ |
|
|
|
|
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */ |
|
|
|
|
code span.ch { color: #4070a0; } /* Char */ |
|
|
|
|
code span.cn { color: #880000; } /* Constant */ |
|
|
|
|
code span.co { color: #60a0b0; font-style: italic; } /* Comment */ |
|
|
|
|
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */ |
|
|
|
|
code span.do { color: #ba2121; font-style: italic; } /* Documentation */ |
|
|
|
|
code span.dt { color: #902000; } /* DataType */ |
|
|
|
|
code span.dv { color: #40a070; } /* DecVal */ |
|
|
|
|
code span.er { color: #ff0000; font-weight: bold; } /* Error */ |
|
|
|
|
code span.ex { } /* Extension */ |
|
|
|
|
code span.fl { color: #40a070; } /* Float */ |
|
|
|
|
code span.fu { color: #06287e; } /* Function */ |
|
|
|
|
code span.im { } /* Import */ |
|
|
|
|
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */ |
|
|
|
|
code span.kw { color: #007020; font-weight: bold; } /* Keyword */ |
|
|
|
|
code span.op { color: #666666; } /* Operator */ |
|
|
|
|
code span.ot { color: #007020; } /* Other */ |
|
|
|
|
code span.pp { color: #bc7a00; } /* Preprocessor */ |
|
|
|
|
code span.sc { color: #4070a0; } /* SpecialChar */ |
|
|
|
|
code span.ss { color: #bb6688; } /* SpecialString */ |
|
|
|
|
code span.st { color: #4070a0; } /* String */ |
|
|
|
|
code span.va { color: #19177c; } /* Variable */ |
|
|
|
|
code span.vs { color: #4070a0; } /* VerbatimString */ |
|
|
|
|
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */ |
|
|
|
|
</style> |
|
|
|
|
</head> |
|
|
|
|
<body> |
|
|
|
|
<header> |
|
|
|
|
<h1 class="title">LoRaWAN Signal Mapping - User Manual</h1> |
|
|
|
|
</header> |
|
|
|
|
<h1 id="msc-summer-project">Msc-Summer-Project</h1> |
|
|
|
|
<blockquote> |
|
|
|
|
<p>My Msc Summer Project</p> |
|
|
|
|
</blockquote> |
|
|
|
|
<p>This repository contains my Masters-level summer project. The title is <em>LoRaWAN Signal Mapping</em>. The software contained here provides a complete system for mapping and visualising the signal coverage of <a href="https://thethingsnetwork.org/">The Things Network</a>.</p> |
|
|
|
|
<h2 id="structure">Structure</h2> |
|
|
|
|
<p>The system is split up into 2 primary parts:</p> |
|
|
|
|
<ol type="1"> |
|
|
|
|
<li>An Arduino program that collects the data.</li> |
|
|
|
|
<li>A Node.js program that stores and analyses the data</li> |
|
|
|
|
</ol> |
|
|
|
|
<p>It’s best explained with the aid of a diagram or two:</p> |
|
|
|
|
<p><img src="images/Manual%20Diagrams-Workflow.png" /></p> |
|
|
|
|
<p>The above flowchart describes the workflow when using the system:</p> |
|
|
|
|
<ol type="1"> |
|
|
|
|
<li>First, the IoT device is turned on and the TTN Listener is launched.</li> |
|
|
|
|
<li>Then, the IoT device is taken around the target area that needs mapping. This can be done by anyone - the device does not require operation beyond turning it on and off once provisioned.</li> |
|
|
|
|
<li>Once the device has been carried around the target area, the <code>DATA.TSV</code> file on the IoT device’s microSD card is copied off and placed on the server.</li> |
|
|
|
|
<li>The server data processor is then run to fold the data into the database.</li> |
|
|
|
|
<li>The AI trainer is now run on the collected data</li> |
|
|
|
|
<li>The data is displayed in a web browser, with the help of a web server</li> |
|
|
|
|
</ol> |
|
|
|
|
<p>The flow of data during use can be shown using a diagram:</p> |
|
|
|
|
<p><img src="images/Manual%20Diagrams-Data%20Flow.png" /></p> |
|
|
|
|
<h2 id="system-requirements">System requirements</h2> |
|
|
|
|
<p>This software has a number of requirements in order to function properly:</p> |
|
|
|
|
<ul> |
|
|
|
|
<li>A Linux-based server (<em>Ubuntu Server LTS</em> is recommended), with the following installed: |
|
|
|
|
<ul> |
|
|
|
|
<li>Node.js v10+ with npm 6+ - see <a href="https://github.com/nodesource/distributions">this website</a> for instructions on how to install a recent version of Node.js (for the various application server stuff)</li> |
|
|
|
|
<li>Git (for cloning the code repository)</li> |
|
|
|
|
<li><code>awk</code> (for text processing by the build script)</li> |
|
|
|
|
<li>Bash v4+ (for the build script)</li> |
|
|
|
|
<li>PHP (for the temporary test web server; not otherwise used any static web server will do)</li> |
|
|
|
|
</ul></li> |
|
|
|
|
<li>Arduino IDE (for programming the IoT device)</li> |
|
|
|
|
<li>Git (for cloning the code repository)</li> |
|
|
|
|
<li></li> |
|
|
|
|
</ul> |
|
|
|
|
<h2 id="getting-started">Getting Started</h2> |
|
|
|
|
<h3 id="step-0-initial-setup">Step 0: Initial setup</h3> |
|
|
|
|
<p>Before doing anything, clone this git repository to both the server (for the TTN listener etc.) and your local machine (for programming the IoT device):</p> |
|
|
|
|
<div class="sourceCode" id="cb1"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb1-1" title="1"><span class="fu">git</span> clone https://git.starbeamrainbowlabs.com/sbrl/Msc-Summer-Project.git</a></code></pre></div> |
|
|
|
|
<p><em>If you have somehow obtained a static copy of the code (e.g. through the University’s marking system), then skip the above and use that instead.</em></p> |
|
|
|
|
<p>Next, <code>cd</code> to the root of the repository and then run the setup task of the build script:</p> |
|
|
|
|
<div class="sourceCode" id="cb2"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb2-1" title="1"><span class="ex">./build</span> setup</a></code></pre></div> |
|
|
|
|
<p>On Windows, this should be run in <em>Git Bash</em> (accessible from the start menu when <em>Git</em> is installed - don’t forget to <code>cd</code> to the root of the repository).</p> |
|
|
|
|
<h3 id="step-1-build-a-device">Step 1: Build a device</h3> |
|
|
|
|
<p>First, a device must be built and provisioned. See <code>HARDWARE.md</code> in this repository for detailed instructions on how to do this.</p> |
|
|
|
|
<p>Once built, copy <code>iot/main/settings.custom.cpp.example</code> to <code>iot/main/settings.custom.cpp</code> and follow the instructions fill in the fields there. To do this, you’ll need to register the device using ABP (Activation By Personalisation) on <em>The Things Network</em>. <a href="https://www.thethingsnetwork.org/docs/devices/registration.html">This guide</a> tells you how to do this.</p> |
|
|
|
|
<p>It is suggested that the following Bash one-liner be used to generate a new encryption key in the right formats:</p> |
|
|
|
|
<div class="sourceCode" id="cb3"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb3-1" title="1"><span class="fu">head</span> -c16 /dev/urandom <span class="kw">|</span> <span class="fu">od</span> -tx1 <span class="kw">|</span> <span class="fu">awk</span> <span class="st">'{ gsub(/^0+\s+/, "", $0); toml=$0; gsub(/\s+/, "", toml); print("settings.toml format: " toml); arduino=toupper($0); gsub(/\s+/, ", 0x", arduino); print("settings.custom.cpp format: 0x" arduino); exit }'</span></a></code></pre></div> |
|
|
|
|
<p><em>(Paste it into a terminal and hit enter)</em></p> |
|
|
|
|
<p>Then, review <code>iot/main/settings.h</code> to make sure it matches your setup (e.g. all the pin numbers are correct)</p> |
|
|
|
|
<p>Next, copy the folders in <code>iot/libraries</code> to your Arduino IDE libraries folder.</p> |
|
|
|
|
<p>Finally, open <code>iot/main/main.ino</code> in the Arduino IDE and program the IoT device itself.</p> |
|
|
|
|
<h3 id="step-2-the-things-network-setup">Step 2: The Things Network Setup</h3> |
|
|
|
|
<p><em>This step should be completed on the server.</em></p> |
|
|
|
|
<p>Once a device has been constructed, running the <em>The Things Network</em> listener is next. This requires giving the system the <em>The Things Network</em> credentials.</p> |
|
|
|
|
<p>Edit the file called <code>settings.toml</code> in the root of this repository (or create it if it doesn’t exist), and make sure it contains the following:</p> |
|
|
|
|
<pre class="toml"><code>[ttn] |
|
|
|
|
app_id = "{APPLICATION_ID}" |
|
|
|
|
access_key = "{TTN_ACCESS_ID}" |
|
|
|
|
|
|
|
|
|
encryption_key = "{ENCRYPTION_KEY_FROM_STEP_1}" |
|
|
|
|
|
|
|
|
|
port = 8883 |
|
|
|
|
tls = true |
|
|
|
|
|
|
|
|
|
devices = [ "{DEVICE_NAME_FROM_TTN}" ]</code></pre> |
|
|
|
|
<p>The <code>app_id</code> and <code>access_key</code> can be obtained from <em>The Things Network Console</em>:</p> |
|
|
|
|
<p><img src="images/TTN-Main.png" /></p> |
|
|
|
|
<p>The device name can be obtained from step #1, when you registered the device with <em>The Things Network</em>. Alternatively, if the device is already registered, it can be obtained from the device list if you click the “X registered devices” text.</p> |
|
|
|
|
<p>With <code>settings.toml</code> filled in, you can now start TTN Listener:</p> |
|
|
|
|
<div class="sourceCode" id="cb5"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb5-1" title="1"><span class="ex">./build</span> ttn-listener</a></code></pre></div> |
|
|
|
|
<p>….it will display an error message if you forgot a value.</p> |
|
|
|
|
<p>Now that the TTN listener is running, the IoT device can be carried around and data collected.</p> |
|
|
|
|
<h3 id="step-3-processing-the-data">Step 3: Processing the data</h3> |
|
|
|
|
<p><em>This step should be completed on the server.</em></p> |
|
|
|
|
<p>Once data has been collected by the IoT device, it can then be processed by the Node.js server application.</p> |
|
|
|
|
<p>Copy the <code>DATA.TSV</code> file from the IoT device’s microSD card to the root of the repository on the server. It is suggested that <code>scp</code> (Linux) or <a href="https://winscp.net/">WinSCP</a> be used to do this.</p> |
|
|
|
|
<p>Next, fold <code>DATA.TSV</code> into the database like so:</p> |
|
|
|
|
<div class="sourceCode" id="cb6"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb6-1" title="1"><span class="ex">./build</span> process-data</a></code></pre></div> |
|
|
|
|
<p>Then, train the AIs on the collected data like this:</p> |
|
|
|
|
<div class="sourceCode" id="cb7"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb7-1" title="1"><span class="ex">./build</span> train-ai</a></code></pre></div> |
|
|
|
|
<p>The architecture of the neural networks trained can be customised by editing <code>settings.toml</code>. Check <code>server/settings.default.toml</code> for a guide on the different options available.</p> |
|
|
|
|
<h3 id="step-4-viewing-the-ai-output">Step 4: Viewing the AI output</h3> |
|
|
|
|
<p>With the AIs trained, the browser-based web interface can be used to display the output as a map. Any static web server will do, but the build script has one built-in (if PHP is installed) for convenience:</p> |
|
|
|
|
<div class="sourceCode" id="cb8"><pre class="sourceCode bash"><code class="sourceCode bash"><a class="sourceLine" id="cb8-1" title="1"><span class="ex">./build</span> server</a></code></pre></div> |
|
|
|
|
<p>If an alternative server is to be used, it should serve the <code>app/</code> directory that can be found in the root of this repository.</p> |
|
|
|
|
<h2 id="credits">Credits</h2> |
|
|
|
|
<ul> |
|
|
|
|
<li>Build Engine: <a href="https://gitlab.com/sbrl/lantern-build-engine">Lantern Build Engine</a> (written by me; proof available upon request)</li> |
|
|
|
|
</ul> |
|
|
|
|
<h3 id="iot-device">IoT Device</h3> |
|
|
|
|
<ul> |
|
|
|
|
<li>Random number generation: <a href="https://github.com/taoyuan/Entropy">Entropy</a></li> |
|
|
|
|
<li>LoRa Modem Driver: <a href="https://github.com/matthijskooijman/arduino-lmic">LMiC</a></li> |
|
|
|
|
<li>MicroSD card access: <a href="https://github.com/greiman/SdFat">SdFat</a></li> |
|
|
|
|
<li>GPS Decoder: <a href="https://github.com/mikalhart/TinyGPS">TinyGPS</a></li> |
|
|
|
|
<li>Free memory analyser: <a href="https://github.com/mpflaga/Arduino-MemoryFree/">Arduino-MemoryFree</a></li> |
|
|
|
|
</ul> |
|
|
|
|
<h3 id="node.js-server">Node.js Server</h3> |
|
|
|
|
<ul> |
|
|
|
|
<li>AI: <a href="https://www.npmjs.com/package/brain.js">Brain.js</a></li> |
|
|
|
|
<li>Database access: <a href="https://github.com/JoshuaWise/better-sqlite3/">better-sqlite3</a></li> |
|
|
|
|
<li>Encryption: <a href="https://www.npmjs.com/package/aes-js">aes-js</a></li> |
|
|
|
|
<li>The Things Network Access: <a href="https://github.com/mqttjs/async-mqtt">async-mqtt</a></li> |
|
|
|
|
<li>Dependency Injection: <a href="https://www.npmjs.com/package/awilix">Awilix</a></li> |
|
|
|
|
</ul> |
|
|
|
|
<h3 id="web-interface">Web Interface</h3> |
|
|
|
|
<ul> |
|
|
|
|
<li>Mapping Library: <a href="https://leafletjs.com/">Leaflet</a></li> |
|
|
|
|
<li><a href="https://github.com/SamHerbert/SVG-Loaders">Loading Animation</a></li> |
|
|
|
|
<li>Packaging system: <a href="https://rollupjs.org/">Rollup</a></li> |
|
|
|
|
</ul> |
|
|
|
|
<h2 id="useful-links">Useful Links</h2> |
|
|
|
|
<ul> |
|
|
|
|
<li>Entropy extraction from the watchdog timer vs the internal clock: https://github.com/taoyuan/Entropy</li> |
|
|
|
|
</ul> |
|
|
|
|
</body> |
|
|
|
|
</html> |
