Browse Source

Fill out the README with instructions.

master
Starbeamrainbowlabs 2 years ago
parent
commit
04120c9f6b
  1. 16
      HARDWARE.md
  2. 159
      README.md
  3. 18
      build
  4. 0
      images/Manual Diagrams.drawio
  5. 0
      images/Signal Mapping Architecture.drawio
  6. 5
      server/settings.default.toml

16
HARDWARE.md

@ -0,0 +1,16 @@
# Hardware
TODO: Fill this out with a tutorial
## Parts list
- [RFM95 w/ breakout](https://onecall.farnell.com/seeed-studio/114990615/868mhz-dragino-lora-shield-for/dp/SC14518) (alternatively [this](https://shop.pimoroni.com/products/adafruit-rfm95w-lora-radio-transceiver-breakout?variant=19595325639))
- [GPS](https://www.hobbytronics.co.uk/ublox-gps-neo6m) (or [this](https://thepihut.com/products/gps-module-with-enclosure?variant=27741075537))
- [microSD breakout](https://www.hobbytronics.co.uk/microsd-card-regulated-v2)
- DC in + USB-a power regulator (or maybe [this](https://www.adafruit.com/product/1833))
- [microSD Card](https://smile.amazon.co.uk/SanDisk-microSDHC-Memory-Adapter-Performance/dp/B073K14CVB/) - any card that's 1GiB+ will do.
- [Arduino Uno](https://smile.amazon.co.uk/Elegoo-Board-ATmega328P-ATMEGA16U2-Arduino/dp/B01EWOE0UU/)
- [Breadboard](https://shop.pimoroni.com/products/solderless-breadboard-400-point)
- Jumper cables (e.g. [a](https://thepihut.com/products/adafruit-premium-female-male-extension-jumper-wires-20-x-3), [b](https://thepihut.com/products/adafruit-premium-male-male-jumper-wires-20-x-6-150mm), and [c](https://thepihut.com/products/adafruit-premium-female-female-jumper-wires-20-x-6-150mm))
- [Battery](https://smile.amazon.co.uk/Anker-PowerCore-Aluminum-Portable-Lipstick-Sized-Black/dp/B005QI1A8C/) - Size currently unknown - I'll need to run some tests to see how much power it uses to figure out the right battery size.
- [Project box](https://smile.amazon.co.uk/waterproof-plastic-Enclosure-Power-junction-Grey/dp/B01JHBHNMA/) - if we can't 3D print one
- [TPL-5111](https://www.adafruit.com/product/3573) - also available from Pimoroni

159
README.md

@ -2,35 +2,148 @@
> My Msc Summer Project
## Hardware Used
- [RFM95 w/ breakout](https://onecall.farnell.com/seeed-studio/114990615/868mhz-dragino-lora-shield-for/dp/SC14518) (alternatively [this](https://shop.pimoroni.com/products/adafruit-rfm95w-lora-radio-transceiver-breakout?variant=19595325639))
- [GPS](https://www.hobbytronics.co.uk/ublox-gps-neo6m) (or [this](https://thepihut.com/products/gps-module-with-enclosure?variant=27741075537))
- [microSD breakout](https://www.hobbytronics.co.uk/microsd-card-regulated-v2)
- DC in + USB-a power regulator (or maybe [this](https://www.adafruit.com/product/1833))
- [microSD Card](https://smile.amazon.co.uk/SanDisk-microSDHC-Memory-Adapter-Performance/dp/B073K14CVB/) - any card that's 1GiB+ will do.
- [Arduino Uno](https://smile.amazon.co.uk/Elegoo-Board-ATmega328P-ATMEGA16U2-Arduino/dp/B01EWOE0UU/)
- [Breadboard](https://shop.pimoroni.com/products/solderless-breadboard-400-point)
- Jumper cables (e.g. [a](https://thepihut.com/products/adafruit-premium-female-male-extension-jumper-wires-20-x-3), [b](https://thepihut.com/products/adafruit-premium-male-male-jumper-wires-20-x-6-150mm), and [c](https://thepihut.com/products/adafruit-premium-female-female-jumper-wires-20-x-6-150mm))
- [Battery](https://smile.amazon.co.uk/Anker-PowerCore-Aluminum-Portable-Lipstick-Sized-Black/dp/B005QI1A8C/) - Size currently unknown - I'll need to run some tests to see how much power it uses to figure out the right battery size.
- [Project box](https://smile.amazon.co.uk/waterproof-plastic-Enclosure-Power-junction-Grey/dp/B01JHBHNMA/) - if we can't 3D print one
- [TPL-5111](https://www.adafruit.com/product/3573) - also available from Pimoroni
This repository contains my Masters-level summer project. The title is _LoRaWAN Signal Mapping_. The software contained here provides a complete system for mapping and visualising the signal coverage of [The Things Network](https://thethingsnetwork.org/).
## Structure
The system is split up into 2 primary parts:
1. An Arduino program that collects the data.
2. A Node.js program that stores and analyses the data
It's best explained with the aid of a diagram or two:
![](images/Manual Diagrams-Workflow.png)
The above flowchart describes the workflow when using the system:
1. First, the IoT device is turned on and the TTN Listener is launched.
2. Then, the IoT device is taken around the target area that needs mapping. This can be done by anyone - the device does not require operation beyond turning it on and off once provisioned.
3. Once the device has been carried around the target area, the `DATA.TSV` file on the IoT device's microSD card is copied off and placed on the server.
4. The server data processor is then run to fold the data into the database.
5. The AI trainer is now run on the collected data
6. The data is displayed in a web browser, with the help of a web server
The flow of data during use can be shown using a diagram:
![](images/Manual Diagrams-Data Flow.png)
## System requirements
This software has a number of requirements in order to function properly:
- A Linux-based server (_Ubuntu Server LTS_ is recommended), with the following installed:
+ Node.js v10+ with npm 6+ - see [this website](https://github.com/nodesource/distributions) for instructions on how to install a recent version of Node.js (for the various application server stuff)
+ Git (for cloning the code repository)
+ `awk` (for text processing by the build script)
+ Bash v4+ (for the build script)
+ PHP (for the temporary test web server; not otherwise used any static web server will do)
- Arduino IDE (for programming the IoT device)
- Git (for cloning the code repository)
-
## Getting Started
Run `bash ./build setup` from the command line at the root of this repository.
### IoT Device
- Copy `settings.custom.cpp.example` to `settings.custom.cpp` and fill in the fields there
- This Bash one-liner can be used to generate a new encryption key in the right formats: `head -c16 /dev/urandom | od -tx1 | awk '{ gsub(/^0+\s+/, "", $0); toml=$0; gsub(/\s+/, "", toml); print("settings.toml format: " toml); arduino=toupper($0); gsub(/\s+/, ", 0x", arduino); print("settings.custom.cpp format: 0x" arduino); exit }'`
- Review `settings.h` to make sure it matches your setup
- Copy the folders in `iot/libraries` to your Arduino IDE libraries folder
### Step 0: Initial setup
Before doing anything, clone this git repository to both the server (for the TTN listener etc.) and your local machine (for programming the IoT device):
```bash
git clone https://git.starbeamrainbowlabs.com/sbrl/Msc-Summer-Project.git
```
_If you have somehow obtained a static copy of the code (e.g. through the University's marking system), then skip the above and use that instead._
Next, `cd` to the root of the repository and then run the setup task of the build script:
```bash
./build setup
```
On Windows, this should be run in _Git Bash_ (accessible from the start menu when _Git_ is installed - don't forget to `cd` to the root of the repository).
### Step 1: Build a device
First, a device must be built and provisioned. See `HARDWARE.md` in this repository for detailed instructions on how to do this.
Once built, copy `iot/main/settings.custom.cpp.example` to `iot/main/settings.custom.cpp` and follow the instructions fill in the fields there. To do this, you'll need to register the device using ABP (Activation By Personalisation) on _The Things Network_. [This guide](https://www.thethingsnetwork.org/docs/devices/registration.html) tells you how to do this.
It is suggested that the following Bash one-liner be used to generate a new encryption key in the right formats:
```bash
head -c16 /dev/urandom | od -tx1 | awk '{ gsub(/^0+\s+/, "", $0); toml=$0; gsub(/\s+/, "", toml); print("settings.toml format: " toml); arduino=toupper($0); gsub(/\s+/, ", 0x", arduino); print("settings.custom.cpp format: 0x" arduino); exit }'
```
_(Paste it into a terminal and hit enter)_
Then, review `iot/main/settings.h` to make sure it matches your setup (e.g. all the pin numbers are correct)
Next, copy the folders in `iot/libraries` to your Arduino IDE libraries folder.
Finally, open `iot/main/main.ino` in the Arduino IDE and program the IoT device itself.
### Step 2: The Things Network Setup
_This step should be completed on the server._
Once a device has been constructed, running the _The Things Network_ listener is next. This requires giving the system the _The Things Network_ credentials.
Edit the file called `settings.toml` in the root of this repository (or create it if it doesn't exist), and make sure it contains the following:
```toml
[ttn]
app_id = "{APPLICATION_ID}"
access_key = "{TTN_ACCESS_ID}"
encryption_key = "{ENCRYPTION_KEY_FROM_STEP_1}"
port = 8883
tls = true
devices = [ "{DEVICE_NAME_FROM_TTN}" ]
```
The `app_id` and `access_key` can be obtained from _The Things Network Console_:
![](images/TTN-Main.png)
The device name can be obtained from step #1, when you registered the device with _The Things Network_. Alternatively, if the device is already registered, it can be obtained from the device list if you click the "X registered devices" text.
With `settings.toml` filled in, you can now start TTN Listener:
```bash
./build ttn-listener
```
....it will display an error message if you forgot a value.
Now that the TTN listener is running, the IoT device can be carried around and data collected.
### Step 3: Processing the data
_This step should be completed on the server._
Once data has been collected by the IoT device, it can then be processed by the Node.js server application.
Copy the `DATA.TSV` file from the IoT device's microSD card to the root of the repository on the server. It is suggested that `scp` (Linux) or [WinSCP](https://winscp.net/) be used to do this.
Next, fold `DATA.TSV` into the database like so:
```bash
./build process-data
```
Then, train the AIs on the collected data like this:
```bash
./build train-ai
```
The architecture of the neural networks trained can be customised by editing `settings.toml`. Check `server/settings.default.toml` for a guide on the different options available.
### Server
- The `bash ./build setup` should have already installed the necessary dependencies.
- Create `settings.toml` in the root of the repository if it doesn't exist already.
- Review `server/settings.default.toml` and adjust `settings.toml` to fill in the values for the required fields, not forgetting the encryption key generated when setting up the IoT device.
### Step 4: Viewing the AI output
With the AIs trained, the browser-based web interface can be used to display the output as a map. Any static web server will do, but the build script has one built-in (if PHP is installed) for convenience:
### Client-side web interface
```bash
./build server
```
If an alternative server is to be used, it should serve the `app/` directory that can be found in the root of this repository.
## Credits
- AI: [TensorFlow.js](https://www.tensorflow.org/js/)

18
build

@ -197,9 +197,14 @@ task_server() {
fi
task_begin "Starting development server";
php -S "[::1]:40382" -t "app" &
exit_code=$?;
[[ "${exit_code}" -eq "0" ]] && echo $! >/tmp/summer-project-dev-server.pid;
php -S "[::]:40382" -t "app" &
exit_code_a=$?;
pid_a=$!;
php -S "127.0.0.1:40382" -t "app" &
exit_code_b=$?;
pid_b=$!;
[[ "${exit_code_a}" -eq "0" ]] && echo "${pid_a}" >/tmp/summer-project-dev-server-v6.pid;
[[ "${exit_code_b}" -eq "0" ]] && echo "${pid_b}" >/tmp/summer-project-dev-server-v4.pid;
task_end $?; # Should be 0 unless php died for some reason
sleep 1;
}
@ -207,13 +212,14 @@ task_server() {
task_server-stop() {
task_begin "Stopping development server";
if [ ! -f "/tmp/summer-project-dev-server.pid" ]; then
if [ ! -f "/tmp/summer-project-dev-server-v6.pid" ]; then
echo -e "${HC}${FRED}Error: The development server doesn't appear to be running, so it can't be stopped. Have you tried running ./build dev-server?${RESET}";
return 1;
fi
kill "$(cat /tmp/summer-project-dev-server.pid)";
rm /tmp/summer-project-dev-server.pid;
kill "$(cat /tmp/summer-project-dev-server-v4.pid)";
kill "$(cat /tmp/summer-project-dev-server-v6.pid)";
rm /tmp/summer-project-dev-server-v[46].pid;
task_end $?;
}

0
Diagrams/Manual Diagrams.drawio → images/Manual Diagrams.drawio

0
Diagrams/Signal Mapping Architecture.drawio → images/Signal Mapping Architecture.drawio

5
server/settings.default.toml

@ -1,6 +1,9 @@
# Default settings file.
#
# DO NOT EDIT THIS FILE. Instead edit ../settings.toml (or create it if it doesn't exist yet).
# **********************************************************
# ***************** DO NOT EDIT THIS FILE. *****************
# **********************************************************
# Instead edit ../settings.toml (or create it if it doesn't exist yet).
program_name = "LoRaWAN Signal Mapper"
version = "v0.1"

Loading…
Cancel
Save