Grafana and InfluxDB on a Pi

Instructions as of late 2019! I used a Pi Zero W. This works but was pretty slow and awful for a my application. I then tried with a RPi 4B 2Gb and a RPi 3B.

Preliminary

From linux docs or see my notes. I used the latest Raspian Lite (“Buster”).

InfluxDB

Follow Buster instructions with apt at https://pimylifeup.com/raspberry-pi-influxdb/, or

# Manual Install (not recommended) of 1.8.3 (last v1)
wget https://dl.influxdata.com/influxdb/releases/influxdb-1.8.3_linux_armhf.tar.gz
tar xvfz influxdb-1.8.3_linux_armhf.tar.gz
cd influxdb-1.8.3-1
sudo cp -rp usr/* /usr ; sudo cp -rp etc/* /etc ; sudo cp -rp var/* /var
sudo cp /usr/lib/influxdb/scripts/influxdb.service  /etc/systemd/system
sudo systemctl unmask influxdb.service
sudo systemctl enable influxdb
sudo systemctl start influxdb

Note the shell is SLLLOOOWWW to startup on a Zero. Need more power and memory. <time passes, money spent>. The Pi 4B is faster at about 10s. Even on a real PC it’s slow, so this is ok.

Grafana

See the download page. TL>DR below.

Pi Zero

# Package libfontconfig1 is not installed if you install grafana now
apt --fix-broken install
wget https://dl.grafana.com/oss/release/grafana-rpi_6.4.4_armhf.deb
sudo dpkg -i grafana-rpi_6.4.4_armhf.deb

Pi 4b

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
# This fails:
#sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
#sudo apt-get install -y apt-transport-https
#sudo apt-get install -y software-properties-common wget
# ... so do this
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list 
sudo apt-get update
sudo apt-get install grafana
# Installed, now setup
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server
sudo /bin/systemctl start grafana-server

Extending Life

RPi setup to minimise writes. There are many options.

sudo apt-get remove dphys-swapfile # no swappy
# follow log2ram directions, set /etc/log2ram.conf to 100M and reboot

Selected Options

On the 4b, I used

sudo iotop -Pa -d0.5

… to monitor disk usage and just played around until I go fewest writes/runs:

Don’t change the

Configure Influx

Edit influxdb.conf and try to limit SD card hits:

[http]
log-enabled = false
# suppress-write-log = false # option: log-enabled but only reads logs
[data]
 query-log-enabled = false
[logging]
level = "warn"
[monitor]
store-enabled = false 
# store-interval = "60s"
[coordinator]
# Optional. Might improve stability
 max-concurrent-queries = 100
 query-timeout = "20s"
 max-select-point = 10000000
[continuous_queries]
   log-enabled = false
   run-interval = "1m"

Disable Journaling

See here on this risky move:

sudo -i 
echo u > /proc/sysrq-trigger  
echo s > /proc/sysrq-trigger  
tune2fs -O ^has_journal /dev/mmcblk0p2 
e2fsck -fy /dev/mmcblk0p2 
echo s > /proc/sysrq-trigger  
echo b > /proc/sysrq-trigger
# and now check it:
sudo debugfs -R features /dev/mmcblk0p2

Other options

Mounting External Store (USB thumb drive)

I considered putting the database files on a separate USB drive, and run the OS from the SD card. SSD are too bulky. Apparently USB flash drives are the same, more or less, as SD cards. Having separate data means I spread the load and can replace the OS without losing data. Instructions FTFM here.

However, with a 32Gb card I’ll try without a separate drive first, as it is simpler.

Configuring PhiSaver

The name pi machine with influx should be called “phisaver”
iotawatt should be called UNITNAME so that device=unitname
change passwords
- iotawatt: via gui or config.txt
- grafana: add a new user and disable admin
- influx: via cli
You can run phisaver-* setup scripts from another machine, so you don’t need to install anything. But if ya wanna:

Installing PhiSaver Setup Scripts (optional)

# Get (private) repo
sudo apt-get install git
git install https://github.com/brettbeeson/phisaver.git
# Use Python3 
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1
sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.7 2
# Get requirements
sudo apt-get -y install python3-pip
pip3 install -r requirements.txt
# sudo apt-get -y install php # big install due to dependancies. Only required for auto-login and could be a standalone install
sudo systemctl disable apache2 # if php installed it

Then setup the databases and grafana:

# on a pc with phisaver scripts:
phi-grafana-setup-user.py http://localhost:3000 UNITNAME dashboards/phisaver-1-display.json dashboards/phisaver-1-analysis.json 
scp server-config/grafana.local.ini phisaver.local:grafana.ini # template

# Then on the pi (i.e. locally)
sudo grafana-cli plugins install grafana-piechart-panel
# sudo mv grafana.ini /etc/grafana/grafana.ini # and/or
sudo vim /etc/grafana/grafana.ini # and check setup/org/permissions/owner
sudo setcap 'cap_net_bind_service=+ep' /usr/sbin/grafana-server # bind to :80
sudo systemctl restart grafana-server

Optionally, setup a tunnel:

# pc
scp create-ssh-tunnel.sh pi@raspberrypi.local:.
# warning: keys to the kingdom
scp ~/.ssh/id_rsa pi@raspberrypi.local:.
# pi: add server-config/cron.list to crontab
mv id_rsa .ssh/
crontab -e

Other Things to Automate for PhiSaver on Pi

Sample at 10s and update auto-refresh for this. Then could resample data to 1m intervals for a year.
Grafana install: anony login to grafana for local user, pie-chart plugin, port 80, anony user to UNITNAME
Iotawatt: post to phisaver.local instead of live2.phisaver.com
ssh tunnel