curious.bio/planktoscope
curious.bio/planktoscope
9
0
Fork 0
Code Issues 10 Pull requests Packages Projects 7 Releases Wiki Activity

Expert setup update and update script split

Browse source
This commit is contained in:
Romain Bazile 2021-07-01 15:56:15 +02:00
parent f316796e99
commit 7d02dfa27f
3 changed files with 91 additions and 69 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

119
docs/expert_setup.md
View file

@ -53,7 +53,7 @@ First of all, and to ensure you have the latest documentation available locally,
Simply run the following in a terminal: Simply run the following in a terminal:
```sh ```sh
git clone https://github.com/PlanktonPlanet/PlanktonScope/ git clone https://github.com/PlanktonPlanet/PlanktoScope/
``` ```
### Enable Camera/SSH/I2C in raspi-config ### Enable Camera/SSH/I2C in raspi-config
@ -63,26 +63,38 @@ You can now launch the configuration tool:
sudo raspi-config sudo raspi-config
``` ```
While you're here, a wise thing to do would be to change the default password for the `pi` user. This is very warmly recommended if your PlanktoScope is connected to a shared network you do not control. Just select the first option `1 Change User Password`. While you're here, a wise thing to do would be to change the default password for the `pi` user. This is very warmly recommended if your PlanktoScope is connected to a shared network you do not control. Just select the first option `1 System Options`, the `S3 Password`.
You may also want to change the default hostname of your Raspberry. To do so, choose option `2 Network Options` then `N1 Hostname`. Choose a new hostname. We recommend using `planktoscope`. You may also want to change the default hostname of your Raspberry. To do so, choose option `1 System Options` then `S4 Hostname`. Choose a new hostname. We recommend using `planktoscope` as this name will then appear.
We need to activate a few things for the PlanktoScope to work properly. We need to activate a few things for the PlanktoScope to work properly.
First, we need to activate the camera interface. Choose `5 Interfacing Options`, then `P1 Camera` and `Yes`. First, we need to activate the camera interface. Choose `3 Interface Options`, then `P1 Camera` and `Yes`.
Now, you can go to `5 Interfacing Options`, then `P2 SSH`. Choose `Yes` to activate the SSH access. Now, you can go to `3 Interface Options`, then `P2 SSH`. Choose `Yes` to activate the SSH access.
Again, select `5 Interfacing Options`, then `P4 SPI`. Choose `Yes` to enable the SPI interface. Again, select `3 Interface Options`, then `P4 SPI`. Choose `Yes` to enable the SPI interface.
One more, select `5 Interfacing Options`, then `P5 I2C`. Choose `Yes` to enable the ARM I2C interface of the Raspberry. One more, select `3 Interface Options`, then `P5 I2C`. Choose `Yes` to enable the ARM I2C interface of the Raspberry.
Finally, select `5 Interfacing Options`, then `P6 Serial`. Finally, select `3 Interface Options`, then `P6 Serial`.
This time, choose `No` to deactivate the login shell on the serial connection, but then choose `Yes` to keep the Serial port hardware enabled. This time, choose `No` to deactivate the login shell on the serial connection, but then choose `Yes` to keep the Serial port hardware enabled.
Last steps we need to do is to increase the amount of memory available to the GPU. Select `4 Performance Options`, then `P2 GPU Memory`. Write `256` in the field and choose OK.
These steps can also be done from the Raspberry Pi Configuration GUI tool that you can find in `Main Menu > Preferences`. Go to the `Interfaces` tab. Pay attention, here the Serial Port must be enabled, but the Serial Port Console must be disabled. These steps can also be done from the Raspberry Pi Configuration GUI tool that you can find in `Main Menu > Preferences`. Go to the `Interfaces` tab. Pay attention, here the Serial Port must be enabled, but the Serial Port Console must be disabled.
!!! Special optionnal step: overclocking
We are first going to make sure that your PlanktoScope receives proper PPS signal. We need to add the following line at the end of `/boot/config.txt`. Open the file with `sudo nano /boot/config.txt` and add the following at the end:
```
# Pi overclock
over_voltage=6
arm_freq=2000
```
Those settings were verified to be stable, but if you notice any weird behavior under a high load, remove those lines.
Reboot your Pi safely. Reboot your Pi safely.
```sh ```sh
sudo reboot now sudo reboot now
@ -101,16 +113,15 @@ sudo apt install build-essential python3 python3-pip
mkdir test libraries mkdir test libraries
``` ```
### Install CircuitPython ### Install all python libraries
Start by following [Adafruit's guide](https://learn.adafruit.com/circuitpython-on-raspberrypi-linux/installing-circuitpython-on-raspberry-pi). You can start at the chapter `Install Python Libraries`. To simplify setup, we provide requirements.txt:
For the record, the command are as following, however, Adafruit's page might have been updated, so please make sure this is still needed:
```sh ```sh
sudo pip3 install RPI.GPIO pip3 install -U -r /home/pi/PlanktoScope/requirements.txt
sudo pip3 install adafruit-blinka
sudo pip3 install adafruit-circuitpython-motorkit
``` ```
### Check CircuitPython's install
Start by following [Adafruit's guide](https://learn.adafruit.com/circuitpython-on-raspberrypi-linux/installing-circuitpython-on-raspberry-pi). You can start at the chapter `Install Python Libraries`.
#### Testing the installation and the wiring #### Testing the installation and the wiring
It is recommended to test this setup by creating this small script under the name `test/blinkatest.py` and running it (you can use the editor nano if you are using the terminal). If you are using the image provided, you may find that the script is already there. It is recommended to test this setup by creating this small script under the name `test/blinkatest.py` and running it (you can use the editor nano if you are using the terminal). If you are using the image provided, you may find that the script is already there.
@ -266,6 +277,8 @@ pi@planktoscope:~ $ gpsmon
(72) $GPRMC,110924.000,A,4533.4809,N,00103.7367,W,0.01,156.23,210720,,,D*71 (72) $GPRMC,110924.000,A,4533.4809,N,00103.7367,W,0.01,156.23,210720,,,D*71
``` ```
You can leave with `CTRL+C`.
#### Bonus Configuration: Automatic time update from GPSD #### Bonus Configuration: Automatic time update from GPSD
The Adafruit GPS HAT allows your PlanktoScope to automatically sets its time to the GPS received one. Moreover, since the PPS (Pulse Per Second) output is activated, you can even set your PlanktoScope to act as a stratum 1 timeserver. The Adafruit GPS HAT allows your PlanktoScope to automatically sets its time to the GPS received one. Moreover, since the PPS (Pulse Per Second) output is activated, you can even set your PlanktoScope to act as a stratum 1 timeserver.
@ -402,7 +415,6 @@ Raspberry Pi Details:
You will also need to install some python modules: You will also need to install some python modules:
```sh ```sh
sudo apt install python3-smbus i2c-tools sudo apt install python3-smbus i2c-tools
sudo pip3 install Adafruit-SSD1306
``` ```
More information can be found on Yahboom website, on the page [Installing RGB Cooling HAT](https://www.yahboom.net/study/RGB_Cooling_HAT). More information can be found on Yahboom website, on the page [Installing RGB Cooling HAT](https://www.yahboom.net/study/RGB_Cooling_HAT).
@ -416,50 +428,33 @@ sudo apt install mosquitto mosquitto-clients
``` ```
### Install mqtt-paho ### Check OpenCV's installation
In order to send and receive data from python, you need this library. Run the following:
```
sudo pip3 install paho-mqtt
```
### Install OpenCV
We need to install the latest OpenCV version. Unfortunately, it is not available in the repositories. We are going to install it directly by using pip. We need to install the latest OpenCV version. Unfortunately, it is not available in the repositories. We are going to install it directly by using pip.
First, we need to install the needed dependencies, then we will directly install opencv: First, we need to install the needed dependencies, then we will directly install opencv:
```sh ```sh
sudo apt install libgtk-3-0 libavformat58 libtiff5 libcairo2 libqt4-test libpango-1.0-0 libopenexr23 libavcodec58 libilmbase23 libatk1.0-0 libpangocairo-1.0-0 libwebp6 libqtgui4 libavutil56 libjasper1 libqtcore4 libcairo-gobject2 libswscale5 libgdk-pixbuf2.0-0 libhdf5-dev libilmbase-dev libopenexr-dev libgstreamer1.0-dev libavcodec-dev libavformat-dev libswscale-dev libwebp-dev libatlas-base-dev sudo apt install libgtk-3-0 libavformat58 libavcodec58 libqt4-test libopenexr23 libilmbase23 libqtgui4 libavutil56 libjasper1 libqtcore4 libcairo-gobject2 libswscale5 libhdf5-dev libilmbase-dev libopenexr-dev libgstreamer1.0-dev libavcodec-dev libavformat-dev libswscale-dev libwebp-dev libatlas-base-dev
sudo pip3 install "picamera[array]"
sudo pip3 install opencv-contrib-python==4.1.0.25
``` ```
You can now check that opencv is properly installed by running a python interpreter and importing the cv2 module. You can now check that opencv is properly installed by running a python interpreter and importing the cv2 module.
```sh ```sh
pi@planktoscope:~ $ python3 pi@planktoscope:~ $ python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59) Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux [GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information. Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2 >>> import cv2
>>> cv2.__version__ >>> cv2.__version__
'4.1.0' '4.4.0'
>>> quit() >>> quit()
``` ```
If all goes well, the displayed version number should be `4.1.0`. If all goes well, the displayed version number should be `4.4.0`.
More detailed information can be found on this [website](https://www.pyimagesearch.com/2019/09/16/install-opencv-4-on-raspberry-pi-4-and-raspbian-buster/). More detailed information can be found on this [website](https://www.pyimagesearch.com/2019/09/16/install-opencv-4-on-raspberry-pi-4-and-raspbian-buster/).
### Install MorphoCut ### Check MorphoCut's installation
MorphoCut is packaged on PyPI and can be installed with pip:
```sh
sudo apt install python3-scipy
sudo pip3 install -U git+https://github.com/morphocut/morphocut.git
```
To test the installation, open up once again a python interpreter and import the morphocut module: To test the installation, open up once again a python interpreter and import the morphocut module:
```sh ```sh
@ -476,6 +471,16 @@ Type "help", "copyright", "credits" or "license" for more information.
The MorphoCut documentation can be found [on this page](https://morphocut.readthedocs.io/en/stable/index.html). The MorphoCut documentation can be found [on this page](https://morphocut.readthedocs.io/en/stable/index.html).
### Nginx Setup
```
sudo apt install nginx
sudo rm /etc/nginx/sites-enabled/default
sudo ln -s /home/pi/PlanktoScope/scripts/gallery/gallery.conf /etc/nginx/sites-enabled/gallery.conf
sudo nginx -t && sudo systemctl reload nginx
```
### Install Node-RED ### Install Node-RED
#### Download and installation #### Download and installation
@ -499,24 +504,34 @@ sudo systemctl start nodered.service
#### Check the installation #### Check the installation
Make sure Node-RED is correctly installed by reaching the following page from the browser of your pi http://localhost:1880 or http://planktoscope.local:1880 from another computer on the same network. Make sure Node-RED is correctly installed by reaching the following page from the browser of your pi http://localhost:1880 or http://planktoscope.local:1880 from another computer on the same network.
#### Install the necessary nodes #### Install the necessary nodes and activate necessary features
These nodes will be used by the PlanktoScope software and needs to be installed:
We are going to activate the Projects feature of Node-Red as this will help us manage and track changes to the flows. Open the file `settings.js` with an editor (for example with `nano ~/.node-red/settings.js`) so we can change the following lines (you can use `CTRL+_` to quickly navigate to the line indicated):
- Line 75: uncomment the line (remove the //) that ends with flowFilePretty: true,
- Line 337: set enabled to true
Restart Node-RED to take into account those changes:
```sh ```sh
cd ~/.node-red/ sudo systemctl restart nodered.service
npm install node-red-dashboard node-red-contrib-python3-function node-red-contrib-camerapi node-red-contrib-gpsd node-red-contrib-web-worldmap node-red-contrib-interval
``` ```
We are also going to activate the Projects feature of Node-Red as this will help us manage and track changes to the flows. Open the file `settings.js` with an editor (for example with `nano settings.js`) so we can change the following lines:
We need to move the PlanktoScope folder in the right place, in the `projects` subfolder of Node-Red and link this new folder to our `/home/`. To do so, in the terminal type the following::
```sh
mv /home/pi/PlanktoScope /home/pi/.node-red/projects/
ln -s /home/pi/.node-red/projects/PlanktoScope /home/pi/PlanktoScope
``` ```
Line 68: uncomment the line (remove the //) that ends with flowFilePretty: true,
Line 296: set enabled to true We will now install the missing nodes. These nodes will be used by the PlanktoScope software:
```sh
cd /home/pi/.node-red/
npm install copy-dependencies
node_modules/copy-dependencies/index.js projects/PlanktoScope ./
npm update
``` ```
Save you changes. Save you changes.
The final step before restarting node-red is to link the projects directory from within node-red folder to our main home directory. To do so, just open a terminal and type the following: The final step before restarting node-red is to link the projects directory from within node-red folder to our main home directory. To do so, just open a terminal and type the following:
```bash
ln -s /home/pi/.node-red/projects/PlanktonScope /home/pi/PlanktonScope
```
You can now restart the nodered service: You can now restart the nodered service:
``` ```
@ -527,9 +542,9 @@ sudo systemctl restart nodered.service
If you now open the Node-Red GUI in your browser, it will ask you to setup the project, an email and a username (so if you make changes to the flow and want to share them we can know who made them). If you now open the Node-Red GUI in your browser, it will ask you to setup the project, an email and a username (so if you make changes to the flow and want to share them we can know who made them).
You can now choose to clone an existing repository. Choose a name that makes sense for you, and in the `Git repository URL` field put the main Planktonscope repository: `https://www.github.com/PlanktonPlanet/PlanktonScope.git`. Open your browser and navigate to http://planktoscope.local:1880. In the prompt, select `Open existing project` button at the bottom, choose the PlanktoScope project and click on `Open Project`. Eventually, merge the proposed changes.
The latest flow version will be imported immediately. The latest flow version will be available immediately.
#### More information #### More information
@ -556,7 +571,7 @@ sudo reboot now
Updates are published on Github regurlarly. Make sure to update once in a while by running this command: Updates are published on Github regurlarly. Make sure to update once in a while by running this command:
```sh ```sh
cd PlanktonScope cd PlanktoScope
git pull git pull
``` ```

19
scripts/bash/start_update.sh Normal file
View file

@ -0,0 +1,19 @@
#!/bin/bash
log="echo -e"
${log} "Updating the installer script from $BRANCH"
REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktoScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}')
if [[ $# == 1 ]]; then
if [[ $REMOTE_BRANCHES =~ (^| )$1($| ) ]]; then
BRANCH=$1
else
BRANCH="master"
fi
else
BRANCH="master"
fi
curl "https://raw.githubusercontent.com/PlanktonPlanet/PlanktoScope/$BRANCH/scripts/bash/update.sh" > /tmp/update.sh
exec /tmp/update.sh "$BRANCH"

22
scripts/bash/update.sh
View file

@ -1,7 +1,7 @@
#!/bin/bash #!/bin/bash
log="echo -e" log="echo -e"
REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktonScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}') REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktoScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}')
if [[ $# == 1 ]]; then if [[ $# == 1 ]]; then
if [[ $REMOTE_BRANCHES =~ (^| )$1($| ) ]]; then if [[ $REMOTE_BRANCHES =~ (^| )$1($| ) ]]; then
BRANCH=$1 BRANCH=$1
@ -14,24 +14,13 @@ fi
${log} "Updating the main repository to branch $BRANCH" ${log} "Updating the main repository to branch $BRANCH"
function auto_update(){
git fetch
NEWVERSION=$(git diff --numstat origin/$BRANCH scripts/bash/update.sh | awk '/update.sh/ {print $NF}')
if [[ -n "${NEWVERSION}" ]]; then
${log} "Updating the update script first"
# Update the file and restart the script
git checkout origin/$BRANCH scripts/bash/update.sh
exec scripts/bash/update.sh $BRANCH
fi
}
function restart(){ function restart(){
sudo nginx -t && sudo systemctl reload nginx sudo nginx -t && sudo systemctl reload nginx
sudo systemctl restart nodered.service sudo systemctl restart nodered.service
} }
function update(){ function update(){
cd /home/pi/PlanktonScope || { echo "/home/pi/PlanktonScope does not exist"; exit 1; } cd /home/pi/PlanktoScope || { echo "/home/pi/PlanktoScope does not exist"; exit 1; }
sudo killall -15 raspimjpeg sudo killall -15 raspimjpeg
sudo killall -15 python3 sudo killall -15 python3
git stash git stash
@ -56,23 +45,22 @@ function special_before(){
function special_after(){ function special_after(){
cd /home/pi/.node-red || { echo "/home/pi/.node-red does not exist"; exit 1; } cd /home/pi/.node-red || { echo "/home/pi/.node-red does not exist"; exit 1; }
node_modules/copy-dependencies/index.js projects/PlanktonScope ./ node_modules/copy-dependencies/index.js projects/PlanktoScope ./
# updating and installing now whatever module has been added to package.json # updating and installing now whatever module has been added to package.json
npm update npm update
sudo pip3 install -U -r /home/pi/PlanktonScope/requirements.txt pip3 install -U -r /home/pi/PlanktoScope/requirements.txt
python3 -m pip install -U scikit-image python3 -m pip install -U scikit-image
${log} "Nothing else special to do after updating!" ${log} "Nothing else special to do after updating!"
} }
echo -e "Update on $(date)\n\n\n\n" >> /home/pi/update.log echo -e "Update on $(date)\n\n\n\n" >> /home/pi/update.log
cd /home/pi/PlanktonScope || { echo "/home/pi/PlanktonScope does not exist"; exit 1; } cd /home/pi/PlanktoScope || { echo "/home/pi/PlanktoScope does not exist"; exit 1; }
remote=$(git ls-remote -h origin $BRANCH | awk '{print $1}') remote=$(git ls-remote -h origin $BRANCH | awk '{print $1}')
local=$(git rev-parse HEAD) local=$(git rev-parse HEAD)
if [[ "$local" == "$remote" ]]; then if [[ "$local" == "$remote" ]]; then
${log} "Nothing to do!" ${log} "Nothing to do!"
else else
${log} "Local and Remote are different, we have to update, starting now... Please Wait." ${log} "Local and Remote are different, we have to update, starting now... Please Wait."
auto_update &>> /home/pi/update.log
special_before &>> /home/pi/update.log special_before &>> /home/pi/update.log
${log} "Everything is ready, doing the update now!" ${log} "Everything is ready, doing the update now!"
update &>> /home/pi/update.log update &>> /home/pi/update.log