From 7d02dfa27f39a1819e86ac3a1cc325866f7ffa8d Mon Sep 17 00:00:00 2001 From: Romain Bazile Date: Thu, 1 Jul 2021 15:56:15 +0200 Subject: [PATCH] Expert setup update and update script split --- docs/expert_setup.md | 119 ++++++++++++++++++++--------------- scripts/bash/start_update.sh | 19 ++++++ scripts/bash/update.sh | 22 ++----- 3 files changed, 91 insertions(+), 69 deletions(-) create mode 100644 scripts/bash/start_update.sh diff --git a/docs/expert_setup.md b/docs/expert_setup.md index 4369e5b..3039ec8 100644 --- a/docs/expert_setup.md +++ b/docs/expert_setup.md @@ -53,7 +53,7 @@ First of all, and to ensure you have the latest documentation available locally, Simply run the following in a terminal: ```sh -git clone https://github.com/PlanktonPlanet/PlanktonScope/ +git clone https://github.com/PlanktonPlanet/PlanktoScope/ ``` ### Enable Camera/SSH/I2C in raspi-config @@ -63,26 +63,38 @@ You can now launch the configuration tool: sudo raspi-config ``` -While you're here, a wise thing to do would be to change the default password for the `pi` user. This is very warmly recommended if your PlanktoScope is connected to a shared network you do not control. Just select the first option `1 Change User Password`. +While you're here, a wise thing to do would be to change the default password for the `pi` user. This is very warmly recommended if your PlanktoScope is connected to a shared network you do not control. Just select the first option `1 System Options`, the `S3 Password`. -You may also want to change the default hostname of your Raspberry. To do so, choose option `2 Network Options` then `N1 Hostname`. Choose a new hostname. We recommend using `planktoscope`. +You may also want to change the default hostname of your Raspberry. To do so, choose option `1 System Options` then `S4 Hostname`. Choose a new hostname. We recommend using `planktoscope` as this name will then appear. We need to activate a few things for the PlanktoScope to work properly. -First, we need to activate the camera interface. Choose `5 Interfacing Options`, then `P1 Camera` and `Yes`. +First, we need to activate the camera interface. Choose `3 Interface Options`, then `P1 Camera` and `Yes`. -Now, you can go to `5 Interfacing Options`, then `P2 SSH`. Choose `Yes` to activate the SSH access. +Now, you can go to `3 Interface Options`, then `P2 SSH`. Choose `Yes` to activate the SSH access. -Again, select `5 Interfacing Options`, then `P4 SPI`. Choose `Yes` to enable the SPI interface. +Again, select `3 Interface Options`, then `P4 SPI`. Choose `Yes` to enable the SPI interface. -One more, select `5 Interfacing Options`, then `P5 I2C`. Choose `Yes` to enable the ARM I2C interface of the Raspberry. +One more, select `3 Interface Options`, then `P5 I2C`. Choose `Yes` to enable the ARM I2C interface of the Raspberry. -Finally, select `5 Interfacing Options`, then `P6 Serial`. +Finally, select `3 Interface Options`, then `P6 Serial`. This time, choose `No` to deactivate the login shell on the serial connection, but then choose `Yes` to keep the Serial port hardware enabled. +Last steps we need to do is to increase the amount of memory available to the GPU. Select `4 Performance Options`, then `P2 GPU Memory`. Write `256` in the field and choose OK. + These steps can also be done from the Raspberry Pi Configuration GUI tool that you can find in `Main Menu > Preferences`. Go to the `Interfaces` tab. Pay attention, here the Serial Port must be enabled, but the Serial Port Console must be disabled. +!!! Special optionnal step: overclocking + We are first going to make sure that your PlanktoScope receives proper PPS signal. We need to add the following line at the end of `/boot/config.txt`. Open the file with `sudo nano /boot/config.txt` and add the following at the end: + ``` + # Pi overclock + over_voltage=6 + arm_freq=2000 + ``` + Those settings were verified to be stable, but if you notice any weird behavior under a high load, remove those lines. + + Reboot your Pi safely. ```sh sudo reboot now @@ -101,16 +113,15 @@ sudo apt install build-essential python3 python3-pip mkdir test libraries ``` -### Install CircuitPython -Start by following [Adafruit's guide](https://learn.adafruit.com/circuitpython-on-raspberrypi-linux/installing-circuitpython-on-raspberry-pi). You can start at the chapter `Install Python Libraries`. - -For the record, the command are as following, however, Adafruit's page might have been updated, so please make sure this is still needed: +### Install all python libraries +To simplify setup, we provide requirements.txt: ```sh -sudo pip3 install RPI.GPIO -sudo pip3 install adafruit-blinka -sudo pip3 install adafruit-circuitpython-motorkit +pip3 install -U -r /home/pi/PlanktoScope/requirements.txt ``` +### Check CircuitPython's install +Start by following [Adafruit's guide](https://learn.adafruit.com/circuitpython-on-raspberrypi-linux/installing-circuitpython-on-raspberry-pi). You can start at the chapter `Install Python Libraries`. + #### Testing the installation and the wiring It is recommended to test this setup by creating this small script under the name `test/blinkatest.py` and running it (you can use the editor nano if you are using the terminal). If you are using the image provided, you may find that the script is already there. @@ -266,6 +277,8 @@ pi@planktoscope:~ $ gpsmon (72) $GPRMC,110924.000,A,4533.4809,N,00103.7367,W,0.01,156.23,210720,,,D*71 ``` +You can leave with `CTRL+C`. + #### Bonus Configuration: Automatic time update from GPSD The Adafruit GPS HAT allows your PlanktoScope to automatically sets its time to the GPS received one. Moreover, since the PPS (Pulse Per Second) output is activated, you can even set your PlanktoScope to act as a stratum 1 timeserver. @@ -402,7 +415,6 @@ Raspberry Pi Details: You will also need to install some python modules: ```sh sudo apt install python3-smbus i2c-tools -sudo pip3 install Adafruit-SSD1306 ``` More information can be found on Yahboom website, on the page [Installing RGB Cooling HAT](https://www.yahboom.net/study/RGB_Cooling_HAT). @@ -416,50 +428,33 @@ sudo apt install mosquitto mosquitto-clients ``` -### Install mqtt-paho - -In order to send and receive data from python, you need this library. Run the following: -``` -sudo pip3 install paho-mqtt -``` - - -### Install OpenCV +### Check OpenCV's installation We need to install the latest OpenCV version. Unfortunately, it is not available in the repositories. We are going to install it directly by using pip. First, we need to install the needed dependencies, then we will directly install opencv: ```sh -sudo apt install libgtk-3-0 libavformat58 libtiff5 libcairo2 libqt4-test libpango-1.0-0 libopenexr23 libavcodec58 libilmbase23 libatk1.0-0 libpangocairo-1.0-0 libwebp6 libqtgui4 libavutil56 libjasper1 libqtcore4 libcairo-gobject2 libswscale5 libgdk-pixbuf2.0-0 libhdf5-dev libilmbase-dev libopenexr-dev libgstreamer1.0-dev libavcodec-dev libavformat-dev libswscale-dev libwebp-dev libatlas-base-dev -sudo pip3 install "picamera[array]" -sudo pip3 install opencv-contrib-python==4.1.0.25 +sudo apt install libgtk-3-0 libavformat58 libavcodec58 libqt4-test libopenexr23 libilmbase23 libqtgui4 libavutil56 libjasper1 libqtcore4 libcairo-gobject2 libswscale5 libhdf5-dev libilmbase-dev libopenexr-dev libgstreamer1.0-dev libavcodec-dev libavformat-dev libswscale-dev libwebp-dev libatlas-base-dev ``` You can now check that opencv is properly installed by running a python interpreter and importing the cv2 module. ```sh pi@planktoscope:~ $ python3 -Python 3.7.3 (default, Dec 20 2019, 18:57:59) +Python 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import cv2 >>> cv2.__version__ -'4.1.0' +'4.4.0' >>> quit() ``` -If all goes well, the displayed version number should be `4.1.0`. +If all goes well, the displayed version number should be `4.4.0`. More detailed information can be found on this [website](https://www.pyimagesearch.com/2019/09/16/install-opencv-4-on-raspberry-pi-4-and-raspbian-buster/). -### Install MorphoCut - -MorphoCut is packaged on PyPI and can be installed with pip: - -```sh -sudo apt install python3-scipy -sudo pip3 install -U git+https://github.com/morphocut/morphocut.git -``` +### Check MorphoCut's installation To test the installation, open up once again a python interpreter and import the morphocut module: ```sh @@ -476,6 +471,16 @@ Type "help", "copyright", "credits" or "license" for more information. The MorphoCut documentation can be found [on this page](https://morphocut.readthedocs.io/en/stable/index.html). +### Nginx Setup + + +``` +sudo apt install nginx +sudo rm /etc/nginx/sites-enabled/default +sudo ln -s /home/pi/PlanktoScope/scripts/gallery/gallery.conf /etc/nginx/sites-enabled/gallery.conf +sudo nginx -t && sudo systemctl reload nginx +``` + ### Install Node-RED #### Download and installation @@ -499,24 +504,34 @@ sudo systemctl start nodered.service #### Check the installation Make sure Node-RED is correctly installed by reaching the following page from the browser of your pi http://localhost:1880 or http://planktoscope.local:1880 from another computer on the same network. -#### Install the necessary nodes -These nodes will be used by the PlanktoScope software and needs to be installed: +#### Install the necessary nodes and activate necessary features + +We are going to activate the Projects feature of Node-Red as this will help us manage and track changes to the flows. Open the file `settings.js` with an editor (for example with `nano ~/.node-red/settings.js`) so we can change the following lines (you can use `CTRL+_` to quickly navigate to the line indicated): + - Line 75: uncomment the line (remove the //) that ends with flowFilePretty: true, + - Line 337: set enabled to true + +Restart Node-RED to take into account those changes: ```sh -cd ~/.node-red/ -npm install node-red-dashboard node-red-contrib-python3-function node-red-contrib-camerapi node-red-contrib-gpsd node-red-contrib-web-worldmap node-red-contrib-interval +sudo systemctl restart nodered.service ``` -We are also going to activate the Projects feature of Node-Red as this will help us manage and track changes to the flows. Open the file `settings.js` with an editor (for example with `nano settings.js`) so we can change the following lines: + +We need to move the PlanktoScope folder in the right place, in the `projects` subfolder of Node-Red and link this new folder to our `/home/`. To do so, in the terminal type the following:: +```sh +mv /home/pi/PlanktoScope /home/pi/.node-red/projects/ +ln -s /home/pi/.node-red/projects/PlanktoScope /home/pi/PlanktoScope ``` -Line 68: uncomment the line (remove the //) that ends with flowFilePretty: true, -Line 296: set enabled to true + +We will now install the missing nodes. These nodes will be used by the PlanktoScope software: +```sh +cd /home/pi/.node-red/ +npm install copy-dependencies +node_modules/copy-dependencies/index.js projects/PlanktoScope ./ +npm update ``` Save you changes. The final step before restarting node-red is to link the projects directory from within node-red folder to our main home directory. To do so, just open a terminal and type the following: -```bash -ln -s /home/pi/.node-red/projects/PlanktonScope /home/pi/PlanktonScope -``` You can now restart the nodered service: ``` @@ -527,9 +542,9 @@ sudo systemctl restart nodered.service If you now open the Node-Red GUI in your browser, it will ask you to setup the project, an email and a username (so if you make changes to the flow and want to share them we can know who made them). -You can now choose to clone an existing repository. Choose a name that makes sense for you, and in the `Git repository URL` field put the main Planktonscope repository: `https://www.github.com/PlanktonPlanet/PlanktonScope.git`. +Open your browser and navigate to http://planktoscope.local:1880. In the prompt, select `Open existing project` button at the bottom, choose the PlanktoScope project and click on `Open Project`. Eventually, merge the proposed changes. -The latest flow version will be imported immediately. +The latest flow version will be available immediately. #### More information @@ -556,7 +571,7 @@ sudo reboot now Updates are published on Github regurlarly. Make sure to update once in a while by running this command: ```sh -cd PlanktonScope +cd PlanktoScope git pull ``` diff --git a/scripts/bash/start_update.sh b/scripts/bash/start_update.sh new file mode 100644 index 0000000..7e59185 --- /dev/null +++ b/scripts/bash/start_update.sh @@ -0,0 +1,19 @@ +#!/bin/bash +log="echo -e" + +${log} "Updating the installer script from $BRANCH" + +REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktoScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}') +if [[ $# == 1 ]]; then + if [[ $REMOTE_BRANCHES =~ (^| )$1($| ) ]]; then + BRANCH=$1 + else + BRANCH="master" + fi +else + BRANCH="master" +fi + +curl "https://raw.githubusercontent.com/PlanktonPlanet/PlanktoScope/$BRANCH/scripts/bash/update.sh" > /tmp/update.sh + +exec /tmp/update.sh "$BRANCH" diff --git a/scripts/bash/update.sh b/scripts/bash/update.sh index 4922655..6ce5369 100755 --- a/scripts/bash/update.sh +++ b/scripts/bash/update.sh @@ -1,7 +1,7 @@ #!/bin/bash log="echo -e" -REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktonScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}') +REMOTE_BRANCHES=$(git --git-dir=/home/pi/PlanktoScope/.git branch --remotes --list | awk '/HEAD/{next;} split($1, a, "/") {print a[2]}') if [[ $# == 1 ]]; then if [[ $REMOTE_BRANCHES =~ (^| )$1($| ) ]]; then BRANCH=$1 @@ -14,24 +14,13 @@ fi ${log} "Updating the main repository to branch $BRANCH" -function auto_update(){ - git fetch - NEWVERSION=$(git diff --numstat origin/$BRANCH scripts/bash/update.sh | awk '/update.sh/ {print $NF}') - if [[ -n "${NEWVERSION}" ]]; then - ${log} "Updating the update script first" - # Update the file and restart the script - git checkout origin/$BRANCH scripts/bash/update.sh - exec scripts/bash/update.sh $BRANCH - fi -} - function restart(){ sudo nginx -t && sudo systemctl reload nginx sudo systemctl restart nodered.service } function update(){ - cd /home/pi/PlanktonScope || { echo "/home/pi/PlanktonScope does not exist"; exit 1; } + cd /home/pi/PlanktoScope || { echo "/home/pi/PlanktoScope does not exist"; exit 1; } sudo killall -15 raspimjpeg sudo killall -15 python3 git stash @@ -56,23 +45,22 @@ function special_before(){ function special_after(){ cd /home/pi/.node-red || { echo "/home/pi/.node-red does not exist"; exit 1; } - node_modules/copy-dependencies/index.js projects/PlanktonScope ./ + node_modules/copy-dependencies/index.js projects/PlanktoScope ./ # updating and installing now whatever module has been added to package.json npm update - sudo pip3 install -U -r /home/pi/PlanktonScope/requirements.txt + pip3 install -U -r /home/pi/PlanktoScope/requirements.txt python3 -m pip install -U scikit-image ${log} "Nothing else special to do after updating!" } echo -e "Update on $(date)\n\n\n\n" >> /home/pi/update.log -cd /home/pi/PlanktonScope || { echo "/home/pi/PlanktonScope does not exist"; exit 1; } +cd /home/pi/PlanktoScope || { echo "/home/pi/PlanktoScope does not exist"; exit 1; } remote=$(git ls-remote -h origin $BRANCH | awk '{print $1}') local=$(git rev-parse HEAD) if [[ "$local" == "$remote" ]]; then ${log} "Nothing to do!" else ${log} "Local and Remote are different, we have to update, starting now... Please Wait." - auto_update &>> /home/pi/update.log special_before &>> /home/pi/update.log ${log} "Everything is ready, doing the update now!" update &>> /home/pi/update.log