Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
# Libraries to control the steppers for focusing and pumping
|
2020-10-04 23:28:26 +02:00
|
|
|
import adafruit_motor.stepper
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
import adafruit_motorkit
|
|
|
|
|
import time
|
|
|
|
|
import json
|
|
|
|
|
import os
|
|
|
|
|
import planktoscope.mqtt
|
|
|
|
|
import planktoscope.light
|
|
|
|
|
import multiprocessing
|
2020-10-04 23:28:26 +02:00
|
|
|
import RPi.GPIO
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
# Logger library compatible with multiprocessing
|
|
|
|
|
from loguru import logger
|
|
|
|
|
|
|
|
|
|
logger.info("planktoscope.stepper is loaded")
|
|
|
|
|
|
|
|
|
|
|
2020-10-04 23:28:26 +02:00
|
|
|
class StepperWaveshare:
|
2020-11-25 16:58:32 +01:00
|
|
|
"""A bipolar stepper motor using the Waveshare HAT."""
|
2020-10-04 23:28:26 +02:00
|
|
|
|
|
|
|
|
def __init__(self, dir_pin, step_pin, enable_pin):
|
|
|
|
|
self.dir_pin = dir_pin
|
|
|
|
|
self.step_pin = step_pin
|
|
|
|
|
self.enable_pin = enable_pin
|
|
|
|
|
|
|
|
|
|
RPi.GPIO.setmode(RPi.GPIO.BCM)
|
|
|
|
|
RPi.GPIO.setwarnings(False)
|
2020-11-06 12:45:29 +01:00
|
|
|
RPi.GPIO.setup(
|
|
|
|
|
[self.dir_pin, self.step_pin, self.enable_pin],
|
|
|
|
|
RPi.GPIO.OUT,
|
|
|
|
|
initial=RPi.GPIO.HIGH,
|
|
|
|
|
)
|
2020-10-04 23:28:26 +02:00
|
|
|
self.release()
|
|
|
|
|
|
|
|
|
|
def release(self):
|
|
|
|
|
"""Releases all the coils so the motor can free spin, also won't use any power"""
|
|
|
|
|
self.__digital_write(self.enable_pin, 1)
|
|
|
|
|
|
|
|
|
|
def __digital_write(self, pin, value):
|
|
|
|
|
RPi.GPIO.output(pin, value)
|
|
|
|
|
|
|
|
|
|
def stop(self):
|
|
|
|
|
self.__digital_write(self.enable_pin, 1)
|
|
|
|
|
|
|
|
|
|
def onestep(self, *, direction=adafruit_motor.stepper.FORWARD, style=""):
|
|
|
|
|
"""Performs one step.
|
|
|
|
|
:param int direction: Either `FORWARD` or `BACKWARD`"""
|
|
|
|
|
|
|
|
|
|
if direction == adafruit_motor.stepper.FORWARD:
|
|
|
|
|
self.__digital_write(self.enable_pin, 0)
|
|
|
|
|
self.__digital_write(self.dir_pin, 1)
|
|
|
|
|
elif direction == adafruit_motor.stepper.BACKWARD:
|
|
|
|
|
self.__digital_write(self.enable_pin, 0)
|
|
|
|
|
self.__digital_write(self.dir_pin, 0)
|
|
|
|
|
else:
|
|
|
|
|
logger.error(
|
|
|
|
|
"The direction must be : adafruit_motor.stepper.FORWARD or adafruit_motor.stepper.BACKWARD"
|
|
|
|
|
)
|
|
|
|
|
self.release()
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
# This delay is just to make sure the chip had time to take the dir/enable pin
|
|
|
|
|
# into account, min delay is 650ns
|
|
|
|
|
time.sleep(0.000001)
|
|
|
|
|
self.__digital_write(self.step_pin, True)
|
|
|
|
|
# This delay is the minimal time high for the step impulse, 2µs
|
|
|
|
|
time.sleep(0.000005)
|
|
|
|
|
self.__digital_write(self.step_pin, False)
|
|
|
|
|
|
|
|
|
|
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
class stepper:
|
2020-10-04 23:28:26 +02:00
|
|
|
def __init__(self, stepper, style=adafruit_motor.stepper.SINGLE, size=0):
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
"""Initialize the stepper class
|
|
|
|
|
|
|
|
|
|
Args:
|
2020-10-04 23:28:26 +02:00
|
|
|
stepper (adafruit_motorkit.Motorkit().stepper or StepperWaveshare): reference to the object that controls the stepper
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
style (adafruit_motor.stepper): style of the movement SINGLE, DOUBLE, MICROSTEP
|
|
|
|
|
size (int): maximum number of steps of this stepper (aka stage size). Can be 0 if not applicable
|
|
|
|
|
"""
|
|
|
|
|
self.__stepper = stepper
|
|
|
|
|
self.__style = style
|
|
|
|
|
self.__size = size
|
|
|
|
|
self.__position = 0
|
|
|
|
|
self.__goal = 0
|
|
|
|
|
self.__direction = ""
|
|
|
|
|
self.__next_step_date = time.monotonic()
|
|
|
|
|
self.__delay = 0
|
|
|
|
|
# Make sure the stepper is released and do not use any power
|
|
|
|
|
self.__stepper.release()
|
|
|
|
|
|
|
|
|
|
def step_waiting(self):
|
|
|
|
|
"""Is there a step waiting to be actuated
|
|
|
|
|
|
|
|
|
|
Returns:
|
|
|
|
|
Bool: if time has come to push the step
|
|
|
|
|
"""
|
|
|
|
|
return time.monotonic() > self.__next_step_date
|
|
|
|
|
|
|
|
|
|
def at_goal(self):
|
|
|
|
|
"""Is the motor at its goal
|
|
|
|
|
|
|
|
|
|
Returns:
|
|
|
|
|
Bool: True if position and goal are identical
|
|
|
|
|
"""
|
|
|
|
|
return self.__position == self.__goal
|
|
|
|
|
|
|
|
|
|
def next_step_date(self):
|
|
|
|
|
"""set the next step date"""
|
|
|
|
|
self.__next_step_date = self.__next_step_date + self.__delay
|
|
|
|
|
|
|
|
|
|
def initial_step_date(self):
|
|
|
|
|
"""set the initial step date"""
|
|
|
|
|
self.__next_step_date = time.monotonic() + self.__delay
|
|
|
|
|
|
|
|
|
|
def move(self):
|
|
|
|
|
"""move the stepper
|
|
|
|
|
|
|
|
|
|
Returns:
|
|
|
|
|
Bool: True if the step done was the last one of the movement
|
|
|
|
|
"""
|
|
|
|
|
if self.step_waiting() and not self.at_goal():
|
|
|
|
|
self.__stepper.onestep(
|
|
|
|
|
direction=self.__direction,
|
|
|
|
|
style=self.__style,
|
|
|
|
|
)
|
|
|
|
|
if self.__direction == adafruit_motor.stepper.FORWARD:
|
|
|
|
|
self.__position += 1
|
|
|
|
|
elif self.__direction == adafruit_motor.stepper.BACKWARD:
|
|
|
|
|
self.__position -= 1
|
|
|
|
|
if not self.at_goal():
|
|
|
|
|
self.next_step_date()
|
|
|
|
|
else:
|
|
|
|
|
logger.success("The stepper has reached its goal")
|
|
|
|
|
self.__stepper.release()
|
|
|
|
|
return True
|
|
|
|
|
return False
|
|
|
|
|
|
|
|
|
|
def go(self, direction, distance, delay):
|
|
|
|
|
"""move in the given direction for the given distance
|
|
|
|
|
|
|
|
|
|
Args:
|
|
|
|
|
direction (adafruit_motor.stepper): gives the movement direction
|
|
|
|
|
distance
|
|
|
|
|
delay
|
|
|
|
|
"""
|
|
|
|
|
self.__delay = delay
|
|
|
|
|
self.__direction = direction
|
|
|
|
|
if self.__direction == adafruit_motor.stepper.FORWARD:
|
|
|
|
|
self.__goal = self.__position + distance
|
|
|
|
|
elif self.__direction == adafruit_motor.stepper.BACKWARD:
|
|
|
|
|
self.__goal = self.__position - distance
|
|
|
|
|
else:
|
|
|
|
|
logger.error(f"The given direction is wrong {direction}")
|
|
|
|
|
self.initial_step_date()
|
|
|
|
|
|
|
|
|
|
def shutdown(self):
|
|
|
|
|
"""Shutdown everything ASAP"""
|
|
|
|
|
self.__goal = self.__position
|
|
|
|
|
self.__stepper.release()
|
|
|
|
|
|
|
|
|
|
def release(self):
|
|
|
|
|
self.__stepper.release()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
class StepperProcess(multiprocessing.Process):
|
|
|
|
|
|
|
|
|
|
focus_steps_per_mm = 40
|
|
|
|
|
# 507 steps per ml for Planktonscope standard
|
|
|
|
|
# 5200 for custom NEMA14 pump with 0.8mm ID Tube
|
|
|
|
|
pump_steps_per_ml = 507
|
|
|
|
|
# focus max speed is in mm/sec and is limited by the maximum number of pulses per second the Planktonscope can send
|
|
|
|
|
focus_max_speed = 0.5
|
|
|
|
|
# pump max speed is in ml/min
|
|
|
|
|
pump_max_speed = 30
|
|
|
|
|
|
2020-10-04 23:28:26 +02:00
|
|
|
stepper_type = "adafruit"
|
|
|
|
|
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
def __init__(self, event):
|
|
|
|
|
super(StepperProcess, self).__init__()
|
2020-10-06 11:40:25 +02:00
|
|
|
logger.info("Initialising the stepper process")
|
2020-11-25 16:58:32 +01:00
|
|
|
RPi.GPIO.setup([12, 4], RPi.GPIO.OUT, initial=RPi.GPIO.HIGH)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
self.stop_event = event
|
|
|
|
|
|
|
|
|
|
if os.path.exists("/home/pi/PlanktonScope/hardware.json"):
|
|
|
|
|
# load hardware.json
|
|
|
|
|
with open("/home/pi/PlanktonScope/hardware.json", "r") as config_file:
|
|
|
|
|
configuration = json.load(config_file)
|
|
|
|
|
logger.debug(f"Hardware configuration loaded is {configuration}")
|
|
|
|
|
else:
|
|
|
|
|
logger.info(
|
|
|
|
|
"The hardware configuration file doesn't exists, using defaults"
|
|
|
|
|
)
|
2020-10-27 13:43:10 +01:00
|
|
|
configuration = {}
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
reverse = False
|
2020-11-06 12:45:29 +01:00
|
|
|
microsteps = 16
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
# parse the config data. If the key is absent, we are using the default value
|
|
|
|
|
reverse = configuration.get("stepper_reverse", reverse)
|
2020-11-06 12:45:29 +01:00
|
|
|
microsteps = configuration.get("microsteps", microsteps)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
self.focus_steps_per_mm = configuration.get(
|
|
|
|
|
"focus_steps_per_mm", self.focus_steps_per_mm
|
|
|
|
|
)
|
|
|
|
|
self.pump_steps_per_ml = configuration.get(
|
|
|
|
|
"pump_steps_per_ml", self.pump_steps_per_ml
|
|
|
|
|
)
|
|
|
|
|
self.focus_max_speed = configuration.get(
|
|
|
|
|
"focus_max_speed", self.focus_max_speed
|
|
|
|
|
)
|
|
|
|
|
self.pump_max_speed = configuration.get("pump_max_speed", self.pump_max_speed)
|
2020-10-04 23:28:26 +02:00
|
|
|
self.stepper_type = configuration.get("stepper_type", self.stepper_type)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
# define the names for the 2 exsting steppers
|
2020-10-04 23:28:26 +02:00
|
|
|
if self.stepper_type == "adafruit":
|
|
|
|
|
logger.info("Loading the adafruit configuration")
|
2020-11-06 12:45:29 +01:00
|
|
|
kit = adafruit_motorkit.MotorKit(steppers_microsteps=microsteps)
|
2020-10-06 11:42:07 +02:00
|
|
|
if reverse:
|
|
|
|
|
self.pump_stepper = stepper(kit.stepper2, adafruit_motor.stepper.DOUBLE)
|
|
|
|
|
self.focus_stepper = stepper(
|
2020-10-04 23:28:26 +02:00
|
|
|
kit.stepper1, adafruit_motor.stepper.MICROSTEP, size=45
|
2020-10-06 11:42:07 +02:00
|
|
|
)
|
|
|
|
|
else:
|
|
|
|
|
self.pump_stepper = stepper(kit.stepper1, adafruit_motor.stepper.DOUBLE)
|
|
|
|
|
self.focus_stepper = stepper(
|
2020-10-04 23:28:26 +02:00
|
|
|
kit.stepper2, adafruit_motor.stepper.MICROSTEP, size=45
|
|
|
|
|
)
|
|
|
|
|
elif self.stepper_type == "waveshare":
|
|
|
|
|
logger.info("Loading the waveshare configuration")
|
2020-11-06 12:45:29 +01:00
|
|
|
logger.debug(
|
|
|
|
|
f"Configured microsteps is {microsteps}, check the hardware switches if the stage does not move the intended distance"
|
|
|
|
|
)
|
2020-10-04 23:28:26 +02:00
|
|
|
if reverse:
|
|
|
|
|
self.pump_stepper = stepper(
|
|
|
|
|
StepperWaveshare(dir_pin=24, step_pin=18, enable_pin=4)
|
|
|
|
|
)
|
|
|
|
|
self.focus_stepper = stepper(
|
|
|
|
|
StepperWaveshare(dir_pin=13, step_pin=19, enable_pin=12),
|
|
|
|
|
size=45,
|
|
|
|
|
)
|
|
|
|
|
else:
|
|
|
|
|
self.pump_stepper = stepper(
|
|
|
|
|
StepperWaveshare(dir_pin=13, step_pin=19, enable_pin=12)
|
|
|
|
|
)
|
|
|
|
|
self.focus_stepper = stepper(
|
|
|
|
|
StepperWaveshare(dir_pin=24, step_pin=18, enable_pin=4),
|
|
|
|
|
size=45,
|
2020-10-06 11:42:07 +02:00
|
|
|
)
|
2020-10-04 23:28:26 +02:00
|
|
|
else:
|
|
|
|
|
logger.error(
|
|
|
|
|
"The stepper control type is not recognized. Should be 'adafruit' or 'waveshare'"
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
)
|
2020-10-04 23:28:26 +02:00
|
|
|
logger.error(f"{self.stepper_type} is what was supplied")
|
|
|
|
|
return
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
2020-10-06 11:40:25 +02:00
|
|
|
logger.info(f"Stepper initialisation is over")
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
2020-11-16 17:34:00 +01:00
|
|
|
def __message_pump(self, last_message):
|
|
|
|
|
logger.debug("We have received a pumping command")
|
|
|
|
|
if last_message["action"] == "stop":
|
|
|
|
|
logger.debug("We have received a stop pump command")
|
|
|
|
|
self.pump_stepper.shutdown()
|
|
|
|
|
|
|
|
|
|
# Print status
|
|
|
|
|
logger.info("The pump has been interrupted")
|
|
|
|
|
|
|
|
|
|
# Publish the status "Interrupted" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/pump", '{"status":"Interrupted"}'
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Set the LEDs as Green
|
|
|
|
|
planktoscope.light.setRGB(0, 255, 0)
|
|
|
|
|
|
|
|
|
|
elif last_message["action"] == "move":
|
|
|
|
|
logger.debug("We have received a move pump command")
|
|
|
|
|
# Set the LEDs as Blue
|
|
|
|
|
planktoscope.light.setRGB(0, 0, 255)
|
|
|
|
|
|
|
|
|
|
if (
|
|
|
|
|
"direction" not in last_message
|
|
|
|
|
or "volume" not in last_message
|
|
|
|
|
or "flowrate" not in last_message
|
|
|
|
|
):
|
|
|
|
|
logger.error(
|
|
|
|
|
f"The received message has the wrong argument {last_message}"
|
|
|
|
|
)
|
|
|
|
|
self.actuator_client.client.publish("status/pump", '{"status":"Error"}')
|
|
|
|
|
return
|
|
|
|
|
# Get direction from the different received arguments
|
|
|
|
|
direction = last_message["direction"]
|
|
|
|
|
# Get delay (in between steps) from the different received arguments
|
|
|
|
|
volume = float(last_message["volume"])
|
|
|
|
|
# Get number of steps from the different received arguments
|
|
|
|
|
flowrate = float(last_message["flowrate"])
|
|
|
|
|
|
|
|
|
|
# Print status
|
|
|
|
|
logger.info("The pump is started.")
|
|
|
|
|
self.pump(direction, volume, flowrate)
|
|
|
|
|
else:
|
|
|
|
|
logger.warning(f"The received message was not understood {last_message}")
|
|
|
|
|
|
|
|
|
|
def __message_focus(self, last_message):
|
|
|
|
|
logger.debug("We have received a focusing request")
|
|
|
|
|
# If a new received command is "focus" but args contains "stop" we stop!
|
|
|
|
|
if last_message["action"] == "stop":
|
|
|
|
|
logger.debug("We have received a stop focus command")
|
|
|
|
|
self.focus_stepper.shutdown()
|
|
|
|
|
|
|
|
|
|
# Print status
|
|
|
|
|
logger.info("The focus has been interrupted")
|
|
|
|
|
|
|
|
|
|
# Publish the status "Interrupted" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/focus", '{"status":"Interrupted"}'
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Set the LEDs as Green
|
|
|
|
|
planktoscope.light.setRGB(0, 255, 0)
|
|
|
|
|
|
|
|
|
|
elif last_message["action"] == "move":
|
|
|
|
|
logger.debug("We have received a move focus command")
|
|
|
|
|
# Set the LEDs as Yellow
|
|
|
|
|
planktoscope.light.setRGB(255, 255, 0)
|
|
|
|
|
|
|
|
|
|
if "direction" not in last_message or "distance" not in last_message:
|
|
|
|
|
logger.error(
|
|
|
|
|
f"The received message has the wrong argument {last_message}"
|
|
|
|
|
)
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/focus", '{"status":"Error"}'
|
|
|
|
|
)
|
|
|
|
|
# Get direction from the different received arguments
|
|
|
|
|
direction = last_message["direction"]
|
|
|
|
|
# Get number of steps from the different received arguments
|
|
|
|
|
distance = float(last_message["distance"])
|
|
|
|
|
|
|
|
|
|
# Print status
|
|
|
|
|
logger.info("The focus movement is started.")
|
|
|
|
|
self.focus(direction, distance)
|
|
|
|
|
else:
|
|
|
|
|
logger.warning(f"The received message was not understood {last_message}")
|
|
|
|
|
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
def treat_command(self):
|
|
|
|
|
command = ""
|
|
|
|
|
if self.actuator_client.new_message_received():
|
|
|
|
|
logger.info("We received a new message")
|
|
|
|
|
last_message = self.actuator_client.msg["payload"]
|
|
|
|
|
logger.debug(last_message)
|
|
|
|
|
command = self.actuator_client.msg["topic"].split("/", 1)[1]
|
|
|
|
|
logger.debug(command)
|
|
|
|
|
self.actuator_client.read_message()
|
|
|
|
|
|
|
|
|
|
# If the command is "pump"
|
|
|
|
|
if command == "pump":
|
2020-11-16 17:34:00 +01:00
|
|
|
self.__message_pump(last_message)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
# If the command is "focus"
|
|
|
|
|
elif command == "focus":
|
2020-11-16 17:34:00 +01:00
|
|
|
self.__message_focus(last_message)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
elif command != "":
|
|
|
|
|
logger.warning(
|
|
|
|
|
f"We did not understand the received request {command} - {last_message}"
|
|
|
|
|
)
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
def focus(self, direction, distance, speed=focus_max_speed):
|
|
|
|
|
"""moves the focus stepper
|
|
|
|
|
|
|
|
|
|
direction is either UP or DOWN
|
|
|
|
|
distance is received in mm
|
|
|
|
|
speed is in mm/sec"""
|
|
|
|
|
|
|
|
|
|
logger.info(
|
|
|
|
|
f"The focus stage will move {direction} for {distance}mm at {speed}mm/sec"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Validation of inputs
|
2020-10-27 13:43:10 +01:00
|
|
|
if direction not in ["UP", "DOWN"]:
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.error("The direction command is not recognised")
|
|
|
|
|
logger.error("It should be either UP or DOWN")
|
|
|
|
|
return
|
|
|
|
|
|
|
|
|
|
if distance > 45:
|
|
|
|
|
logger.error("You are trying to move more than the stage physical size")
|
|
|
|
|
return
|
|
|
|
|
|
2020-11-06 12:45:29 +01:00
|
|
|
# We are going to use 32 microsteps, so we need to multiply by 32 the steps number
|
|
|
|
|
nb_steps = round(self.focus_steps_per_mm * distance * 32, 0)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.debug(f"The number of steps that will be applied is {nb_steps}")
|
2020-11-06 12:45:29 +01:00
|
|
|
steps_per_second = speed * self.focus_steps_per_mm * 32
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.debug(f"There will be a speed of {steps_per_second} steps per second")
|
|
|
|
|
|
|
|
|
|
if steps_per_second > 400:
|
|
|
|
|
steps_per_second = 400
|
|
|
|
|
logger.warning("The requested speed is faster than the maximum safe speed")
|
|
|
|
|
logger.warning(
|
2020-11-06 12:45:29 +01:00
|
|
|
f"The speed of the motor is going to be limited to {steps_per_second/32/self.focus_steps_per_mm}mm/sec"
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# On linux, the minimal acceptable delay managed by the system is 0.1ms
|
|
|
|
|
# see https://stackoverflow.com/questions/1133857/how-accurate-is-pythons-time-sleep
|
|
|
|
|
# However we have a fixed delay of at least 2.5ms per step due to the library
|
|
|
|
|
# Our maximum speed is thus about 400 pulses per second or 0.5mm/sec of stage speed
|
2020-10-04 23:28:26 +02:00
|
|
|
if self.stepper_type == "adafruit":
|
2020-10-06 11:42:07 +02:00
|
|
|
delay = max((1 / steps_per_second) - 0.0025, 0)
|
2020-10-04 23:28:26 +02:00
|
|
|
else:
|
|
|
|
|
delay = 1 / steps_per_second
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.debug(f"The delay between two steps is {delay}s")
|
|
|
|
|
|
|
|
|
|
# Publish the status "Started" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/focus",
|
|
|
|
|
f'{{"status":"Started", "duration":{nb_steps / steps_per_second}}}',
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Depending on direction, select the right direction for the focus
|
|
|
|
|
if direction == "UP":
|
|
|
|
|
self.focus_stepper.go(adafruit_motor.stepper.FORWARD, nb_steps, delay)
|
|
|
|
|
|
|
|
|
|
if direction == "DOWN":
|
|
|
|
|
self.focus_stepper.go(adafruit_motor.stepper.BACKWARD, nb_steps, delay)
|
|
|
|
|
|
|
|
|
|
# The pump max speed will be at about 400 full steps per second
|
|
|
|
|
# This amounts to 0.9mL per seconds maximum, or 54mL/min
|
|
|
|
|
# NEMA14 pump with 3 rollers is 0.509 mL per round, actual calculation at
|
|
|
|
|
# Stepper is 200 steps/round, or 393steps/ml
|
|
|
|
|
# https://www.wolframalpha.com/input/?i=pi+*+%280.8mm%29%C2%B2+*+54mm+*+3
|
|
|
|
|
def pump(self, direction, volume, speed=pump_max_speed):
|
|
|
|
|
"""moves the pump stepper
|
|
|
|
|
|
|
|
|
|
direction is either FORWARD or BACKWARD
|
|
|
|
|
volume is in mL
|
|
|
|
|
speed is in mL/min"""
|
|
|
|
|
|
|
|
|
|
logger.info(f"The pump will move {direction} for {volume}mL at {speed}mL/min")
|
|
|
|
|
|
|
|
|
|
# Validation of inputs
|
2020-10-27 13:43:10 +01:00
|
|
|
if direction not in ["FORWARD", "BACKWARD"]:
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.error("The direction command is not recognised")
|
|
|
|
|
logger.error("It should be either FORWARD or BACKWARD")
|
|
|
|
|
return
|
|
|
|
|
|
2020-10-19 22:59:16 +02:00
|
|
|
nb_steps = round(self.pump_steps_per_ml * volume, 0)
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.debug(f"The number of steps that will be applied is {nb_steps}")
|
|
|
|
|
steps_per_second = speed * self.pump_steps_per_ml / 60
|
|
|
|
|
logger.debug(f"There will be a speed of {steps_per_second} steps per second")
|
|
|
|
|
|
|
|
|
|
if steps_per_second > 400:
|
|
|
|
|
steps_per_second = 400
|
|
|
|
|
logger.warning("The requested speed is faster than the maximum safe speed")
|
|
|
|
|
logger.warning(
|
|
|
|
|
f"The speed of the motor is going to be limited to {steps_per_second*60/self.pump_steps_per_ml}mL/min"
|
|
|
|
|
)
|
|
|
|
|
# On linux, the minimal acceptable delay managed by the system is 0.1ms
|
|
|
|
|
# see https://stackoverflow.com/questions/1133857/how-accurate-is-pythons-time-sleep
|
|
|
|
|
# However we have a fixed delay of at least 2.5ms per step due to the library
|
|
|
|
|
# Our maximum speed is thus about 400 pulses per second or 2 turn per second of the pump
|
|
|
|
|
# 10mL => 6140 pas
|
|
|
|
|
# 1.18gr => 1.18mL
|
|
|
|
|
# Actual pas/mL => 5200
|
|
|
|
|
# Max speed is 400 steps/sec, or 4.6mL/min
|
|
|
|
|
# 15mL at 3mL/min
|
|
|
|
|
# nb_steps = 5200 * 15 = 78000
|
|
|
|
|
# sps = 3mL/min * 5200s/mL = 15600s/min / 60 => 260sps
|
2020-10-04 23:28:26 +02:00
|
|
|
if self.stepper_type == "adafruit":
|
2020-10-06 11:42:07 +02:00
|
|
|
delay = max((1 / steps_per_second) - 0.0025, 0)
|
2020-10-04 23:28:26 +02:00
|
|
|
else:
|
|
|
|
|
delay = 1 / steps_per_second
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
logger.debug(f"The delay between two steps is {delay}s")
|
|
|
|
|
|
|
|
|
|
# Publish the status "Started" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/pump",
|
|
|
|
|
f'{{"status":"Started", "duration":{nb_steps / steps_per_second}}}',
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Depending on direction, select the right direction for the focus
|
|
|
|
|
if direction == "FORWARD":
|
|
|
|
|
self.pump_stepper.go(adafruit_motor.stepper.FORWARD, nb_steps, delay)
|
|
|
|
|
|
|
|
|
|
if direction == "BACKWARD":
|
|
|
|
|
self.pump_stepper.go(adafruit_motor.stepper.BACKWARD, nb_steps, delay)
|
|
|
|
|
|
|
|
|
|
@logger.catch
|
|
|
|
|
def run(self):
|
|
|
|
|
"""This is the function that needs to be started to create a thread"""
|
|
|
|
|
logger.info(
|
|
|
|
|
f"The stepper control process has been started in process {os.getpid()}"
|
|
|
|
|
)
|
|
|
|
|
|
|
|
|
|
# Creates the MQTT Client
|
|
|
|
|
# We have to create it here, otherwise when the process running run is started
|
|
|
|
|
# it doesn't see changes and calls made by self.actuator_client because this one
|
|
|
|
|
# only exist in the master process
|
|
|
|
|
# see https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
|
|
|
|
|
self.actuator_client = planktoscope.mqtt.MQTT_Client(
|
|
|
|
|
topic="actuator/#", name="actuator_client"
|
|
|
|
|
)
|
|
|
|
|
# Publish the status "Ready" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish("status/pump", '{"status":"Ready"}')
|
|
|
|
|
# Publish the status "Ready" to via MQTT to Node-RED
|
|
|
|
|
self.actuator_client.client.publish("status/focus", '{"status":"Ready"}')
|
2020-10-06 11:40:25 +02:00
|
|
|
|
|
|
|
|
logger.success("Stepper is READY!")
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
while not self.stop_event.is_set():
|
|
|
|
|
# check if a new message has been received
|
|
|
|
|
self.treat_command()
|
|
|
|
|
if self.pump_stepper.move():
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/pump",
|
|
|
|
|
'{"status":"Done"}',
|
|
|
|
|
)
|
|
|
|
|
if self.focus_stepper.move():
|
|
|
|
|
self.actuator_client.client.publish(
|
|
|
|
|
"status/focus",
|
|
|
|
|
'{"status":"Done"}',
|
|
|
|
|
)
|
|
|
|
|
logger.info("Shutting down the stepper process")
|
|
|
|
|
self.actuator_client.client.publish("status/pump", '{"status":"Dead"}')
|
|
|
|
|
self.actuator_client.client.publish("status/focus", '{"status":"Dead"}')
|
|
|
|
|
self.pump_stepper.shutdown()
|
|
|
|
|
self.focus_stepper.shutdown()
|
|
|
|
|
self.actuator_client.shutdown()
|
2020-10-06 11:40:25 +02:00
|
|
|
logger.success("Stepper process shut down! See you!")
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
|
|
|
|
|
|
|
|
|
|
# This is called if this script is launched directly
|
|
|
|
|
if __name__ == "__main__":
|
2020-10-06 11:42:07 +02:00
|
|
|
# TODO This should be a test suite for this library
|
Extraction and refactor of the python code from node-red flow
The rationale for this rewrite is to improve the readability, the modularity, the reliability and the future-proofing of the main python script.
All in all, this is now 124 commits that are going to be squashed and merged together, spanning more than two weeks of development and testing.
Please test away this release and break things. An upgrading guide will be published in the coming days, along with a new image for people to use if they don't want to upgrade on their own.
Read along if you want to know all the goodies!
As a starter, the python script was extracted from the main flow, and now lives in its own files at `PlantonScope/scripts/*`.
We set up the auto formatting of the code by using [Black](https://github.com/psf/black). This make the code clearer and uniform. We are using the default settings, so if you just install Black and set your editor to format on save using it, you should be good to go.
The code is separated in four main processes, each with a specific set of responsibilities:
- The main process controls all the others, starts everything up and cleans up on shutdown
- The stepper process manages the stepper movements. It's now possible to have simultaneous movements of both motors (this closes #38 ).
- The imager process controls the camera and the streaming server via a state machine.
- The segmenter process manages the segmentation and its outputs. The segmentation happens recursively in all folders in `/home/pi/PlanktonScope/img/`. Each folder has its own output archive, and bug #26 is now closed.
Those processes communicates together using MQTT and json messages. Each message is adressed to one topic. The high level topic controls which process receives the message. The details of each topic is at the end of this commit message.
Every imaging sessions has now its own folder, under the `img` root. Metadata are saved individually for every session, in a JSON file in the same directory as the pictures.
The configuration is not parsed from `config.json` anymore and passed directly through MQTT messages to the concerned process.
A new configuration file has been created: `hardware.json`. This file contains information related to your specific hardware configuration. You can choose to reverse the connection of the motors for example, but you can also define specific speed limits and steps number for your pump and focus stage. This will make it easier for people who wants to experiment with different kind of hardware. It's not necessary to have this file though. If it doesn't exists, the default configuration will be applied.
The code is architectured around 6 modules and about 10 classes. I encourage you to have a look at the files, they're pretty straightforward to understand.
There is a lot of work left around the node-red code refactoring, dashboard ui improvements, better and clearer LED messages, OLED screen integration and finer control of the segmentation process, but this is quite good for now.
Here is the topic lists for MQTT and the corresponding messages.
- actuator : This topic adresses the stepper control thread
No publication under this topic should happen from the python process
- actuator/pump : Control of the pump
The message is a json object
{"action":"move", "direction":"FORWARD", "volume":10, "flowrate":1}
to move 10mL forward at 1mL/min
action can be "move" or "stop"
Receive only
- actuator/focus : Control of the focus stage
The message is a json object, speed is optional
{"action":"move", "direction":"UP", "distance":0.26, "speed":1}
to move up 10mm
action can be "move" or "stop"
Receive only
- imager/image : This topic adresses the imaging thread
Is a json object with
{"action":"image","sleep":5,"volume":1,"nb_frame":200}
sleep in seconds, volume in mL
Can also receive a config update message:
{"action":"config","config":[...]}
Can also receive a camera settings message:
{"action":"settings","iso":100,"shutter_speed":40}
Receive only
- segmenter/segment : This topic adresses the segmenter process
Is a json object with
{"action":"segment"}
Receive only
- status : This topics sends feedback to Node-Red
No publication or receive at this level
- status/pump : State of the pump
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/focus : State of the focus stage
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Done, Interrupted
Publish only
- status/imager : State of the imager
Is a json object with
{"status":"Start", "time_left":25}
Status is one of Started, Ready, Completed or 12_11_15_0.1.jpg has been imaged.
Publish only
- status/segmenter : Status of the segmentation
- status/segmenter/name
- status/segmenter/object_id
- status/segmenter/metric
Here is the original commit history:
* Extract python main.py from flow
* Fix bug in server addresses
These addresses should be the loopback device instead of the network
address of the device. Using the loopback address will not necessitate
to update the script when the network address changes.
* clean up picamera import
* changes to main python and flow:
update MQTT requests address to localhost (bugfix)
update streaming output address to nothing
update main flow to remove python script references and location
* Automatically initialise imaging led on startup to off state.
* Add the ability to invert outputs of the motor
We added a key to config.json "hardware_config" with a subkey
"stepper_reverse". If this key is present in the config file and set to
1, the output of the motors are inversed (stepper2 becomes the pump
motor and stepper1 the focus motor)
* move all non main script to a subfolder
* add __init__.py to package
* light module rewrite
* json cleanup and absolute path for config file
* light.py forgot to import subprocess
#oups
* Add command to turn the leds off
* Auto formatting of main.py
I've used Black with default settings, see https://github.com/psf/black
* First commit of stepper.py
Pump parameters still needs to be checked and tuned.
* addition of hardware details in config.json
* Introduce hardware.json to replace the `hardware_config` of config.json
* stepper.py: calibration, typos
* creates the MQTT_Client class
* pump_max_speed is now in ml/min to help readability
* forgot to add self to the class def
* addition of threading capabilities to stepper.py (UNTESTED)
* mqtt: fix topic bug
* remove counter
* mqtt add doc about topics
* stepper.py creates an "actuator/*/state" topic
* stepper.py: rename mqtt_client to pump_client
* mqtt.py: add details about topics
* stepper.py: rename pump_client to actuator_client
* topic was not split properly and a part was lost
* switch to f-strings for mqtt.py
* cosmetic update
* stepper.py: folder name will be planktoscope change calls
* hardware.json became more straightforward
* stepper.py syntax bugs
* stepper.py addition of a received stop command
* stepper.py: update to max travel distance
* stepper.py: several typos here
* rename folder
* main.py: reword to reflect folder rename
* main.py: remove logic that has been moved to stepper.py and mqtt.py
* main.py: update to add mqtt imaging client
* mqtt.py: make command and args local to class and output more verbose
* make stepper.py a class
* main.py: instantiate stepper class and call it
* main.py: name mqtt client
* update to main.json to reflect main.py changes
* fix bugs around pump control
* update flows to latest version from Thibault
* distance can be a small value, and definitevely should be a float.
* unify mqtt topics
* unify mqtt output in the main flow
* first logger implementation, uses loguru
* mqtt: add reason to on_connect
* mqtt: add on_disconnect handler
* stepper: add more logger calls for debug mainly
* main: add levels for logger
* imager.py: first move of the imager logic
* imager: time import cleanup
* imager: morphocut import cleanup
* imager: skimage import cleanup
* imager: finishing import cleanup
* imager: Class creation - WIP
Also provides a fix for #26 (see line 190).
* imager: threading is needed for Condition()
* streamer: get the streamer server its own file
* imager: creates start_camera and get the server creation out
* imager: subclass multiprocessing.Process
* imager: get Pipeline creation its own function
* imager: cleanup of self calls
* main: code removal and corresponding calls to newly created classes
* imager: various formatting changes
* main: management of signal shutdown
* add requirements.txt
* mqtt: messages are now json objects
Also, addition of a flag on receiving a new message
* mqtt: make message private and add logic to synchronise
* stepper: creates the stepper class
* stepper: use the new class
* stepper: uses the new logic
* stepper: add the shutdown event
* stepper: add shutdown method
* main: add shutdown event
* imager: graceful shutdown
* stepper: nicer way of checking the Eevnt
* self is a required first argument for a method in a class
Especially if you use said class private members!
* python: various typos and small errors in import
* stepper: create mqtt client during init
* stepper: instanciate the mqtt client inside run
Otherwise it's not accessible from inside the loop. It's a PITA,
more information at https://stackoverflow.com/questions/17172878/using-pythons-multiprocessing-process-class
* stepper: little bugs and typos all around
* mqtt: add shutdown method
* mqtt: add connect in init
* stepper: fix bugs, sanitize inputs
* stepper: work on delay prediction improvements
* stepper: json is mean, double quote are mandatory inside
* mqtt: add details about message exchanged
* imager: first implementation of json messages
* main.json: add new tab for RPi management + json for payloads
* imager: add state_machine class
* stepper: publish last will
* imager: major refactor
* main: make streaming server process a daemon
* mqtt: insert debug statement on close
* main: reorder imports
* imager: make it work!
Reinsert the streaming server logic in there, because there is a problem with the synchronisation part otherwise.
Also, eventually, StreamingOuput() will have to be made not global
Final very critical learning: it's super duper important to make sure the memory split is at least 256Meg for the GPU.
Chaos ensues otherwise
* main: changes to accomodate the streamer/imager fusino
* imager_state_machine: insert states transition description
* stepper: cleanup of code
* segmenter: creation of the class
* python: include segmenter changes
* remove unused files
* stepper: check existence of hardware.json
* main.json: changes to reflect the python script evolution
* remove unecessary TODOs and add some others
* main: add check for config and directories
* imager: update_config is implemented and we have better management of directories now
* segmenter: now work recursively in all folders
* flow: the configuration is now sent via mqtt
* segmenter: better manage pipeline error
* segmenter: declaration of archive_fn in init
* imager: small bugs and typos
* main: add uniqueID output
* imager: add the camera settings message
We can now update the ISO, shutter speed and resolution from Node-Red
* package.json: update dependencies
2020-09-28 11:05:27 +02:00
|
|
|
# Starts the stepper thread for actuators
|
|
|
|
|
# This needs to be in a threading or multiprocessing wrapper
|
|
|
|
|
stepper_thread = StepperProcess()
|
|
|
|
|
stepper_thread.start()
|
|
|
|
|
stepper_thread.join()
|