Technological choices

This document is meant to answer why certain technological choices have been made and why certain frameworks have been chosen.

Main

The TinyMLaaS-main -repository is meant for mostly building the whole software with docker and documentation. However, some development is also done in this repository. All tensorflow modules are coded here in Jupyter notebooks.

Jupyter notebooks and NBDEV

As jupyter notebooks can not be used as modules in python, they need to be exported into python modules. This is done with NBDEV. Nbdev also automatically creates documentation from the jupyter notebooks and deploys them to Github pages

Docker

Running the software is meant to be done with docker. Docker allows running the software on different computers, without the software being platform spesific. Also, all the dependencies required for the software do not need to be installed on the host, rather, they will all be installed in the seperate docker container. If you are not familiar with docker, check out University of Helsinkis course Devops with Docker materials to get a basic understanding of docker.

Sysbox

There are parts of the software that require starting their own docker containers. For example, the relay will start containers to compile arduino sketches and to install them to the devices. When running the relay itself inside a docker container, there are a few ways of starting a new docker container from this docker container. First, is by using so called sibling containers. This gives the docker container access to the host machines docker socket, which allows it to control other containers on the host machine. However, this has a big security flaw, as if someone gets access to this docker container, they will be able to control all other docker containers on the host machine and start their own containers. The other way is by using docker inside docker, which allows docker containers to be started inside the docker container recursively. This approach requires privileged mode to be set for the docker container, meaning that it will have root privileges on the host machine. In order to run docker machines without privileged mode, Sysbox runtime is used. This allows starting docker containers inside the docker containers without having the docker container in privileged mode, which is a lot more secure and allows for better isolation of the docker containers.

Frontend

Streamlit

The frontend of the software is build with Streamlit. This is done to make the development process faster, as this frontend is mainly meant for demo purposes. Streamlit makes it easy to create good looking websites, however, there isn’t much room for cutomization and some features can be quite difficult to create.

There is still a dependecy on usbutils. This will be talked more about in the Bridge-section

Backend

The backend is the heart of the software. It does all the communication between all the other modules and does a lot of the heavy lifting of the software. It is created as a API.

FastAPI

The backend is created with FastAPI. FastAPI is very powerful for creating API:s, as it has great data validation with the help of Pydantic, it automatically creates good documentation about the different API requests and is simple to understand. To checkout more, read the Starting documentation.

When deploying the API to production, the api will most likely be behind a proxy with some URL that has prefixes. For example, it might be deployed to example.uri.com/api/. For the API to function correctly, the root-path of the API needs to be declared for FastAPI, in this case, /api/.

SQLAlchemy

The backend talks with the database with sqlalchemy. This means that it is able to talk with any SQL-database without any changes to the backends software.

As of now, the database in use is sqlite. However, this is meant to be more of a temporary solution to make development easier. For more information, checkout the suggestions in Suggestions for further development

Bridge

The relay is the part of the software to which the microcontrollers are connected to. It is also done in API style.

Flask

Unlike the backend, the bridge is created with Flask. Flask is lightweight and easy to understand, which makes sence for the bridge, as the hardware, on which the bridge runs, might not be that powerfull.

Usbutils

To find USB-devices, the software does not use pythons libraries, such as PyUSB. This is because these softwares also have OS dependencies, that need to be installed and do not work that well in docker containers. USButils works great in contianers and is easy to install, which is why it has been chosen over pythons own libraries.

Command-line interface

An API client is automatically generated from the OpenAPI definition provided by FastAPI. The generation is done with OpenAPI Generator. A command line tool interfacing the autogenerated client is built with Typer.

Open API Generator

OpenAPI Generator enables building extensive Python clients with documentation. Generating a client makes it easy to design customized workflows around the API. This project uses the client as the main component of the command-line interface. The autogeneration also builds templates for tests. More generally using a generator was a way to test automatic code generation.

Typer

Typer is an easy to use Python library for building command-line interfaces. Typer can be used to build light weight CLI’s so it’s a good fit for the autogenerated client. Typer also uses Python type hints and provides automatic help functions that include required arguments and a command’s description from docstrings.