* Bump black from 22.8.0 to 24.3.0 in /python/web
Bumps [black](https://github.com/psf/black) from 22.8.0 to 24.3.0.
- [Release notes](https://github.com/psf/black/releases)
- [Changelog](https://github.com/psf/black/blob/main/CHANGES.md)
- [Commits](https://github.com/psf/black/compare/22.8.0...24.3.0)
---
updated-dependencies:
- dependency-name: black
dependency-type: direct:development
...
Signed-off-by: dependabot[bot] <support@github.com>
* Github CI: Backend web test runner use python 3.9.19
* Reformat python sources with black
* Docker: Bump to python 3.9-slim image for pytest
* Bump pytest version to 8.1.1
* Bump more library versions, and freeze them
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Daniel Markstedt <daniel@mindani.net>
* Don't log names of internal symbols that can be used for attacks
* Add integrity check to dropzone.js library
* Better a11y of web form labels
* Safer handling of file download paths
* Don't invert boolean check
* Make backend auth check a flask abort
* Clean up indentation to remove unwanted tabs
* Run workflow either on PR events, or branch pushes, not both
* Upload to tmp file name then rename if successful
* Move the dropzone.js operations back into web.py
* Move list_images() from file commands into piscsi commands (it was the only class method in that package that calls the protobuf interface)
* Remove now-redundant helptext
- Allow create, copy and move file operations to opt-in to overwrite
- Use this when extracting packaged properties to avoid it getting stuck in the images dir
- Better error handling in common file operation methods
- File operation class methods create parent dirs if they don't exist
- Avoid stripping path from file names in several places
- Simplify prop file matching logic: check for existence of file
- Remove list_files() method which is now unused
* New class method for creating image file with python; use python file operations across the board.
Also fixed several bugs:
Added check that a file exists before copying/moving/creating
Check that parent dir exists always passed
Incorrect return message token
- Break out generate_iso() into its own class method in file_cmds
- Add form and handling of using local file to generate iso image
- Reorder index page sections to group actions that create new image files
* Rebrand project to PiSCSI
- rascsi ->piscsi
- rasctl -> scsictl
- rasdump -> scsidump
- ras* -> piscsi* (rasutil -> piscsi_util, etc.)
* Refined the formatting and wording of the app startup banner
* Kept some references to rascsi and rasctl where backwards compatibility is concerned
* Point to the new github repo URL
Co-authored-by: nucleogenic <nr@nucleogenic.com>
Co-authored-by: Uwe Seimet <Uwe.Seimet@seimet.de>
- Fixed ignore patterns in .dockerignore
- Added healthchecks to backend and web containers
- Reduced Docker image sizes
- Removed RaSCSI references in various areas (e.g. rascsi -> backend)
- Added compilation-only step to easyinstall.sh
- Moved apt package lists to variables
- Revert to triggering GitHub Actions runs on push
- Updated web/frontend_checks workflow to run black and flake8 against all Python sources
- Capture log files from backend/web containers
- Fix None to float conversion bug when user agent is absent or unrecognised
* Update config for black and flake8
* Auto-format Python sources with black
* Fix issues reported by flake8
* Exclude protobuf files from black
* Address formatting feedback
- Display the pretty hostname as system name in header
- Move IP and hostname down to the footer
- New endpoint for setting the pretty hostname, plus form field in the Web UI
- (unrelated) Use platform.uname() instead of shell uname
- (unrelated) Better logic for fetching the Mac HD Drivers zip file in easyinstall.sh
- Bring back the shutdown splash, with the tweak that is blanks out after 700ms
- Restore the non-Latin transliteration originally from https://github.com/akuker/RASCSI/pull/449 which was lost when the `common` package was introduced
- Bump to the latest libraries, while removing implicit dependencies from requirements.txt
- Shorter duration of the startup splash
- Reintroduce shell shutdown/reboot methods for use with the Web UI. This addresses https://github.com/akuker/RASCSI/issues/538 (the ctrlboard client will continue to use the built-in rascsi system calls which are slightly faster)
- Remove overt references to RPi. This addresses https://github.com/akuker/RASCSI/issues/990
- Other refactorings
* Expose hardware info in env string
* Improve error handling for get_properties_by_drive_name
* Verbose uname parameters.
* Close open file in introspect method
* Add error handling for when no drive name was matched with props in the database
- New "format as" option when creating new images; removing the image creation options from easyinstall
- Bring in HFSer as new submodule providing the driver binaries; removing the Lido driver binary from this repo
- Add SpeedTools driver option
- Point to github mirror of hfdisk, since the original git server is down
- While rearranging the easyinstall options, moved the CtrlBoard option up to the main section
- Add an easyinstall script to configure Samba, while consolidating file sharing with Netatalk
- Sanitize file paths with Path: for flat file structures, always extract Path().name, and for nested file structures either look for absolute paths, or someone trying to use ".." to traverse the dir strucutre.
- Reduce redundancy in network bridge detection method, and return somewhat more informative messages
- Make all endpoints return exactly one message
- Move some warning messages to logging
- Use tempfile for iso generation temp file handling
- Use Path objects for file operations
- Use urllib to sanitize URLs
- Some explicit type conversions
- Consistent regex syntax
- Add rudimentary logging when archive extraction caching fails
- Fixed two cases of the property file creation not being notified in the Flash message
- added doctype and html lang attribute to the web server 502 page
- copy/move/delete file class methods now take Path objects as arguments
- The file download endpoint in the Web UI uses the safer download from dir method
- Simplified logging
- Merged nested if statements
- Removed naked handling of unknown error states
- Added fallback empty list for drive_properties, to avoid errors when json file is missing or broken
- Move drive_properties to env[]
- Constants for common error messages
- Dummy variable for list comprehension
- class method to get arbitrary file contents
- new web UI endpoint to fetch the contents of rascsi man pages (rendered txt files)
- Link to manpage in page footer
- Link to new wiki page for image type docs
- Other UI tweaks
new:
- new templates to render structured info contents in
- get_diskinfo() class method that calls disktype and returns the results
- /diskinfo endpoint in the Flask app that renders the results from get_diskinfo()
changed:
- /logs/show and /scsi/info endpoints in the Flask app render in templates
- Now using the "RaSCSI Reloaded Control Page" header to function as the link back to the homepage (instead of the github project) which is in line with how most webapps work
- Removed the center style for "Attached!" to allow the ? button to be placed on the same line
- Remove individual device info, and introduced show all device info in a template
Add --token parameter to easyinstall.sh
Add --skip-token parameter to easyinstall.sh
Install required apt packages explicitly (--no-install-recommends)
Allow standalone RaSCSI and web UI installations to specify an auth token
Add development mode to web UI (web/start.sh --dev-mode)
Initial Docker-based development environment for Python and web UI
Bump protobuf version
Workaround for Flask development server and asyncio incompatibility
Build Python protobuf interface on container launch, if it doesn’t exist
Allow containers to be configured with environment variables, add support for token authentication
Move web UI live editing setup out of main Docker Compose config
Update dockerignore to exclude by default
Update README
Add OS_DISTRO, OS_VERSION and OS_ARCH build args
Allow extracted files to be moved to target when crossing a filesystem boundary
Reduce noise from watchmedo auto-restarts
Update Docker tag structure to rascsi:{build}-{platform}-{variant}
Prevent Docker Compose from attempting to pull images from Docker registry
Add workaround for issue #821
Allow container processes to be stopped with Ctrl+C
Update README, bind to ports 8080/8443 on the Docker host by default
Update README to clarify audience and no board connectivity
Add AIBOM and GAMERNIUM to --connect_type validation
Update cfilesystem.patch following rebase
Summary ov most important changes triggered by the SASI code removal:
- Removed the SASI controller code
- New controller management. There is a new controller base class AbstractController and a class ControllerManager managing the controller lifecycle. The lifecycle management was removed from rasci.cpp and is covered by unit tests.
- New device management. The DeviceFactory manages the device lifecycle instead of rascsi.cpp. The new code is covered by unit tests.
- The lifecycle managment uses C++ collections with variable size instead of arrays with hard-coded sizes.
- The ScsiController method contains most of what was previously contained in scsidev_ctrl.cpp plus the code from sasidev_ctrl.cpp that was relevant for SCSI.
- scsi_command_util contains helper methods used for identical SCSI command implementations of more than one device
- Devices know their controllers, so that the controller instance does not need to be passed to each SCSI command. This change helps to decouple the devices from the controller. The phase_handler interface is also part of this decoupling.
- Use scsi_command_exception for propagating SCSI command execution errors, This resolves issues with the previous error handling, which was based on return values and often on magic numbers.
- Removed legacy SCSI error codes, all errors are now encoded by sense_key::, asc:: and status::.
- Fixed various warnings reported with -Wextra, -Weffc++ and -Wpedantic.
- Use constructor member initialization lists (recommended for ISO C++)
- Consistently use new/delete instead of malloc/free (recommended for ISO C++), resulting in better type safety and error handling
- Replaced variable sized arrays on the stack (violates ISO C++ and can cause a stack overflow)
- Replaced NULL by nullptr (recommended for C++), resulting in better type safety
- Use more const member functions in order to avoid side effects
- The format device page can now also be changed for hard disk drives (Fujitsu M2624S supports this, for instance), not just for MOs.
- Better encapsulation, updated access specifiers in many places
- Removed unused methods and method arguments
- Fixed a number of TODOs
- Added/updated unit tests for a lot of non-legacy classes
- Makefile support for creating HTML coverage reports with lcov/genhtml
* Move the oled script's PiCmds module to common, and rename it SysCmds.
* Use sys_cmds.get_ip_and_host() in web UI code.
* Move the auth_active() method to device_utils
* Rename device_utils to web_utils. Make auth_active() method take the group as argument.
* Migrate all pi_cmds methods to the SysCmds common class.
* Display hostname and ip in Web UI.
* Resolve or suppress pylint warnings.
* Resolve a pylint warning.
* Resolve or suppress pylint warnings.
* Import libraries at the top level for readability. In my testing on a Pi3B+, this leads to ~1.5k more memory being used by the python3 process.
* Change page title as requested by akuker.
* Reenable the import-outside-toplevel pylint rule.
* Resolve pylint warnings.
* Fix error following refactoring.
* Minor UI tweaks.
* Cleanup.
* Break out bridge config validation into a utility method.
* Move the dropzonejs method into the web_utils package
* Move get_logs method into SysCmds class.
* Improve get logs UI.
* Resolve pylint warning.
* Standardize class instance name.
* Add capabilities to RaCtlCmds.get_device_types() to return the image file support boolean, and list of supported block sizes.
* Inquire rascsi backend about the min block size rather than hard coding values in the web UI.
* Add class methods for getting lists of certain device types.
* Use the new class methods to get lists of device types in the web ui.
* Make use of the new class methods in the oled script.
* Remove now unused constants, and simplify logic in common_settings
* Improve device name mapping to extend the existing dictionary rather than creating a new data structure.
* Use jinja2 sort filters instead of sorting in python code. Removing redundant variables.
* Introduce the get_device_name() utility method which returns the translated name for a device acronym. Use the new method to display device name when attaching devices.
* Fix typo
* Rename Support Device to Periperal Device. General tweaks to UI strings.
* Tweak UI string.
* Fix error.
* Tentative Web UI for the Printer device.
* timeout must be a positive value
* Change the attach device class method to accept a dict with arbitrary key value pairs of parameters to be passed to the protobuf interface, rather than hard coded ones. Also renames the RaCtlCmds.attach_image() class method to attach_device().
* Make download_to_iso() use the new attach interface.
* Dynamically get the form items for support devices.
* Change the data structure returned by RaCtlCmds.get_device_types() to a dict, which contains the supported parameters and their default values. Leverage this data in the web ui to derive the form fields from the capabilities of rascsi.
* Tweak UI labels.
* Update FileCmds.read_config() to work with the new RaCtlCmds.attach_device() method.
* Check for numeric value.
* Streamline the UI for support devices.
* Handle support devices better by the oled screen.
* Clean up html.
* Dynamically adjust form field size based on data length.
* Use the comparison operator that Python3 likes.
* Add SUPPORT_DEVICE_TYPES category, and add Host Service to it.
* Add webapp UI for attaching a support device.
* Make the OLED screen aware of the support device type.
* Tweak test to make it clear that this is experimental functionality.
* Tweak device type names
* Improve the device type selection UI
* Extend the image_files data structure with human readable device name, instead of having complex for loops in the jinja2 template.
* Leverage device type constants in the OLED monitor script
* Fix typo
* Generate the list of valid network devices that can be attached programmatically
* Fix typo
