Commit Graph

192 Commits

Author SHA1 Message Date
Oliver
7ab5ddcd7d
Plugin auto migrate (#5668)
* Improved handling of race condition when saving setting value

* Improvements for managing pending migrations

- Inform user if there are outstanding migrations
- reload plugin registry (if necessary)

* Increase django-q polling time

According to this thread, should reduce multiple workers taking the same task:

https://github.com/Koed00/django-q/issues/183#issuecomment-239676084

* Revert default behavior

* Better logging

* Remove comment

* Update unit test

* Revert maintenance mode behaviour

* raise ValidationError in settings
2023-10-06 11:38:01 +11:00
Lukas
8ec04028d5
Native LDAP support (#5226)
* Added basic ldap support

* Added new settings to config template

* Added missing apk dependency to dockerfile

* update requirements.txt

* Remove requirements and added docs

* Update advanced.md

* Added more ldap options

* allow direct bind
* add ldap debug mode
* add ldap global_options
* add start_tls
* use get_boolean_setting

* Update advanced.md to use a warning box

* fix: style

* Update advanced.md

* Remove ldap install instructions for non docker installs
2023-10-05 07:52:36 +11:00
Matthias Mair
71c416bafd
Add flake8-logging linting (#5620)
* added flake8-logging to make logging more robust

* fixed LOG005 warnings

* fixed LOG008 warnings

* fixed LOG011 warnings

* fixed more LOG011 errors

* added ignores for intentional logger.error calls

* fixed even more LOG011 errors
2023-09-28 14:53:22 +10:00
Lukas
073a275d89
Skip ready functions if not in main thread or plugins are not loaded yet (#5005)
* Skip ready functions if not in main thread or plugins are not loaded yet

* Debug integration tests

* Update ready.py

* Update ready.py

* Fix isInMainThread and isPluginRegistryLoaded ready functions

* Preload gunicorn app to only invoke the appconfig ready functions once

* debug: test prints for statistics

* Remove debug print

* Test without

* Revert "Test without"

This reverts commit 1bc1872893.

* Second test

* Add checks back to part, label, user model

* Add checks back to inventree, plugin apps

* log server output for debugging

* hopefully I can get the log this time+

* Next test

* Test with --noreload

* Next test

* trigger: ci, because session expired

* block the second ready execution instead of the first

* fix: load order

* Fix test and revert gh actions workflow change

* Added all_apps method to reload machanism

* Changed detect reload mechanism

* Also trigger ready on reload

* Add skipping second reload back for testing mode

* Added doc string back

* Update InvenTree/plugin/base/integration/AppMixin.py
2023-07-26 08:33:13 +10:00
Seth Price
60f344a360
Remove container-name:s from Compose file (#5339)
* Remove `container-name:`s from Compose file

* Remove `container-name:`s from dev Compose file

I forgot this one existed, oops.
2023-07-25 12:51:37 +10:00
Oliver
b717011f06
Docker build: Update python deps (#5270)
* Update python deps

* Update requirements.in

* Fix requirements-dev.txt
2023-07-18 19:43:45 +10:00
Oliver
f4f39e274f
Add https comment to nginx file (#5243) 2023-07-14 08:53:37 +10:00
Lukas
4a1f73331f
Refactor: Dockerimage (#5007)
* refactor docker image to reduce size

* Added required deps for database drivers

* Added alpine test dockerfile

* Readded image libs to Dockerfile

* Fix link in dockerfile

* Replaced Dockerfile

* Remove bash from image

* Added base target to Dockerfile

* Added missing mysql dev headers

* Add dev target back to Dockerfile

* fix: Dockerfile style

* Add old comments back to dockerfile

* Removed wrong comment

* Added back piwheels repo
2023-07-10 19:52:33 +10:00
Oliver
8b730884d7
Handle exception when creating default labels (#5163)
* Handle exception when creating default labels

- Running workers in parallel may cause race conditions
- Catch any exception which is raised

* Prevent password from being logged

* Update default timeout for docker
2023-07-04 22:46:03 +10:00
Oliver
198245d0d2
Pin mysqlclient lib for docker (#5091)
- Recent updates have caused breakage
2023-06-23 20:21:26 +10:00
Matthias Mair
21ed4b2081
Fix common spelling mistakes (#4956)
* add codespell

* first fixes

* doc fixes

* fix docstrings and comments

* functional changes

* docstrings again

* and docs again

* rename args

* add ignore

* use pre-commit for filtering instead

* ups

* fix typo in filter
2023-06-04 00:04:52 +10:00
Matthias Mair
aa2f5e330a
Make nginx more clear that changing the port is 'dangerous' (#4880)
* Make nginx more clear that changing the port is 'dangerous'
See https://github.com/inventree/InvenTree/issues/4859#issuecomment-1560025566

* also add note to prod docker-compose
2023-05-24 09:05:16 +10:00
Oliver
0f445ea6e4
[Feature] Stocktake reports (#4345)
* Add settings to control upcoming stocktake features

* Adds migration for "cost range" when performing stocktake

* Add cost data to PartStocktakeSerializer

Implement a new custom serializer for currency data type

* Refactor existing currency serializers

* Update stocktake table and forms

* Prevent trailing zeroes in forms

* Calculate cost range when adding manual stocktake entry

* Display interactive chart for part stocktake history

* Ensure chart data are converted to common currency

* Adds new model for building stocktake reports

* Add admin integration for new model

* Adds API endpoint to expose list of stocktake reports available for download

- No ability to edit or delete via API

* Add setting to control automated deletion of old stocktake reports

* Updates for settings page

- Load part stocktake report table
- Refactor function to render a downloadable media file
- Fix bug with forcing files to be downloaded
- Split js code into separate templates
- Make use of onPanelLoad functionalitty

* Fix conflicting migration files

* Adds API endpoint for manual generation of stocktake report

* Offload task to generate new stocktake report

* Adds python function to perform stocktake on a single part instance

* Small bug fixes

* Various tweaks

- Prevent new stocktake models from triggering plugin events when created
- Construct a simple csv dataset

* Generate new report

* Updates for report generation

- Prefetch related data
- Add extra columns
- Keep track of stocktake instances (for saving to database later on)

* Updates:

- Add confirmation message
- Serializer validation checks

* Ensure that background worker is running before manually scheduling a new stocktake report

* Add extra fields to stocktake models

Also move code from part/models.py to part/tasks.py

* Add 'part_count' to PartStocktakeReport table

* Updates for stocktake generation

- remove old performStocktake javascript code
- Now handled by automated server-side calculation
- Generate report for a single part

* Add a new "role" for stocktake

- Allows fine-grained control on viewing / creating / deleting stocktake data
- More in-line with existing permission controls
- Remove STOCKTAKE_OWNER setting

* Add serializer field to limit stocktake report to particular locations

* Use location restriction when generating a stocktake report

* Add UI buttons to perform stocktake for a whole category tree

* Add button to perform stocktake report for a location tree

* Adds a background tasks to handle periodic generation of stocktake reports

- Reports are generated at fixed intervals
- Deletes old reports after certain number of days

* Implement notifications for new stocktake reports

- If manually requested by a user, notify that user
- Cleanup notification table
- Amend PartStocktakeModel for better notification rendering

* Hide buttons on location and category page if stocktake is not enabled

* Cleanup log messages during server start

* Extend functionality of RoleRequired permission mixin

- Allow 'role_required' attribute to be added to an API view
- Useful when using a serializer class that does not have a model defined

* Add boolean option to toggle whether a report will be generated

* Update generateStocktake function

* Improve location filtering

- Don't limit the actual stock items
- Instead, select only parts which exist within a given location tree

* Update API version

* String tweaks

* Fix permissions for PartStocktake API

* More unit testing for stocktake functionality

* QoL fix

* Fix for assigning inherited permissions
2023-02-17 11:42:48 +11:00
Oliver
362f94eae2
Docker build fix (#4298)
* Configure safe directory for git

* Test docker run on PR

* Disable docker build for PR
2023-02-03 07:14:17 +11:00
Matthias Mair
c6df0dbb2d
[BUG] Increase worker-timeout to account for install times (#4285)
Fixes #4284
2023-02-01 10:25:10 +11:00
Oliver
ee2290da28
Gunicorn options (#4192)
* Add option for controlling gunicorn timeout

* Adds gunicorn timeout as docker env variable
2023-01-10 07:53:57 +11:00
Oliver
faaed3dc14
docker venv fix (#4118)
* docker venv fix

- Run once with --upgrade-deps
- Required to ensure that setuptools is automatically upgraded

* Fix comment
2022-12-30 22:55:24 +11:00
Oliver
0f55cbf7f5
Import setuptools in manage.py (#4061)
* Import setuptools in manage.py

Ref: https://github.com/pypa/setuptools/issues/3706

* Import all the things

* Show setuptools version

* Update setuptools to latest verseion

* Force upgrade of setuptools version

* Remove debug

* Prevent docker builds on PR

* Remove setuptools imports
2022-12-16 07:42:07 +11:00
Oliver
53ecca34d8
gunicorn: revert support for gevent (#4035)
- For some reason this causes production docker image to fail to build
- but only in GitHub CI ??
2022-12-09 16:51:58 +11:00
Oliver
6b659ba22f
Improvements for docker / gunicorn (#4031)
* Additional options for gunicorn configuration file (in docker):

Ref: https://pythonspeed.com/articles/gunicorn-in-docker/

* Catch potential error on startup

* Tweak log message for docker

* Wrap news feed update in try block
2022-12-08 23:06:02 +11:00
Oliver
4a3ca6d326
re-enable plugins by default for debug docker environment (#3894) 2022-11-02 09:51:36 +11:00
Matthias Mair
182bc29053
Add db an media backups (#3783)
* [FR] Add backup into inventrees lifecycle
Fixes #3778

* Add env to testing enviros

* block backup from running app commands

* Add more commands

* fix postgres version

* Update used env

* add daily task to run backups

* add installer changes
2022-10-17 00:09:31 +11:00
Matthias Mair
a3c9331a08
Add content headers to config (#3731)
* add content header

* add content header

* fix format

* fix format
2022-09-30 10:47:56 +10:00
Oliver
e9b0f02ecd
Update demo data hook to copy media files (#3441)
* Simplify settings.py / config.py

- get_setting function has been streamlined
- move some functions into config.py

* Spelling fix: IGNORRED is IGNORED

* Ensure yaml is installed as part of docker image

- invoke path is still mucking us around

* Fix broken migration

* Copy media files from demo dataset when installing test data

* Cleanup settings.py

* Fix for configuration file traversal

* Line fix

* Update quickstart guide for docker

* Allow plugin file and plugin dir to be specified in configuration file

* Cleanup config template file

* Allow secret_key information to be provided in configuration file

* Adjust root paths for CI tests

* resolve paths

* Revert paths for CI step

* remove dead code

* Revert configuration variables to old names

- Prevent breaking changes

* Simplify secret key generation

* Fix default timeout for background worker process

* Revert change for customization options
2022-07-31 23:16:58 +10:00
Oliver
703731ec4d
Fix for docker-compose v1 (#3434)
v1 does not support the name attribute
2022-07-30 11:43:15 +10:00
Oliver
3fc9a36d4b
Simplification of development docker-compose file (#3429)
* Simplification of development docker-compose file

- Rename .env file
- Remove requirement for variable interpolation within the docker-compose file itself

* Add debug flag to CI test

* Additional quick-start docs (for docker)

* docker-compose update

- Introspection did not work quite as expected
- Set project name inside docker-compose file

* Fixes for "setup_test" task

- Check if directory exists before first deleting it
- Parameterize the "path" variable
- Add option to ignore update step

* Add demo data dir to .gitignore

* Remove debug call

* Update CONTRIBUTING.md
2022-07-30 10:34:16 +10:00
Oliver
ab654c3ae7
Disable redis image by default (#3416)
- Enable using the docker --profile command
2022-07-27 13:02:26 +10:00
Oliver
7cabb78964
Docker compose relative path (#3410)
* Update development docker-compose file

- Remove external volume definition
- v2 does not allow relative path spec here
- Simplification is only required for dev version

* Remove old debug messages

* Update docker build CI step

- Check that required directories / files have been created in the correct location(s)

* Add check for pgdb directory

* Run partial docker check on pull_request

* Disable workflow running on pull requests
2022-07-27 07:50:54 +10:00
Oliver
622c0636ef
Remove references to "plugins" volume in production docker-compose file (#3406) 2022-07-26 15:29:37 +10:00
Oliver
5bb76a3730
Docker compose plugins fix (#3357)
* Simplify dockerfile - remove INVENTREE_DEV_DIR

- We can simply use INVENTREE_DATA_DIR
- Will need to update the documentation

* Remove plugin dir support from production docker-compose file
2022-07-20 09:05:05 +10:00
Matthias Mair
cbae4474ab
Allow installation of local plugins via docker (#3348)
* Add option to mpunt plugins to docker

* [FR] Allow installation of local plugins via docker
Fixes #3343
2022-07-19 08:18:54 +10:00
Oliver
f23cef0400
docker: prevent ports being exposed outside container context (#3351) 2022-07-18 14:08:33 +10:00
Matthias Mair
7549d033df
Use env-variable for image tag (#3350) 2022-07-18 09:09:04 +10:00
Oliver
f223ea224d
Docker requirements update (#3266)
* Pin mariadb package <= 1.1.0

- Recent update to 1.1.2 requires newer version of mariadb connector

* Change <= to <
2022-06-28 13:47:10 +10:00
Oliver
63f1e58ca9
Fix container priority for docker compose recipe (#3180)
- Cache must be running *before* the server
- Server must be running *before* the worker
2022-06-11 19:58:36 +10:00
Oliver
d9efe27f8a
Adds redis support to production docker-compose (#3171)
* Adds a redis container to the production docker-compose script

* Fix ports
2022-06-10 20:16:19 +10:00
Matthias Mair
f38386b13c
Add more checks to pre-commit (#3132)
* Add bandit to pre-commit checks

* fix catchall exceptions

* remove unused definitons

* remove unuseed ariables

* Add docstring

* fix B006, B008 errors

* fix B007 error

* ignore B009

* Add checks for formatting and naming
2022-06-06 08:56:52 +10:00
Oliver
b9fd263899
Docker improvements (#3042)
* Simplified dockerfile

- Changed from alpine to python:slim
- Removed some database libs (because we *connect* to a db, not host it)

* - Add gettext as required package
- Only create inventree user as part of production build (leave admin access for dev build)

* Tweaks for tasks.py

* Fix user permissions (drop to inventree user)

* Drop to the 'inventree' user level as part of init.sh

- As we have mounted volumes at 'run time' we need to ensure that the inventree user has correct permissions!
- Ref: https://stackoverflow.com/questions/39397548/how-to-give-non-root-user-in-docker-container-access-to-a-volume-mounted-on-the

* Adjust user setup

- Only drop to non-root user as part of "production" build
- Mounted external volumes make it tricky when in the dev build
- Might want to revisit this later on

* More dockerfile changes

- reduce required system packages
-

* Add new docker github workflow

* Print some more debug

* GITHUB_BASE_REF

* Add gnupg to base requirements

* Improve debug output during testing

* Refactoring updates for label printing API

- Update weasyprint version to 55.0
- Generate labels as pdf files
- Provide filename to label printing plugin
- Additional unit testing
- Improve extraction of some hidden debug data during TESTING
- Fix a spelling mistake (notifaction -> notification)

* Working on github action

* More testing

* Add requirement for pdf2image

* Fix label printing plugin and update unit testing

* Add required packages for CI

* Move docker files to the top level directory

- This allows us to build the production image directly from soure
- Don't need to re-download the source code from github
- Note: The docker install guide will need to be updated!

* Fix for docker ci file

* Print GIT SHA

* Bake git information into the production image

* Add some exta docstrings to dockerfile

* Simplify version check script

* Extract git commit info

* Extract docker tag from check_version.py

* Newline

* More work on the docker workflow

* Dockerfile fixes

- Directory / path issues

* Dockerfile fixes

- Directory / path issues

* Ignore certain steps on a pull request

* Add poppler-utils to CI

* Consolidate version check into existing CI file

* Don't run docker workflow on pull request

* Pass docker image tag through to the build

Also check .j2k files

* Add supervisord.conf example file back in

* Remove --no-cache-dir option from pip install
2022-05-29 09:40:37 +10:00
Matthias Mair
e1d22f538d resort imports 2022-05-20 17:24:51 +02:00
Matthias Mair
93b6934127 fix trailing 2022-05-20 13:20:55 +02:00
Oliver Walters
7e6d3d81b9 Update dockerfile to 3.14 2022-05-16 21:47:09 +10:00
Oliver Walters
495798dc98 Install libwebp-dev as part of dockerfile 2022-05-15 23:20:12 +10:00
Oliver Walters
bb5d6815df Mark inventree home directory as safe for git.
Ref: https://github.blog/2022-04-12-git-security-vulnerability-announced/
2022-05-10 23:24:32 +10:00
Oliver
86f562f3b8 Add gdk-pixbuf
- Required for weasyprint (reports) to render jpg files
2022-04-27 13:46:50 +10:00
Oliver Walters
0cd5a2b005 Revert location of init.sh for Dockerfile 2022-04-21 20:17:12 +10:00
Oliver
6d56dbd266 Adds project names for docker-compose files 2022-04-21 12:04:05 +10:00
Oliver Walters
67196b866e Compose script fixes
- Define INVENTREE_DB_NAME
- Cache built image
2022-04-21 01:13:34 +10:00
Oliver Walters
a376625088 Force INVENTREE_DB_USER and INVENTREE_DB_PASSWORD to be set manually
- Only for the "production" docker setup
2022-04-21 01:04:46 +10:00
Oliver Walters
ced9c6c050 Sqlite docker-compose file no longer requires .env file
As this is only used for CI this is OK
2022-04-21 01:02:55 +10:00
Oliver Walters
0974fde59e Add sqlite-config file back in
- required for CI steps
- might refactor this out later on
2022-04-21 00:34:24 +10:00