* Adds new model for DataImportSession
* Add file extension validation
Expose to admin interface also
* Switch to new 'importer' app
* Refactoring to help prevent circular imports
* Add serializer registry
- Use @register_importer tag for any serializer class
* Cleanup migration file
- Do not use one-time hard-coded values here
* Refactor code into registry.py
* Add validation for the uploaded file
- Must be importable by tablib
* Refactoring
* Adds property to retrieve matching serializer class
* Update helper functions
* Add hook to auto-assign columns on initial creation
* Rename field
* Enforce initial status value
* Add model for individual rows in the data import
* Add DataImportRow model
* Extract data rows as dict
* Update fields
- Remove "progress" field (will be calculated)
- Added "timestamp" field
- Added "complete" field to DataImportRow
* Auto-map column names
- Provide "sensible" default values
* Add API endpoint for DataImportSession
* Offload data import operation
- For large data files this may take a significant amount of time
- Offload it to the background worker process
* Refactor data import code
* Update models
- Add "columns" field to DataImportSession
- Add "errors" field to DataImportRow
* Move field mapping to a new model type
- Simpler validation
* Save "valid" status for each data row
* Include session defaults when validating row data
* Update content_excludes
- Ignore importer models in import/export
* Remove port from ALLOWED_HOST entries
* Skip table events for importer models
* Bug fixes
* Serializer updates
* Add more endpoints
- DataImportColumnMappingList
- DataImportRowList
* further updates:
- Add 'get_api_url' method
- Handle case where
* Expose "available fields" to the DataImportSession serializer
Uses the (already available) inventree metadata middleware
* Add detail endpoints
* Clear existing column mappings
* Add endpoint for accepting column mappings
* Add API endpoint exposing available importer serializers
* Add simple playground area for testing data importer
* Adds simple form to start new import session
- Needs work, file field does not currently function correctly
* data_file is *not* read_only
* Add check for file type
* Remove debug statements
* Refactor column mapping
- Generate mapping for each column
- Remove "columns" field
- Column names are calculated dynamically
* Fix uniqueness requirements on mapping table
* Admin updates
- Prevent deletion of mappings
- Prevent addition of mappings
* API endpoint updates
- Prevent mappings from being deleted
- Prevent mappings from being created
* Update importer drawer
* Add widget for selecting data columns
* UI tweaks
* Delete import session when closing modal
* Allow empty string value
* Complete column mapping
* Adds ability to remove rows
* Adjust drawer specs
* Add column 'description' to serializer
* Add option to hide labels in API form field
* Update column heading
* Fix frontend linting errors
* Revert drawer position
* Return correct type
* Fix shadowing
* Fix f-string
* simplify frontend code
* Move importer app
* Update API version
* Reintroduce export formats
* Add new models to RuleSet
* typescript cleanup
* Typescript cleanup
* Improvement for Switch / boolean field
* Display original row data on popover
* Only display mapped columns
* Add DataExportMixin class
- Replaces existing APIDownloadMixin
- Uses DRF serializers for exporting
- *much* more efficient
* Create new file: importer.mixins.py
* Add new mixin to existing views which support data export
* Better error handling
* Cleanup:
- Remove references to APIDownloadMixin
- Remove download_queryset method
- All now handled by API-based export functionality
* Replace table with InvenTreeTable
- Paginate imported rows
- Data can be searched, ordered,
* Make 'pathstring' fields read-only
* Expose list of valid importer types to the API
* Exclude read-only fields
* Cleanup
* Updates for session model
- Column is now editable on mapping object
- Field is no longer editable
- Improve admin integration
* Adds new custom hook for controlling data import session
* Refactor column mapping widget
* Refactor ImportDataSelector
* Working on ImportDataSelector component
* Adds method for editing fields in import table
- Cell edit mode
- Row edit mode
- Form submission still needs work!
* Adds background task for removing old import sessions
* Fix api_version.py
* Update src/frontend/src/components/importer/ImportDataSelector.tsx
Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
* Update model verbose names
* Rename mixin class
* Add serializer mixin classes
- Will allow for fine-tuning of the import/export proces
* @register_importer requires specific mixin
* Check subclass for export
* Fix typos
* Refactor export serializer
- Keep operations local to the class
* Add shim class to process an individual row before exporting it
* Add mixin to existing serializers
* Add export functionality for company serializers
* Adds placeholder for custom admin class
* Update mantine styling
* spacing -> gap
* Add functionality to pre-process form data before upload
* Remove old references to download_queryset
* Improvements for data import drawer:
- Pin title at top of drawer
* Further improvements
* Fix column selection input
* Formatting improvements
* Use a <Stepper> component for better progress display
* Cleanup text
* Add export-only fields to BuildItem queryset
* Expand "export" fields for BuildItem dataset
* Skip backup and static steps in CI
* Remove hard-coded paths
* Fix for "accept_mapping" method
* Present required fields first on import session
* Add "get_importable_fields" method
* Add method for commiting imported row to database
* Cleanup
* Save "complete" state after row import
* Allow prevention of column caching
* Remove debug statement
* Add basic admin table for import sessions
* Fix for table filter functions
- New mantine version requires string values
* Add filters for import session table
* Remove debug message
* fix for <FilterItem />
* Create new import session from admin page
* Cleanup playground
* Re-open an existing import session
* Memoize cell value
* Update <ImportDataSelector>
* Enable download of build line data
* Add extra detail fields
* Register data importers for the stock app
* Enable download of stock item tracking data
* Register importerrs for "company" app
* Register importers for the "order" app
* Add extra fields to purchase order line item serializer
* Update verbose names for order models
* Cleanup import data table rendering
* Pass session information through to cell renderer
* add separate 'field_overrides' field
* Expose 'field_overrides' to API
* Refactor import field selection
* Use override data if provided
* Fix data extraction
- Ignore columns which are not mapped
* Fix fields.pop
- Provide 'None' argument
* Update import data rendering
* Handle missing / empty column names when importing data
* Bug fixin'
* Update hook
* Adds button to upload data straight to table
* Cache "available_fields"
- Reduces API access time by 85%
* Fix calculation of completed_row_count
* Import individual rows from import session
* Allow import of multiple simultaneous records
* Improve extraction of metadata
- Especially for related fields
- Request object no longer required
* Implement suspended rendering of model instances
* Cleanup
* Implement more columns for StockTable
* Allow stock filtering by packaging field
* Fix "stock_value" column
* Improve metadata extraction
- Handle read_only_fields in Meta
- Handle write_only_fields in Meta
* Increase maximum number of importable rows
* Force data import to run on background worker
* Add export-only fields to StockItemSerializer class
* Data conversion when performing initial import
* Various tweaks
* Fix order of operations for data import
* Rename component
* Allow import/export of more model types
* Fix verbose name
* Import rows as a bulk db operation
* Enable download for PartCategoryTemplateTable
* Update stock item export
* Updates for unit tests
* Remove xls format for now
- Causes some bug in tablib
- Surely xlsx is OK?
* More unit test updates
* Future proof migration
* Updates
* unit tests
* Unit test fix
* Remove 'field_overrides'
- field_defaults will suffice
* Remove 'xls' as download option from frontend
* Add simple unit test for data import
* PUI tweaks
---------
Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
* bump pre-commit tools
* move uv config to pyproject
* style fix
* bump pre-commit deps again
* add config for code spell
* spelling fixes
* bump regex
* bump pre-commit
* bump versions again
* roll back regex version
* reverse uv bump
* compile for 3.12
* use 3.12 for ci
* remove unused env vars
* only cache main version
* set caching dep path
* remove caching distinction
* pip installl regex
* switch back to 3.9 default
* remove install test
* reset versions
* remove tomli
* Update playwright.config.ts
---------
Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
* Add basic model for handling generic attachments
* Refactor migration
* Data migration to convert old files across
* Admin updates
* Increase comment field max_length
* Adjust field name
* Remove legacy serializer classes / endpoints
* Expose new model to API
* Admin site list filters
* Remove legacy attachment models
- Add new mixin class to designate which models can have attachments
* Update data migration
- Ensure other apps are at the correct migration state beforehand
* Add migrations to remove legacy attachment tables
* Fix for "rename_attachment" callback
* Refactor model_type field
- ContentType does not allow easy API serialization
* Set allowed options for admin
* Update model verbose names
* Fix logic for file upload
* Add choices for serializer
* Add API filtering
* Fix for API filter
* Fix for attachment tables in PUI
- Still not solved permission issues
* Bump API version
* Record user when uploading attachment via API
* Refactor <AttachmentTable /> for PUI
* Display 'file_size' in PUI attachment table
* Fix company migrations
* Include permission informtion in roles API endpoint
* Read user permissions in PUI
* Simplify permission checks for <AttachmentTable />
* Automatically clean up old content types
* Cleanup PUI
* Fix typo in data migration
* Add reverse data migration
* Update unit tests
* Use InMemoryStorage for media files in test mode
* Data migration unit test
* Fix "model_type" field
- It is a required field after all
* Add permission check for serializer
* Fix permission check for CUI
* Fix PUI import
* Test python lib against specific branch
- Will be reverted once code is merged
* Revert STORAGES setting
- Might be worth looking into again
* Fix part unit test
* Fix unit test for sales order
* Use 'get_global_setting'
* Use 'get_global_setting'
* Update setting getter
* Unit tests
* Tweaks
* Revert change to settings.py
* More updates for get_global_setting
* Relax API query count requirement
* remove illegal chars and add unit tests
* Fix unit tests
* Fix frontend unit tests
* settings management updates
* Prevent db write under more conditions
* Simplify settings code
* Pop values before creating filters
* Prevent settings write under certain conditions
* Add debug msg
* Clear db on record import
* Refactor permissions checks
- Allows extension / customization of permission checks at a later date
* Unit test updates
* Prevent delete of attachment without correct permissions
* Adjust odcker.yaml
* Cleanup data migrations
* Tweak migration tests for build app
* Update data migration
- Handle case with missing data
* Prevent debug shell in TESTING mode
* Update migration dependencies
- Ensure all apps are "up to date" before removing legacy tables
* add file size test
* Update migration tests
* Revert some settings caching changes
* Fix incorrect logic in migration
* Update unit tests
* Prevent create on CURRENCY_CODES
- Seems to play havoc with bootup sequence
* Fix unit test
* Some refactoring
- Use get_global_setting
* Fix typo
* Revert change
* Add "tags" and "metadata"
* Include "tags" field in API serializer
* add "metadata" endpoint for attachments
* Fix sales order shipment / completed buttons
- Template logic needed tweaks
* Adjust logic for completing a sales order
* Tweak UV version
- Trying to get CI to run properly
* Pin version
* Ignore uv
* Cleanup
* Fix another --uv command
* use global pin for requests
* unify on yaml for workflo files
* format workflow files
* pin action versions
* fix pinned version
* use system venv
* switch args
* remove uv for now and add setting for pyyaml
* use requirements file
* also switch on docker flow
* generate hashes
* added hashes to reqs
* add hashes for CI too
* add hash checking
* require hashes everywhere possible
* require hashes where possible in docker
* add security commitment
* fix badge path
* set token permissions
see https://github.com/inventree/InvenTree/security/code-scanning/48
* add more chapters
* break up flow text
* spellchecking
* clean diff
* bump setup-python to node 20 version
* fix docker version too
* remove docker-sqlite file
- Do not want to encourage use of sqlite
* Add Caddyfile
* Add default site URL to .env
- Matches Caddyfile
* Cleanup / simplify .env file
* Remove dev nginx conf file
* Further cleanup of .env file
* Update docker-compose.yml
- Use caddy image instead of nginx as proxy
* Set max body size
* gunicorn: enable external logging
* Update file structure
* Cleanup docker-compose file
* Update docker/docker-compose.yml
Co-authored-by: Matthias Mair <code@mjmair.com>
* Update docker/Caddyfile
Co-authored-by: Matthias Mair <code@mjmair.com>
* Fix for postgresql packages
- Need postgresql13-client to be installed, it contains pg_dump
- Without this, backup / restore *does not work*
* Create static_i18n dir if it does not exist
* Reduce output from collectstatic
* Revert gunicorn logging
- Want to see the logs in docker
* Fix trailing slash
Ref: https://github.com/inventree/InvenTree/pull/6551#issuecomment-1962423765
* tasks.py - pass 'nouv' option through
* Update package requirements:
- Allow installation of rapidfuzz without building
* Install uv as part of docker image
* Add environment variable to control downstream URL
* Do not use uv package manager by default
- Currently does not work "correctly" - ignores installed packages
- Requires further work to run reliably
* Fix docker-compose file
- Do not build locally
* Cleanup gunicorn file
- Remove unused lien
* Cleanup docker-compose.yml
- Simpler volume management
* Update Caddyfile
Add newline
* Update requirements.txt
Add newline
* Update tasks.py
Add missing blank line
* Simplify Caddyfile
* Adds option for customizing web port
* cleanup docker-compose.yml
- Better mapping of caddy data
- Cleaner volume setup
* Add django version template
- Ensure all docs links point to the current django version we are using
* docs: cleanup intro.md
* Cleanup serving_files.md
* Cleanup config.md
* docker install docs updates
* Enable code block copying
* Fix include file
* Fix link
* Update docker install docs
* Update docker.md
* Add info about demo dataset
* Tweak heading
* Update docs link checks
* Fix workflow
* Another fix
* More ignore pattearns
---------
Co-authored-by: Matthias Mair <code@mjmair.com>
* use default setup action
* small change to trigger CI
* use enviroment step
* define python version
* us uv a bit more
* clean diff
* globally define python version
* test UV performance
* second dummy change
* fix CI
* set version - uv is already proving to be great
* dummy bump
* bump to newer uv
* use uv in more places
* revert uv
* add flag to disable UV
* fix vevn for CI
* bump uv version
* bump to uv-016
* bump
* bump
* switch to uv
* Remove stat context variables
* Revert "Remove stat context variables"
This reverts commit 0989c308d0.
* remove old backport
* move requirements
* move isort
* move migrations-tests
* add pip-tools as req and add hook
* Use hashes and also run when txt files are added
* Update to hashes in requirements
* update requirements install path
* remove allauth-2fa for a test
* install dev-requirements
* re-enable allauth
* remove hashes from pre normal setup
* Remove version bindings if they are the laest
* move depencies to upper bound
* fix django-money
* remove windows specific requirements
* also install dependencies
* update markdownify
* remove django-admin-shell
* remove unneeded packages
* add docs
* remove pins and fix pre-commit pipeline
* pin django-money
* really remove admin shell
* return default behaviour for money serializer
* pip fix
* update filter name
* set decimal for output
* add migrations
* pin coveralls
* update requirements
Co-authored-by: Oliver Walters <oliver.henry.walters@gmail.com>
* Add pre-commit to the stack
* exclude static
* Add locales to excludes
* fix style errors
* rename pipeline steps
* also wait on precommit
* make template matching simpler
* Use the same code for python setup everywhere
* use step and cache for python setup
* move regular settings up into general envs
* just use full update
* Use invoke instead of static references
* make setup actions more similar
* use python3
* refactor names to be similar
* fix runner version
* fix references
* remove incidential change
* use matrix for os
* Github can't do this right now
* ignore docstyle errors
* Add seperate docstring test
* update flake call
* do not fail on docstring
* refactor setup into workflow
* update reference
* switch to action
* resturcture
* add bash statements
* remove os from cache
* update input checks
* make code cleaner
* fix boolean
* no relative paths
* install wheel by python
* switch to install
* revert back to simple wheel
* refactor import export tests
* move setup keys back to not disturbe tests
* remove docstyle till that is fixed
* update references
* continue on error
* use relativ action references
* Change step / job docstrings
