InvenTree/.github/actions/setup/action.yaml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

102 lines
3.3 KiB
YAML
Raw Permalink Normal View History

name: 'Setup Enviroment'
description: 'Setup the environment for general InvenTree tests'
author: 'InvenTree'
inputs:
python:
required: false
description: 'Install python.'
default: 'true'
npm:
required: false
description: 'Install npm.'
default: 'false'
install:
required: false
description: 'Install the InvenTree requirements?'
default: 'false'
dev-install:
required: false
description: 'Install the InvenTree development requirements?'
default: 'false'
update:
required: false
description: 'Should a full update cycle be run?'
default: 'false'
apt-dependency:
required: false
description: 'Extra APT package for install.'
pip-dependency:
required: false
description: 'Extra python package for install.'
runs:
using: 'composite'
steps:
- name: Checkout Code
2023-10-24 22:18:53 +00:00
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # pin@v4.1.1
# Python installs
- name: Set up Python ${{ env.python_version }}
if: ${{ inputs.python == 'true' }}
uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # pin@v5.0.0
with:
python-version: ${{ env.python_version }}
cache: pip
cache-dependency-path: |
src/backend/requirements.txt
src/backend/requirements-dev.txt
contrib/container/requirements.txt
contrib/dev_reqs/requirements.txt
- name: Install Base Python Dependencies
if: ${{ inputs.python == 'true' }}
shell: bash
run: |
python3 -m pip install -U pip
pip3 install -U invoke wheel
pip3 install 'uv<0.3.0'
- name: Allow uv to use the system Python by default
run: echo "UV_SYSTEM_PYTHON=1" >> $GITHUB_ENV
shell: bash
- name: Install Specific Python Dependencies
if: ${{ inputs.pip-dependency }}
shell: bash
run: uv pip install ${{ inputs.pip-dependency }}
# NPM installs
- name: Install node.js ${{ env.node_version }}
if: ${{ inputs.npm == 'true' }}
2023-10-24 22:18:53 +00:00
uses: actions/setup-node@1a4442cacd436585916779262731d5b162bc6ec7 # pin to v3.8.2
with:
node-version: ${{ env.node_version }}
cache: 'npm'
cache-dependency-path: src/backend/package-lock.json
- name: Install npm packages
if: ${{ inputs.npm == 'true' }}
shell: bash
run: cd src/backend && npm install
# OS installs
- name: Install OS Dependencies
if: ${{ inputs.apt-dependency }}
shell: bash
run: |
sudo apt-get update
sudo apt-get install ${{ inputs.apt-dependency }}
sudo apt-get install ${{ inputs.apt-dependency }}
# Invoke commands
- name: Install dev requirements
if: ${{ inputs.dev-install == 'true' || inputs.install == 'true' }}
shell: bash
run: uv pip install --require-hashes -r src/backend/requirements-dev.txt
- name: Run invoke install
if: ${{ inputs.install == 'true' }}
shell: bash
run: invoke install --uv
- name: Run invoke update
if: ${{ inputs.update == 'true' }}
shell: bash
[WIP] Data importer (#6911) * Adds new model for DataImportSession * Add file extension validation Expose to admin interface also * Switch to new 'importer' app * Refactoring to help prevent circular imports * Add serializer registry - Use @register_importer tag for any serializer class * Cleanup migration file - Do not use one-time hard-coded values here * Refactor code into registry.py * Add validation for the uploaded file - Must be importable by tablib * Refactoring * Adds property to retrieve matching serializer class * Update helper functions * Add hook to auto-assign columns on initial creation * Rename field * Enforce initial status value * Add model for individual rows in the data import * Add DataImportRow model * Extract data rows as dict * Update fields - Remove "progress" field (will be calculated) - Added "timestamp" field - Added "complete" field to DataImportRow * Auto-map column names - Provide "sensible" default values * Add API endpoint for DataImportSession * Offload data import operation - For large data files this may take a significant amount of time - Offload it to the background worker process * Refactor data import code * Update models - Add "columns" field to DataImportSession - Add "errors" field to DataImportRow * Move field mapping to a new model type - Simpler validation * Save "valid" status for each data row * Include session defaults when validating row data * Update content_excludes - Ignore importer models in import/export * Remove port from ALLOWED_HOST entries * Skip table events for importer models * Bug fixes * Serializer updates * Add more endpoints - DataImportColumnMappingList - DataImportRowList * further updates: - Add 'get_api_url' method - Handle case where * Expose "available fields" to the DataImportSession serializer Uses the (already available) inventree metadata middleware * Add detail endpoints * Clear existing column mappings * Add endpoint for accepting column mappings * Add API endpoint exposing available importer serializers * Add simple playground area for testing data importer * Adds simple form to start new import session - Needs work, file field does not currently function correctly * data_file is *not* read_only * Add check for file type * Remove debug statements * Refactor column mapping - Generate mapping for each column - Remove "columns" field - Column names are calculated dynamically * Fix uniqueness requirements on mapping table * Admin updates - Prevent deletion of mappings - Prevent addition of mappings * API endpoint updates - Prevent mappings from being deleted - Prevent mappings from being created * Update importer drawer * Add widget for selecting data columns * UI tweaks * Delete import session when closing modal * Allow empty string value * Complete column mapping * Adds ability to remove rows * Adjust drawer specs * Add column 'description' to serializer * Add option to hide labels in API form field * Update column heading * Fix frontend linting errors * Revert drawer position * Return correct type * Fix shadowing * Fix f-string * simplify frontend code * Move importer app * Update API version * Reintroduce export formats * Add new models to RuleSet * typescript cleanup * Typescript cleanup * Improvement for Switch / boolean field * Display original row data on popover * Only display mapped columns * Add DataExportMixin class - Replaces existing APIDownloadMixin - Uses DRF serializers for exporting - *much* more efficient * Create new file: importer.mixins.py * Add new mixin to existing views which support data export * Better error handling * Cleanup: - Remove references to APIDownloadMixin - Remove download_queryset method - All now handled by API-based export functionality * Replace table with InvenTreeTable - Paginate imported rows - Data can be searched, ordered, * Make 'pathstring' fields read-only * Expose list of valid importer types to the API * Exclude read-only fields * Cleanup * Updates for session model - Column is now editable on mapping object - Field is no longer editable - Improve admin integration * Adds new custom hook for controlling data import session * Refactor column mapping widget * Refactor ImportDataSelector * Working on ImportDataSelector component * Adds method for editing fields in import table - Cell edit mode - Row edit mode - Form submission still needs work! * Adds background task for removing old import sessions * Fix api_version.py * Update src/frontend/src/components/importer/ImportDataSelector.tsx Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com> * Update model verbose names * Rename mixin class * Add serializer mixin classes - Will allow for fine-tuning of the import/export proces * @register_importer requires specific mixin * Check subclass for export * Fix typos * Refactor export serializer - Keep operations local to the class * Add shim class to process an individual row before exporting it * Add mixin to existing serializers * Add export functionality for company serializers * Adds placeholder for custom admin class * Update mantine styling * spacing -> gap * Add functionality to pre-process form data before upload * Remove old references to download_queryset * Improvements for data import drawer: - Pin title at top of drawer * Further improvements * Fix column selection input * Formatting improvements * Use a <Stepper> component for better progress display * Cleanup text * Add export-only fields to BuildItem queryset * Expand "export" fields for BuildItem dataset * Skip backup and static steps in CI * Remove hard-coded paths * Fix for "accept_mapping" method * Present required fields first on import session * Add "get_importable_fields" method * Add method for commiting imported row to database * Cleanup * Save "complete" state after row import * Allow prevention of column caching * Remove debug statement * Add basic admin table for import sessions * Fix for table filter functions - New mantine version requires string values * Add filters for import session table * Remove debug message * fix for <FilterItem /> * Create new import session from admin page * Cleanup playground * Re-open an existing import session * Memoize cell value * Update <ImportDataSelector> * Enable download of build line data * Add extra detail fields * Register data importers for the stock app * Enable download of stock item tracking data * Register importerrs for "company" app * Register importers for the "order" app * Add extra fields to purchase order line item serializer * Update verbose names for order models * Cleanup import data table rendering * Pass session information through to cell renderer * add separate 'field_overrides' field * Expose 'field_overrides' to API * Refactor import field selection * Use override data if provided * Fix data extraction - Ignore columns which are not mapped * Fix fields.pop - Provide 'None' argument * Update import data rendering * Handle missing / empty column names when importing data * Bug fixin' * Update hook * Adds button to upload data straight to table * Cache "available_fields" - Reduces API access time by 85% * Fix calculation of completed_row_count * Import individual rows from import session * Allow import of multiple simultaneous records * Improve extraction of metadata - Especially for related fields - Request object no longer required * Implement suspended rendering of model instances * Cleanup * Implement more columns for StockTable * Allow stock filtering by packaging field * Fix "stock_value" column * Improve metadata extraction - Handle read_only_fields in Meta - Handle write_only_fields in Meta * Increase maximum number of importable rows * Force data import to run on background worker * Add export-only fields to StockItemSerializer class * Data conversion when performing initial import * Various tweaks * Fix order of operations for data import * Rename component * Allow import/export of more model types * Fix verbose name * Import rows as a bulk db operation * Enable download for PartCategoryTemplateTable * Update stock item export * Updates for unit tests * Remove xls format for now - Causes some bug in tablib - Surely xlsx is OK? * More unit test updates * Future proof migration * Updates * unit tests * Unit test fix * Remove 'field_overrides' - field_defaults will suffice * Remove 'xls' as download option from frontend * Add simple unit test for data import * PUI tweaks --------- Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
2024-07-06 08:29:52 +00:00
run: invoke update --uv --skip-backup --skip-static