* Add custom user defined states
* make tests more reliable
* fix list options
* Adapt version
* do not engage if rebuilding
* remove unneeded attr
* remove unneeded attr
* fix enum imports
* adapt cove target
* Add status_custom_key to all other serializers
* fix serializer method
* simplify branching
* remove unneeded imports
* inherit read_only status from leader field
* Add more tests
* fix tests
* add test for function
* refactor for easier testing
* move test to seperate class
* Add options testing
* extend serializer
* add test for all states and refactor to reuse already build functions
* use custom field in PUI too
* reset diff
* style fix
* fix comparison
* Add test for str
* test color exceptions too
* remove user state from tracking
* Add intro from model fields too
* update docs
* simplify implementation
* update tests
* fix name
* rename test
* simplify tags and test fully
* extend test to machine status
* move logic for response formatting over
* extend api response with machine status
* ensure only direct subclasses are discovered
* test for length of total respone too
* use new fields on PUI too
* fix test assertion with plugins enabled
* also observe rendering in filters
* Add managment endpoints and APIs
* Add contenttypes to PUI renderes
* use filteres instead
* fix import order
* fix api route definition
* move status choices to serializer
* fix lookup
* fix filtering
* remove admin integration
* cleanup migration
* fix migration change
* cleanup code location
* fix imports
* Add docs for custom states
* add links to custom status
* Adds new model for DataImportSession
* Add file extension validation
Expose to admin interface also
* Switch to new 'importer' app
* Refactoring to help prevent circular imports
* Add serializer registry
- Use @register_importer tag for any serializer class
* Cleanup migration file
- Do not use one-time hard-coded values here
* Refactor code into registry.py
* Add validation for the uploaded file
- Must be importable by tablib
* Refactoring
* Adds property to retrieve matching serializer class
* Update helper functions
* Add hook to auto-assign columns on initial creation
* Rename field
* Enforce initial status value
* Add model for individual rows in the data import
* Add DataImportRow model
* Extract data rows as dict
* Update fields
- Remove "progress" field (will be calculated)
- Added "timestamp" field
- Added "complete" field to DataImportRow
* Auto-map column names
- Provide "sensible" default values
* Add API endpoint for DataImportSession
* Offload data import operation
- For large data files this may take a significant amount of time
- Offload it to the background worker process
* Refactor data import code
* Update models
- Add "columns" field to DataImportSession
- Add "errors" field to DataImportRow
* Move field mapping to a new model type
- Simpler validation
* Save "valid" status for each data row
* Include session defaults when validating row data
* Update content_excludes
- Ignore importer models in import/export
* Remove port from ALLOWED_HOST entries
* Skip table events for importer models
* Bug fixes
* Serializer updates
* Add more endpoints
- DataImportColumnMappingList
- DataImportRowList
* further updates:
- Add 'get_api_url' method
- Handle case where
* Expose "available fields" to the DataImportSession serializer
Uses the (already available) inventree metadata middleware
* Add detail endpoints
* Clear existing column mappings
* Add endpoint for accepting column mappings
* Add API endpoint exposing available importer serializers
* Add simple playground area for testing data importer
* Adds simple form to start new import session
- Needs work, file field does not currently function correctly
* data_file is *not* read_only
* Add check for file type
* Remove debug statements
* Refactor column mapping
- Generate mapping for each column
- Remove "columns" field
- Column names are calculated dynamically
* Fix uniqueness requirements on mapping table
* Admin updates
- Prevent deletion of mappings
- Prevent addition of mappings
* API endpoint updates
- Prevent mappings from being deleted
- Prevent mappings from being created
* Update importer drawer
* Add widget for selecting data columns
* UI tweaks
* Delete import session when closing modal
* Allow empty string value
* Complete column mapping
* Adds ability to remove rows
* Adjust drawer specs
* Add column 'description' to serializer
* Add option to hide labels in API form field
* Update column heading
* Fix frontend linting errors
* Revert drawer position
* Return correct type
* Fix shadowing
* Fix f-string
* simplify frontend code
* Move importer app
* Update API version
* Reintroduce export formats
* Add new models to RuleSet
* typescript cleanup
* Typescript cleanup
* Improvement for Switch / boolean field
* Display original row data on popover
* Only display mapped columns
* Add DataExportMixin class
- Replaces existing APIDownloadMixin
- Uses DRF serializers for exporting
- *much* more efficient
* Create new file: importer.mixins.py
* Add new mixin to existing views which support data export
* Better error handling
* Cleanup:
- Remove references to APIDownloadMixin
- Remove download_queryset method
- All now handled by API-based export functionality
* Replace table with InvenTreeTable
- Paginate imported rows
- Data can be searched, ordered,
* Make 'pathstring' fields read-only
* Expose list of valid importer types to the API
* Exclude read-only fields
* Cleanup
* Updates for session model
- Column is now editable on mapping object
- Field is no longer editable
- Improve admin integration
* Adds new custom hook for controlling data import session
* Refactor column mapping widget
* Refactor ImportDataSelector
* Working on ImportDataSelector component
* Adds method for editing fields in import table
- Cell edit mode
- Row edit mode
- Form submission still needs work!
* Adds background task for removing old import sessions
* Fix api_version.py
* Update src/frontend/src/components/importer/ImportDataSelector.tsx
Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
* Update model verbose names
* Rename mixin class
* Add serializer mixin classes
- Will allow for fine-tuning of the import/export proces
* @register_importer requires specific mixin
* Check subclass for export
* Fix typos
* Refactor export serializer
- Keep operations local to the class
* Add shim class to process an individual row before exporting it
* Add mixin to existing serializers
* Add export functionality for company serializers
* Adds placeholder for custom admin class
* Update mantine styling
* spacing -> gap
* Add functionality to pre-process form data before upload
* Remove old references to download_queryset
* Improvements for data import drawer:
- Pin title at top of drawer
* Further improvements
* Fix column selection input
* Formatting improvements
* Use a <Stepper> component for better progress display
* Cleanup text
* Add export-only fields to BuildItem queryset
* Expand "export" fields for BuildItem dataset
* Skip backup and static steps in CI
* Remove hard-coded paths
* Fix for "accept_mapping" method
* Present required fields first on import session
* Add "get_importable_fields" method
* Add method for commiting imported row to database
* Cleanup
* Save "complete" state after row import
* Allow prevention of column caching
* Remove debug statement
* Add basic admin table for import sessions
* Fix for table filter functions
- New mantine version requires string values
* Add filters for import session table
* Remove debug message
* fix for <FilterItem />
* Create new import session from admin page
* Cleanup playground
* Re-open an existing import session
* Memoize cell value
* Update <ImportDataSelector>
* Enable download of build line data
* Add extra detail fields
* Register data importers for the stock app
* Enable download of stock item tracking data
* Register importerrs for "company" app
* Register importers for the "order" app
* Add extra fields to purchase order line item serializer
* Update verbose names for order models
* Cleanup import data table rendering
* Pass session information through to cell renderer
* add separate 'field_overrides' field
* Expose 'field_overrides' to API
* Refactor import field selection
* Use override data if provided
* Fix data extraction
- Ignore columns which are not mapped
* Fix fields.pop
- Provide 'None' argument
* Update import data rendering
* Handle missing / empty column names when importing data
* Bug fixin'
* Update hook
* Adds button to upload data straight to table
* Cache "available_fields"
- Reduces API access time by 85%
* Fix calculation of completed_row_count
* Import individual rows from import session
* Allow import of multiple simultaneous records
* Improve extraction of metadata
- Especially for related fields
- Request object no longer required
* Implement suspended rendering of model instances
* Cleanup
* Implement more columns for StockTable
* Allow stock filtering by packaging field
* Fix "stock_value" column
* Improve metadata extraction
- Handle read_only_fields in Meta
- Handle write_only_fields in Meta
* Increase maximum number of importable rows
* Force data import to run on background worker
* Add export-only fields to StockItemSerializer class
* Data conversion when performing initial import
* Various tweaks
* Fix order of operations for data import
* Rename component
* Allow import/export of more model types
* Fix verbose name
* Import rows as a bulk db operation
* Enable download for PartCategoryTemplateTable
* Update stock item export
* Updates for unit tests
* Remove xls format for now
- Causes some bug in tablib
- Surely xlsx is OK?
* More unit test updates
* Future proof migration
* Updates
* unit tests
* Unit test fix
* Remove 'field_overrides'
- field_defaults will suffice
* Remove 'xls' as download option from frontend
* Add simple unit test for data import
* PUI tweaks
---------
Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
Matthias Mair
Oliver