Commit Graph

21 Commits

Author SHA1 Message Date
Oliver 2c294d6ebe
Sales order improvements (#8445)
* Migration for SalesOrderAllocation

- Allow allocation against order with null shipment

* Enhaced query efficiency

* Further API cleanup

* Adjust serializer

* PUI updates

* Enable editing of allocation shipment

* Improve shipment filtering

* Add sub-table for salesorderlineitem

* Add helper method to SalesOrder to return pending SalesOrderAllocations

* Fix for CUI

* Update form for CUI

* Prevent SalesOrder completion with incomplete allocations

* Fixes for StockItem API

* Frontend refactoring

* Code cleanup

* Annotate shipment information to SalesOrder API endpoint

* Update frontend PUI

* Additional filtering for SalesOrderAllocation

* Bump API version

* Hide panel based on user permissions

* js linting

* Unit test fix

* Update playwright tests

* Revert diff

* Disable playwright test (temporary)

* View output from build table
2024-11-08 23:05:24 +11:00
Oliver 504655c92d
Fix for potential MissingKey error (#8422) 2024-11-03 17:42:24 +11:00
Oliver 8822194b0d
[PUI] stock table (#8396)
* [PUI] Add extra fields to StockItemTable

* Add SKU and MPN to StockItem export data

* Enable sorting

* Bump API version
2024-10-31 12:34:28 +11:00
Oliver 17bf7bb5e3
[Plugin] Enhancements for serial number validation (#8311)
* Error handling for plugin.increment_serial_number

* Improve error handling for plugin validation functions

* Add Part.get_next_serial_number method

- Simplify call for incrementing

* ValidationMixin: add "part" to increment_serial_number func

* Pass part information through to serial number functions

* Fix circular imports

* Allow "get_latest_serial_number" to use plugin system

* Better working example for plugin sample

* Update SampleValidatorPlugin

* Fix indent

* Add code comment

* Cleanup code logic

* Revert previous commit

* Update unit tests
2024-10-18 13:45:35 +11:00
Oliver 48dd3d37c4
Bug fix: Manually create serialized stock (#8306)
* Bug fix: Manually create serialized stock

- fixes bug which extraced location as a string, rather than the model instance

* Fix for StockList.create
2024-10-17 19:45:07 +11:00
Oliver 7443d21854
[Refactor] Serial generation (#8246)
* Refactor "update_serial_number" method

* Refactor serial number validation

- Query is much more efficient now
- Does not have to check each serial number individually
- Makes use of existing Part class method

* Refactor creation of multiple stock items

* Fix for singular item creation

* Refactor serializeStock method:

- Push "rebuild tree" to background worker
- Use bulk_create actions

* Refactor createion of serialized build outputs

* Prevent 1+N DB hits

* Cleanup

* Cleanup

* Reinstate serial number checks

* Add limit for serial number extraction

* Fix cache config

* Revert cache settings

* Fix for unit tests

* Playwright tests

* Bug fix

* Force False cookie mode in testing

* Revert aria-label for PanelGroup items

- No longer works as expected with playwright locators

* Fix playwright vtest

* Further updates

* Playwright test adjustments

* Remove duplicate locator
2024-10-12 10:08:57 +11:00
Matthias Mair 96a2517402
A bit more testing (#8053)
* add more admin testing

* fix assertations

* add test for importer admin

* Add tests for https://github.com/inventree/InvenTree/pull/7164

* add common/attachment test

* fix test

* add tests

* remove unused definition - the view is read only

* Revert "remove unused definition - the view is read only"

This reverts commit 4cad8d16f3.

* more tests in report

* Update tests.py

* make lookup dynamic

* make report assertation dynamic

* add migration test

* extend validation plugin tests

* disable flaky test

* Add test for barcode/uid transition

* test reverse migration

* cleanup new test

* remove empty action

* split and refactor API tests

* refactor test

* Add test for error conditions

* fix double entry

* more migration tests

* also test no history

* fix assertation

* add another migration test

* fix typo

* fix manufacturer filter

* test more filters

* even more filter tests

* move top level test to right place

* add todos for tests that could be more expressive

* add test for checking duplicate serials

* ignore cautious catches
2024-10-01 08:27:57 +10:00
Matthias Mair 1634258783
Enable more Ruff rules (#7930)
* bump version

* Add more checks

* add simplify rules

* Add RUF rules

* small perf imrpovements

* pylint checks

* more style fixes

* fix a number of A002 cases

* fix A001 cases

* disable unsafe fixes

* remove unneeded branches
fixes SIM102

* re-enable .keys for specific case

* Revert "remove unneeded branches"

This reverts commit f74d41bc07.

* fix reference
2024-08-27 09:04:55 +10:00
Matthias Mair 8eea8812e4
Add more liniting rules (mostly for imports) (#7846)
* remove unused imports

* enable pyflake checks

* various fixes

* fix assert
2024-08-10 08:12:58 +10:00
Lukas 32db71cabb
Make Tree Path searchable (#7786)
* make tree searchable by pathstring

* fix related model field colors in dark mode

* remove unused import

---------

Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
2024-08-02 12:17:34 +10:00
Miklós Márton 3cbfcc11cb
Added test statistics (#7164)
* Added test statistics

Fixed #5995

* Bump API version

* Fix javascript varible scopes

* Fix javascript exports

* Remove duplicated import

* Add files modified by the pre-commit scripts

* Move test statistics API urls to a separate endpoint

* Merge test-statistics urls

* Undo unrelated changes

* Formatting fix

* Fix API urls for test statistics in PUI

* Fix prefixing in test statistic functions

---------

Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
2024-07-31 16:23:00 +10:00
Oliver 1f6cd9fc54
[WIP] Data importer (#6911)
* Adds new model for DataImportSession

* Add file extension validation

Expose to admin interface also

* Switch to new 'importer' app

* Refactoring to help prevent circular imports

* Add serializer registry

- Use @register_importer tag for any serializer class

* Cleanup migration file

- Do not use one-time hard-coded values here

* Refactor code into registry.py

* Add validation for the uploaded file

- Must be importable by tablib

* Refactoring

* Adds property to retrieve matching serializer class

* Update helper functions

* Add hook to auto-assign columns on initial creation

* Rename field

* Enforce initial status value

* Add model for individual rows in the data import

* Add DataImportRow model

* Extract data rows as dict

* Update fields

- Remove "progress" field (will be calculated)
- Added "timestamp" field
- Added "complete" field to DataImportRow

* Auto-map column names

- Provide "sensible" default values

* Add API endpoint for DataImportSession

* Offload data import operation

- For large data files this may take a significant amount of time
- Offload it to the background worker process

* Refactor data import code

* Update models

- Add "columns" field to DataImportSession
- Add "errors" field to DataImportRow

* Move field mapping to a new model type

- Simpler validation

* Save "valid" status for each data row

* Include session defaults when validating row data

* Update content_excludes

- Ignore importer models in import/export

* Remove port from ALLOWED_HOST entries

* Skip table events for importer models

* Bug fixes

* Serializer updates

* Add more endpoints

- DataImportColumnMappingList
- DataImportRowList

* further updates:

- Add 'get_api_url' method
- Handle case where

* Expose "available fields" to the DataImportSession serializer

Uses the (already available) inventree metadata middleware

* Add detail endpoints

* Clear existing column mappings

* Add endpoint for accepting column mappings

* Add API endpoint exposing available importer serializers

* Add simple playground area for testing data importer

* Adds simple form to start new import session

- Needs work, file field does not currently function correctly

* data_file is *not* read_only

* Add check for file type

* Remove debug statements

* Refactor column mapping

- Generate mapping for each column
- Remove "columns" field
- Column names are calculated dynamically

* Fix uniqueness requirements on mapping table

* Admin updates

- Prevent deletion of mappings
- Prevent addition of mappings

* API endpoint updates

- Prevent mappings from being deleted
- Prevent mappings from being created

* Update importer drawer

* Add widget for selecting data columns

* UI tweaks

* Delete import session when closing modal

* Allow empty string value

* Complete column mapping

* Adds ability to remove rows

* Adjust drawer specs

* Add column 'description' to serializer

* Add option to hide labels in API form field

* Update column heading

* Fix frontend linting errors

* Revert drawer position

* Return correct type

* Fix shadowing

* Fix f-string

* simplify frontend code

* Move importer app

* Update API version

* Reintroduce export formats

* Add new models to RuleSet

* typescript cleanup

* Typescript cleanup

* Improvement for Switch / boolean field

* Display original row data on popover

* Only display mapped columns

* Add DataExportMixin class

- Replaces existing APIDownloadMixin
- Uses DRF serializers for exporting
- *much* more efficient

* Create new file: importer.mixins.py

* Add new mixin to existing views which support data export

* Better error handling

* Cleanup:

- Remove references to APIDownloadMixin
- Remove download_queryset method
- All now handled by API-based export functionality

* Replace table with InvenTreeTable

- Paginate imported rows
- Data can be searched, ordered,

* Make 'pathstring' fields read-only

* Expose list of valid importer types to the API

* Exclude read-only fields

* Cleanup

* Updates for session model

- Column is now editable on mapping object
- Field is no  longer editable
- Improve admin integration

* Adds new custom hook for controlling data import session

* Refactor column mapping widget

* Refactor ImportDataSelector

* Working on ImportDataSelector component

* Adds method for editing fields in import table

- Cell edit mode
- Row edit mode
- Form submission still needs work!

* Adds background task for removing old import sessions

* Fix api_version.py

* Update src/frontend/src/components/importer/ImportDataSelector.tsx

Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>

* Update model verbose names

* Rename mixin class

* Add serializer mixin classes

- Will allow for fine-tuning of the import/export proces

* @register_importer requires specific mixin

* Check subclass for export

* Fix typos

* Refactor export serializer

- Keep operations local to the class

* Add shim class to process an individual row before exporting it

* Add mixin to existing serializers

* Add export functionality for company serializers

* Adds  placeholder for custom admin class

* Update mantine styling

* spacing -> gap

* Add functionality to pre-process form data before upload

* Remove old references to download_queryset

* Improvements for data import drawer:

- Pin title at top of drawer

* Further improvements

* Fix column selection input

* Formatting improvements

* Use a <Stepper> component for better progress display

* Cleanup text

* Add export-only fields to BuildItem queryset

* Expand "export" fields for BuildItem dataset

* Skip backup and static steps in CI

* Remove hard-coded paths

* Fix for "accept_mapping" method

* Present required fields first on import session

* Add "get_importable_fields" method

* Add method for commiting imported row to database

* Cleanup

* Save "complete" state after row import

* Allow prevention of column caching

* Remove debug statement

* Add basic admin table for import sessions

* Fix for table filter functions

- New mantine version requires string values

* Add filters for import session table

* Remove debug message

* fix for <FilterItem />

* Create new import session from admin page

* Cleanup playground

* Re-open an existing import session

* Memoize cell value

* Update <ImportDataSelector>

* Enable download of build line data

* Add extra detail fields

* Register data importers for the stock app

* Enable download of stock item tracking data

* Register importerrs for "company" app

* Register importers for the "order" app

* Add extra fields to purchase order line item serializer

* Update verbose names for order models

* Cleanup import data table rendering

* Pass session information through to cell renderer

* add separate 'field_overrides' field

* Expose 'field_overrides' to API

* Refactor import field selection

* Use override data if provided

* Fix data extraction

- Ignore columns which are not mapped

* Fix fields.pop

- Provide 'None' argument

* Update import data rendering

* Handle missing / empty column names when importing data

* Bug fixin'

* Update hook

* Adds button to upload data straight to table

* Cache "available_fields"

- Reduces API access time by 85%

* Fix calculation of completed_row_count

* Import individual rows from import session

* Allow import of multiple simultaneous records

* Improve extraction of metadata

- Especially for related fields
- Request object no longer required

* Implement suspended rendering of model instances

* Cleanup

* Implement more columns for StockTable

* Allow stock filtering by packaging field

* Fix "stock_value" column

* Improve metadata extraction

- Handle read_only_fields in Meta
- Handle write_only_fields in Meta

* Increase maximum number of importable rows

* Force data import to run on background worker

* Add export-only fields to StockItemSerializer class

* Data conversion when performing initial import

* Various tweaks

* Fix order of operations for data import

* Rename component

* Allow import/export of more model types

* Fix verbose name

* Import rows as a bulk db operation

* Enable download for PartCategoryTemplateTable

* Update stock item export

* Updates for unit tests

* Remove xls format for now

- Causes some bug in tablib
- Surely xlsx is OK?

* More unit test updates

* Future proof migration

* Updates

* unit tests

* Unit test fix

* Remove 'field_overrides'

- field_defaults will suffice

* Remove 'xls' as download option from frontend

* Add simple unit test for data import

* PUI tweaks

---------

Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
2024-07-06 18:29:52 +10:00
Oliver 6cd0727714
Handle case where deltas is null (#7516) 2024-06-26 13:17:31 +10:00
Oliver 937824dceb
Top level cascade (#7514)
* Add "top_level" filter for PartCategory API endpoint

* Update CUI tables

* Update PUI table

* Similar updates for stock location table

* Fix "parent" field label

* Bump API version
2024-06-26 12:28:33 +10:00
Oliver 432e0c622c
Single table for file attachments (#7420)
* Add basic model for handling generic attachments

* Refactor migration

* Data migration to convert old files across

* Admin updates

* Increase comment field max_length

* Adjust field name

* Remove legacy serializer classes / endpoints

* Expose new model to API

* Admin site list filters

* Remove legacy attachment models

- Add new mixin class to designate which models can have attachments

* Update data migration

- Ensure other apps are at the correct migration state beforehand

* Add migrations to remove legacy attachment tables

* Fix for "rename_attachment" callback

* Refactor model_type field

- ContentType does not allow easy API serialization

* Set allowed options for admin

* Update model verbose names

* Fix logic for file upload

* Add choices for serializer

* Add API filtering

* Fix for API filter

* Fix for attachment tables in PUI

- Still not solved permission issues

* Bump API version

* Record user when uploading attachment via API

* Refactor <AttachmentTable /> for PUI

* Display 'file_size' in PUI attachment table

* Fix company migrations

* Include permission informtion in roles API endpoint

* Read user permissions in PUI

* Simplify permission checks for <AttachmentTable />

* Automatically clean up old content types

* Cleanup PUI

* Fix typo in data migration

* Add reverse data migration

* Update unit tests

* Use InMemoryStorage for media files in test mode

* Data migration unit test

* Fix "model_type" field

- It is a required field after all

* Add permission check for serializer

* Fix permission check for CUI

* Fix PUI import

* Test python lib against specific branch

- Will be reverted once code is merged

* Revert STORAGES setting

- Might be worth looking into again

* Fix part unit test

* Fix unit test for sales order

* Use 'get_global_setting'

* Use 'get_global_setting'

* Update setting getter

* Unit tests

* Tweaks

* Revert change to settings.py

* More updates for get_global_setting

* Relax API query count requirement

* remove illegal chars and add unit tests

* Fix unit tests

* Fix frontend unit tests

* settings management updates

* Prevent db write under more conditions

* Simplify settings code

* Pop values before creating filters

* Prevent settings write under certain conditions

* Add debug msg

* Clear db on record import

* Refactor permissions checks

- Allows extension / customization of permission checks at a later date

* Unit test updates

* Prevent delete of attachment without correct permissions

* Adjust odcker.yaml

* Cleanup data migrations

* Tweak migration tests for build app

* Update data migration

- Handle case with missing data

* Prevent debug shell in TESTING mode

* Update migration dependencies

- Ensure all apps are "up to date" before removing legacy tables

* add file size test

* Update migration tests

* Revert some settings caching changes

* Fix incorrect logic in migration

* Update unit tests

* Prevent create on CURRENCY_CODES

- Seems to play havoc with bootup sequence

* Fix unit test

* Some refactoring

- Use get_global_setting

* Fix typo

* Revert change

* Add "tags" and "metadata"

* Include "tags" field in API serializer

* add "metadata" endpoint for attachments
2024-06-19 14:38:46 +10:00
Oliver 79ea6897ea
Improve stock item tracking API query (#7451)
* Improve stock item tracking API query

- Cache related model lookups into single DB queries
- Significant improvements to query speed
- Ref: https://github.com/inventree/InvenTree/issues/7429

* Handle case where item does not exist in DB
2024-06-16 17:06:09 +10:00
Oliver 798c0ed322
PO shipped complete (#7367)
* Add new setting to bypass "shipped" status

* Bypass "shipped" status optionally

* Update setting description string

* Update unit tests

* Refactor location of status_codes

* Link source code into docs

* Add stock status codes

* And the build order too

* Fix import
2024-05-30 12:44:57 +10:00
Oliver f3223c6d7f
[PUI] Mantine tree (#7357)
* Refactor part category tree

- New "NavigationTree" using native mantine components
- Make it generic, too

* Replace existing <StockLocationTree /> component

* Adjust API filtering for location tree endpoint

* Added playwright tests

* Update api filter classes

* Fix for DetailsImage

- Update to @mantine/core had changed the <AspectRatio> component

* fix for identifierString function

* Adjust playwright tests
2024-05-28 14:24:21 +10:00
Oliver e93d9c4a74
Batch code generation (#7000)
* Refactor framework for generating batch codes

- Provide additional kwargs to plugin
- Move into new file
- Error handling

* Implement API endpoint for generating a new batch code

* Fixes

* Refactor into stock.generators

* Fix API endpoint

* Pass time context through to plugins

* Generate batch code when receiving items

* Create useGenerator hook

- Build up a dataset and query server whenever it changes
- Look for result in response data
- For now, just used for generating batch codes
- may be used for more in the future

* Refactor PurchaseOrderForms to use new generator hook

* Refactor StockForms implementation

* Remove dead code

* add OAS diff

* fix ref

* fix ref again

* wrong branch, sorry

* Update src/frontend/src/hooks/UseGenerator.tsx

Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>

* Bump API version

* Do not override batch code if already generated

* Add serial number generator

- Move to /generate/ API endpoint
- Move batch code generator too

* Update PUI endpoints

* Add debouncing to useGenerator hook

* Refactor useGenerator func

* Add serial number generator to stock form

* Add batch code genereator to build order form

* Update buildfields

* Use build batch code when creating new output

---------

Co-authored-by: Matthias Mair <code@mjmair.com>
Co-authored-by: Lukas <76838159+wolflu05@users.noreply.github.com>
2024-05-20 23:56:45 +10:00
Oliver 2ebe785a75
Remove dead code (#7265)
* Remove dead code

- Previously custom LIST was required
- Improvements render this unnecessary now

* Remove unused imports
2024-05-20 19:30:35 +10:00
Matthias Mair 0bace3f3af
Code structure refactor (#5582)
* moved docker files to /contrib/container

* changed code owners to make more precise

* updated CI to use new subdirs

* added manual trigger for testing

* moved ci files

* moved assets into subdir

* moved deploy template file to contrib

* moved django files to src/backend

* updated paths in scripts etc

* updated reqs path

* fixed version file path

* fixed flake8 path

* fixed path to node ressources

* fixed task paths

* added dep path for node

* removed unused yarn lockfile

* removed unused ci script

* updated internal backend paths for tasks

* updated translation stats path

* fixed source path for coverage

* fixed main commit repo path

* fit in changes from testing

* gather packager improvements (#149)

* Matmair/issue5578 (#143)

* moved docker files to /contrib/container

* changed code owners to make more precise

* updated CI to use new subdirs

* added manual trigger for testing

* moved ci files

* moved assets into subdir

* moved deploy template file to contrib

* moved django files to src/backend

* updated paths in scripts etc

* updated reqs path

* fixed version file path

* fixed flake8 path

* fixed path to node ressources

* fixed task paths

* added dep path for node

* removed unused yarn lockfile

* removed unused ci script

* updated internal backend paths for tasks

* updated translation stats path

* fixed source path for coverage

* fixed main commit repo path

* fix docker path

* use project dir

* move project dir command

* fixed docker paths

* another fix?

* seperate tasks out

* remove tasks

* some debugging

* ci: add .deepsource.toml

* Update .deepsource.toml

* also ignore migrations

* more debugging

* fix path issues

* remove debug script

* fix style

* change locale path

* Fixed paths for requirements

* Added dummy requirements to fool packager

* fixed exec path

* remove deepsource

---------

Co-authored-by: deepsource-io[bot] <42547082+deepsource-io[bot]@users.noreply.github.com>

* Added docs for file structure

* Fixed style errors

* updated deepsource paths

* fix deepsource paths

* fixed reqs

* merge fixes

* move newly added dirs too

* fix reqs files

* another dep fix

* merge upstream/master

* revert removal of tags

* merge upstream

* enabled detection of old config files

* adapt coverage src

* also detect and support old location for plugins.txt

* style fix

* fix ~/init.sh location

* fix requirements path

* fix config to current master

* move new folders

* fix import order

* fix paths for qc_check

* fix docs build

* fix fix path

* set docker project dir

* just use a cd

* set image path?

* set file correct

* fix copy path

* fix tasks dir

* fix init path

* fix copy path

* set prject dir

* fix paths

* remove old prod files

* fix dev env path

* set docker file

* Fix devcontainer docker compose file

* fix login attempt values

* fix init.sh path

* Fix pathing for Docker

* Docker build fix

- Set INVENTREE_BACKEND_DIR separately

* Update init.sh

* Fix path

* Update requirements.txt

* merge

* fix rq merge

* fix docker compose usage

---------

Co-authored-by: deepsource-io[bot] <42547082+deepsource-io[bot]@users.noreply.github.com>
Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
2024-04-03 12:16:59 +11:00