Commit Graph

150 Commits

Author SHA1 Message Date
Matthias Mair 3f93151cca
refactor: replace multiple `==` checks with `in` (#5737)
To check if a variable is equal to one of many values, combine the values into a tuple and check if the variable is contained `in` it instead of checking for equality against each of the values.
This is faster, less verbose, and more readable.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-10-18 07:30:16 +11:00
Matthias Mair faac6b6bf5
refactor: remove blank lines after docstring (#5736)
There shouldn't be any blank lines after the function docstring.
Remove the blank lines to fix this issue.

Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
2023-10-18 07:28:57 +11:00
Oliver 550d549325
Refactor plugin loading (#5726)
- Pre-fetch all PluginConfig objects
- Only update plugin config active status when changed
- Reference: https://github.com/inventree/InvenTree/issues/5586
2023-10-17 21:07:16 +11:00
Oliver a6dbe185c6
Allow reload of plugin registry when config changes (#5662)
* Allow reload of plugin registry when config changes

- Any global settings which control plugin behaviour trigger reload
- Plugin registry hash includes the value of these settings
- Plugin registry hash is now cached

* typo
2023-10-05 00:38:40 +11:00
Oliver 3d8e3f1625
Bug fix for plugin reload mechanism (#5660)
- Save updated hash after reload
- Use actual *value* of active, not the function object!
2023-10-04 23:05:48 +11:00
Oliver ffda700244
Improve unit conversion (#5658)
- In previous implementation, could not have a value "10k" for a unit of "m"
- Now, correct result is obtained (10k [m] = 10[km])
- Still passes all previous unit tests
- Simpler code, too
2023-10-04 17:28:20 +11:00
Oliver 06eb948528
Plugin reload mechanism (#5649)
* Plugin reload mechanism

- Wrap reload_plugins with mutex lock
- Add methods for calculating plugin registry hash

* Perform plugin reload at critical entry points to the registry

- Background worker will correctly reload registry before performing tasks
- Ensures that the background worker plugin regsistry is up  to date
2023-10-04 09:00:11 +11:00
Oliver 88314da7b3
Fix for plugin installation via web interface (#5646)
* Fix for plugin installation via web interface

- Previous implementation did not correctly catch error output
- Refactored into new file plugin.installer.py
- Provides better "output" when plugin does install correctly

* Remove unnecessary debug output

* Update unit test

* Remove placeholder code
2023-10-03 18:47:55 +11:00
Matthias Mair e767516c80
Move broken plugin samples to disabled folders (#5642)
* move broken samples to disabled folders

* ensure there are no name clashes

* added option to collect plugins on load

* made tests aware of reloads

* reverted test name change

* added tests for broken samples

* fixed type signature
2023-10-02 10:39:17 +11:00
Oliver 3cd7ee9cd2
Improve formatting of log messages in settings.py (#5636) 2023-09-30 23:20:55 +10:00
Matthias Mair 71c416bafd
Add flake8-logging linting (#5620)
* added flake8-logging to make logging more robust

* fixed LOG005 warnings

* fixed LOG008 warnings

* fixed LOG011 warnings

* fixed more LOG011 errors

* added ignores for intentional logger.error calls

* fixed even more LOG011 errors
2023-09-28 14:53:22 +10:00
Oliver 47f341d2b5
Plugin loading fixes (#5572)
* Add config function to return external plugins dir

* Enable AppMixin support relative to external plugins directory

* Fix for urls.py

- URL patterns were causing custom app mixin plugins to fail reverse lookup in admin interface
- Brought admin URLs up one level

* simplify urls.py

* Fix plugin registry code which registers plugin URLs

- As we have updated InvenTree.urls.py we need to adjust this logic too

* Adds redirect for favicon.ico

* Handle empty plugins dir
2023-09-20 13:32:34 +10:00
Lukas 073a275d89
Skip ready functions if not in main thread or plugins are not loaded yet (#5005)
* Skip ready functions if not in main thread or plugins are not loaded yet

* Debug integration tests

* Update ready.py

* Update ready.py

* Fix isInMainThread and isPluginRegistryLoaded ready functions

* Preload gunicorn app to only invoke the appconfig ready functions once

* debug: test prints for statistics

* Remove debug print

* Test without

* Revert "Test without"

This reverts commit 1bc1872893.

* Second test

* Add checks back to part, label, user model

* Add checks back to inventree, plugin apps

* log server output for debugging

* hopefully I can get the log this time+

* Next test

* Test with --noreload

* Next test

* trigger: ci, because session expired

* block the second ready execution instead of the first

* fix: load order

* Fix test and revert gh actions workflow change

* Added all_apps method to reload machanism

* Changed detect reload mechanism

* Also trigger ready on reload

* Add skipping second reload back for testing mode

* Added doc string back

* Update InvenTree/plugin/base/integration/AppMixin.py
2023-07-26 08:33:13 +10:00
Matthias Mair 2fc82fd0e3
Adds trigger when plugins are loaded (#5260)
* added trigger when plugins are loaded

* only trigger if db is ready
2023-07-21 08:56:19 +10:00
Oliver 3b4e20b54a
Unit Test Improvements (#5087)
* Disable migration testing

- Compare how long the unit tests take

* Change file

- To get unit tests to run

* Fix format

* Consolidate tasks.py

- Remove coverage task
- Add --coverage flag to test task

* Fix typo

* Run migration unit tests if migration files are updated

* Fix

* Touch migration file

- Should cause migration unit tests to be run

* Force migration checks for docker build

* Prevent default report creation in unit tests

- Should save some time

* Add simple profiling for plugin loading

- Display time taken to load each plugin

* Fix to invoke test

* Disable get_git_log (for testing)

* Disable get_git_path in CI

- Might remove this entirely?
- For now, bypass for unit testing

* Add debug for unit registry

- Display time taken to load registry

* Don't full-reload unit registry

* Adjust migration test workflow

- env var updates
- change paths-filter output

* Fix for migration_test.yaml

- Actually need to set the output

* env fix

* db name

* Prevent sleep if in test mode

* Reduce sleep time on wait_for_db
2023-06-23 17:25:59 +10:00
Matthias Mair 58a33c2e67
[FR] Switch from git to dulwich (#4966)
* [FR] Switch from pure git to dulwich Fixes #4942

* fix lenght

* change length again

* change length again
2023-06-05 19:27:46 +10:00
Matthias Mair 21ed4b2081
Fix common spelling mistakes (#4956)
* add codespell

* first fixes

* doc fixes

* fix docstrings and comments

* functional changes

* docstrings again

* and docs again

* rename args

* add ignore

* use pre-commit for filtering instead

* ups

* fix typo in filter
2023-06-04 00:04:52 +10:00
Oliver 9e77b9fc56
Parameter filtering (#4823)
* adds new field 'parameter type' to PartParameterTemplate model

* Move part parameter settings onto their own page

* Add "choices" and "regex" template types

* Adds validation for PartParameter based on template type

* javascript cleanup

* Fix for serializers.py

* Add unit testing for parameter validation

* Add filters

* Rename "type" field to "param_type"

- Should have seen that one coming

* Coerce 'boolean' value to True/False

* table update

* js linting

* Add requirement for "pint" package

* Add validator for physical unit types

- Revert a previous migration which adds "parameter type" and "validator" fields
- These will get implemented later, too much scope creep for this PR
- Add unit test for validation of "units" field

* Update PartParameter model

- Add data_numeric field (will be used later)
- Add MinLengthValidator to data field

* Run validation for part parameter data

- Ensure it can be converted to internal units

* Update admin interface to display partparameter values inline for a part

* Adds validation of part parameter data value

- Also converts to base units, and stores as "numeric" value
- Display "numeric" value in tables
- Create new file conversion.py for data conversion

* Update unit tests and fix some bugs

* Update docstring

* Add units to parameter columns in parameteric part table

* Allow part list to be ordered by a particular parameter value

- Annotate queryset with new "order_by_parameter" method
- Skeleton method for future work

* Bump API version

* Adds unit testing for sorting parts by parameter value

* Update historical data migrations

- Turns out RunPython.noop is a thing?

* Cache the unit registry

- Creating the unit registry takes a significant amount of time
- Construct when first called, and then cache for subsequent hits
- Massive improvement in performance

* Throw error on empty values when converting between units

* Data migration for converting existing part parameter values

* Handle more error cases

* Show parameteric table on top-level part page too

* Unit test for data migration

* Update credits in docs

* Improved error checking

* WIP docs updates

* Fix parameteric table filtering

* remove zoom property

* Fix for import path

* Update parameter docs

* Run background task to rebuild parameters when template changes

* Make "data_numeric" field nullable

- Defaulting to zero is not appropriate, as the actual value may be zero
- Sorting still seems to work just fine

* Fixes for unit test

* More unit test fixes

* Further fixes for unit tests

---------

Co-authored-by: Matthias Mair <code@mjmair.com>
2023-05-19 13:24:55 +10:00
Oliver b2ceac2c4a
[Feature] Scrap Build Outputs (#4800)
* Update docs for status codes

* Adds API endpoint for scrapping individual build outputs

* Support 'buildorder' reference in stock tracking history

* Add page for build output documentation

* Build docs

* Add example build order process to docs

* remove debug statement

* JS lint cleanup

* Add migration file for stock status

* Add unit tests for build output scrapping

* Increment API version

* bug fix
2023-05-13 22:19:35 +10:00
Matthias Mair 017ccaa27a
Cleanup plugin mixin registry (#4790)
* collect mixins dynamically

* remove unfinsihed option to reorder mixins

* clean up settings

* fix text

* fix mixin lookup

* stupid error

* fix assertations

* use regustered function instead of private dict

* switch to dict for reg

* fix test

* makke sure mixins also works with class

* cleanup

* fix reqs

* fix test assertations
2023-05-12 22:00:25 +10:00
Oliver 68d1682000
Report Plugins (#4643)
* Add plugin mixin class for extending reports

* Expose report context to the plugin system

* Add an example mixin for adding context data to a report

* Add the 'request' object to the plugin code
2023-04-20 14:21:22 +10:00
Matthias Mair 1b8ad70fb6
[FR] Refactor plugin registry (#4340)
* add mixin order ref

* move import

* fix import order

* reorder import

* move activation/deactivation to mixins

* move loaded/unloaded mixins out into seperate modules

* fix deactivation sequence

* switch to classmethods for loading

* only run (de)activation if defined for mixin
Fixes #4184

* fix deactivating

* move reloading back to registry

* fix merge error

* move app mixin deactivation

* fix migration reloading

* reverse deactivation sequence

* Revert "reverse deactivation sequence"

This reverts commit aff17dd07d.
2023-04-19 20:54:42 +10:00
Oliver f6b9b12745
Translation fixes (#4621)
* Remove HTML tags from translateable string

* Improve another string

* Improve strings for plugin version check

* Cleanup duplicate serials msg

* JS translations

* fmt
2023-04-18 00:12:14 +10:00
Matthias Mair 5037e427b6
Auto migrating (#3741)
* base structure for updates

* add base structure

* add settingscheck

* update docstring

* only load plugins if needed

* fix misstyping

* run migration

* check if there are open migrations

* log open migration

* add more logging

* patch in fore reloading on unload

* only run if database is ready

* check every 5 minutes

* remove non implemented feautres  from desc

* add command flag to makr if cmmand runs as worker

* Add tests for migrations

* factor mmigration plan into own function

* Add print statements

* add initial migrations for tests

* remove last assertation

* cleanup migrations after run

* add flag to accept empty source code files

* the flag is enough for reporting

* fix test

* do not run migrations on sqlite3

* make sure migrations don't fail if no plan ran

* increase coverage for  migration

* spell fix

* check for migrations daily

* add a migration check after plugins are installed
2023-02-25 16:52:16 +11:00
Oliver 0f445ea6e4
[Feature] Stocktake reports (#4345)
* Add settings to control upcoming stocktake features

* Adds migration for "cost range" when performing stocktake

* Add cost data to PartStocktakeSerializer

Implement a new custom serializer for currency data type

* Refactor existing currency serializers

* Update stocktake table and forms

* Prevent trailing zeroes in forms

* Calculate cost range when adding manual stocktake entry

* Display interactive chart for part stocktake history

* Ensure chart data are converted to common currency

* Adds new model for building stocktake reports

* Add admin integration for new model

* Adds API endpoint to expose list of stocktake reports available for download

- No ability to edit or delete via API

* Add setting to control automated deletion of old stocktake reports

* Updates for settings page

- Load part stocktake report table
- Refactor function to render a downloadable media file
- Fix bug with forcing files to be downloaded
- Split js code into separate templates
- Make use of onPanelLoad functionalitty

* Fix conflicting migration files

* Adds API endpoint for manual generation of stocktake report

* Offload task to generate new stocktake report

* Adds python function to perform stocktake on a single part instance

* Small bug fixes

* Various tweaks

- Prevent new stocktake models from triggering plugin events when created
- Construct a simple csv dataset

* Generate new report

* Updates for report generation

- Prefetch related data
- Add extra columns
- Keep track of stocktake instances (for saving to database later on)

* Updates:

- Add confirmation message
- Serializer validation checks

* Ensure that background worker is running before manually scheduling a new stocktake report

* Add extra fields to stocktake models

Also move code from part/models.py to part/tasks.py

* Add 'part_count' to PartStocktakeReport table

* Updates for stocktake generation

- remove old performStocktake javascript code
- Now handled by automated server-side calculation
- Generate report for a single part

* Add a new "role" for stocktake

- Allows fine-grained control on viewing / creating / deleting stocktake data
- More in-line with existing permission controls
- Remove STOCKTAKE_OWNER setting

* Add serializer field to limit stocktake report to particular locations

* Use location restriction when generating a stocktake report

* Add UI buttons to perform stocktake for a whole category tree

* Add button to perform stocktake report for a location tree

* Adds a background tasks to handle periodic generation of stocktake reports

- Reports are generated at fixed intervals
- Deletes old reports after certain number of days

* Implement notifications for new stocktake reports

- If manually requested by a user, notify that user
- Cleanup notification table
- Amend PartStocktakeModel for better notification rendering

* Hide buttons on location and category page if stocktake is not enabled

* Cleanup log messages during server start

* Extend functionality of RoleRequired permission mixin

- Allow 'role_required' attribute to be added to an API view
- Useful when using a serializer class that does not have a model defined

* Add boolean option to toggle whether a report will be generated

* Update generateStocktake function

* Improve location filtering

- Don't limit the actual stock items
- Instead, select only parts which exist within a given location tree

* Update API version

* String tweaks

* Fix permissions for PartStocktake API

* More unit testing for stocktake functionality

* QoL fix

* Fix for assigning inherited permissions
2023-02-17 11:42:48 +11:00
Oliver 06eb3a81b6
Force builtin plugins to be active (#4276)
* Force builtin plugins to be active

- Not setting the "active" flag can cause ambiguous errors
- e.g. some settings don't work if the plugin is inactive
- By design, builtin plugins are always "active" anyway
- This PR just forces the active field to True

* Fix unit tests
2023-01-29 15:44:04 +11:00
Bben eda0e74cb4
UrlsMixin plugin update custom urls on load (#4065)
* For pure UrlsMixin plugin, invalidate the url to be able to trigger the update

* Delete stock.json

* Add UrlsMixin Plugin activation methods

* whitespace / breakline removal

* whitespace?

* whitespace again

* linebreak

* Delete .pre-commit-config.yaml

* restore .pre-comiit.yaml
2022-12-21 10:21:07 +11:00
Oliver 06266b48af
Part pricing cache (#3710)
* Create new model for storing Part pricing data

Currently this model does not "do" anything but will be used for caching pre-calculated pricing information

* Define function for accessing pricing information for a specific part

* Adds admin site support for new PartPricing model

* Specify role for PartPricing model

* Allow blank values for PartPricing model fields

* Add some TODO entries

* Update migration files to sync with latest master

* Expose API endpoint for viewing part pricing information

* Update migration file

* Improvements:

- Updated model with new fields
- Code for calculating BOM price
- Code for calculating internal price
- Code for calculating supplier price
- Updated unit testing

* Fix (and test) for API serializer

* Including min/max pricing data in part serializer

* Bump API version

* Add pricing overview information in part table

- Adds helper function for formatting currency data
- No longer pre-render "price strings" on the server

* Overhaul of BOM API

- Pricing data no longer calculated "on the fly"
- Remove expensive annotation operations
- Display cached price range information in BOM table

* Filter BOM items by "has pricing"

* Part API endpoint can be filtered by price range

* Updpated API version notes

* Improvements for price caching calculations

- Handle null price values
- Handle case where conversion rates are missing
- Allow manual update via API

* Button to manually refresh pricing

* Improve rendering of price-break table

* Update supplier part pricing table

* Updated js functions

* Adds background task to update assembly pricing whenever a part price cache is changed

* Updates for task offloading

* HTML tweaks

* Implement calculation of historical purchase cost

- take supplier part pack size into account
- improve unit tests

* Improvements for pricing tab rendering

* Refactor of pricing page

- Move javascript functions out into separate files
- Change price-break tables to use bar graphs
- Display part pricing history table and chart
- Remove server-side rendering for price history data
- Fix rendering of supplier pricing table
- Adds extra filtering options to the SupplierPriceBreak API endpoint

* Refactor BOM pricing chart / table

- Display as bar chart with min/max pricing
- Display simplified BOM table

* Update page anchors

* Improvements for BOM pricing table display

* Refactoring sales data tables

- Add extra data and filter options to sales order API endpoints
- Display sales order history table and chart

* Add extra fields to PartPricing model:

- sale_price_min
- sale_price_max
- sale_history_min
- sale_history_max

* Calculate and cache sale price data

* Update part pricing when PurchaseOrder is completed

* Update part pricing when sales order is completed

* Signals for updating part pricing cache

- Whenever an internal price break is created / edited / deleted
- Whenever a sale price break is created / edited / deleted

* Also trigger part pricing update when BomItem is created  / edited / deleted

* Update part pricing whenever a supplier price break is updated

* Remove has_complete_bom_pricing method

* Export min/max pricing data in BOM file

* Fix pricing data in BOM export

- Calculate total line cost
- Use more than two digits

* Add pricing information to part export

Also some improvements to part exporting

* Allow download of part category table

* Allow export of stock location data to file

* Improved exporting of StockItem data

* Add cached variant pricing data

- New fields in part pricing model
- Display variant pricing overview in "pricing" tab

* Remove outdated "PART_SHOW_PRICE_HISTORY" setting

* Adds scheduled background task to periodically update part pricing

* Internal prices can optionally override other pricing

* Update js file checks

* Update price breaks to use 6 decimal places

* Fix for InvenTreeMoneySerializer class

- Allow 6 decimal places through the API

* Update for supplier price break table

* javascript linting fix

* Further js fixes

* Unit test updates

* Improve rendering of currency in templates

- Do not artificially limit to 2 decimal places

* Unit test fixes

* Add pricing information to part "details" tab

* Tweak for money formatting

* Enable sort-by-price in BOM table

* More unit test tweaks

* Update BOM exporting

* Fixes for background worker process

- To determine if worker is running, look for *any* successful task, not just heartbeat
- Heartbeat rate increased to 5 minute intervals
- Small adjustments to django_q settings

Ref: https://github.com/inventree/InvenTree/issues/3921
(cherry picked from commit cb26003b92)

* Force background processing of heartbeat task when server is started

- Removes the ~5 minute window in which the server "thinks" that the worker is not actually running

* Adjust strategy for preventing recursion

- Rather than looking for duplicate parts, simply increment a counter
- Add a "scheduled_for_update" flag to prevent multiple updates being scheduled
- Consolidate migration files

* Adds helper function for rendering a range of prices

* Include variant cost in calculations

* Fixes for "has_pricing" API filters

* Ensure part pricing status flags are reset when the server restarts

* Bug fix for BOM API filter

* Include BOM quantity in BOM pricing chart

* Small tweaks to pricing tab

* Prevent caching when looking up settings in background worker

- Caching across mnultiple processes causes issues
- Need to move to something like redis to solve this
- Ref: https://github.com/inventree/InvenTree/issues/3921

* Fixes for /part/pricing/ detail API endpoint

* Update pricing tab

- Consistent naming

* Unit test fixes

* Prevent pricing updates when loading test fixtures

* Fix for Part.pricing

* Updates for "check_missing_pricing"

* Change to pie chart for BOM pricing

* Unit test fix

* Updates

- Sort BOM pie chart correctly
- Simplify PartPricing.is_valid
- Pass "limit" through to check_missing_pricing
- Improved logic for update scheduling

* Add option for changing how many decimals to use when displaying pricing data

* remove old unused setting

* Consolidate settings tabs for pricing and currencies

* Fix CI after changing settings page

* Fix rendering for "Supplier Pricing"

- Take unit pricing / pack size into account

* Extra filtering / ordering options for the SupplierPriceBreak API endpoint

* Fix for purchase price history graph

- Use unit pricing (take pack size into account)

* JS fixes
2022-11-14 15:58:22 +11:00
Oliver b50a6826ef
Builtin plugins (#3889)
* Allow loading of "builtin" plugins, even if "plugins" are not explicitly loaded

* Updates for 'admin' buttons:

- Make them work like proper links
- Hidden if 'hide_admin_link' customization option is set
- Check for user staff status

* Cleanup rendering of "plugins" display

* Consolidate InvenTree barcode plugins into single plugin class

* Hide "install plugin" button if plugins are not enabled

* Add info message is external plugins are not enabled

* Fixes for loading plugins

- Always load 'builtin' plugins
- Refactor calls to "is_active" at various points in codebase

* Various tweaks

- Improve builtin plugin descriptions
- Spelling fixes

* Adjust plugin detail for builtin plugins

* Simplify barcode plugin class

* Simplify template rendering

* Bug fix for inventree barcode plugin

* Revert "Simplify template rendering"

This reverts commit 3a6755a659.

* Re-re-improve template rendering

- Required as the template has been refactored for both "active" and "inactive" plugins

* Fixing unit tests for barcode plugin

* Ensure that barcode scan actions do not take a "long time":

- Add a default timeout of 0.1s to any POST or GET request in the testing framework
- Can be overridden by calling method if desired

* Display plugin "builtin" status in admin panel

* Fix unit tests for plugin API

* Further unit testing fixes

* Version number tweaks

* Further tweaks for unit testing

* Allow longer timeout for report printing via API

* Increase default timeout for API tests

- Sometimes CPU spike can cause the test to fail :|

* label printing can take a bit longer

* Remove timeout requirement from API tester

- Too variable to be reliable for CI
2022-11-05 01:05:54 +11:00
Oliver 906ae218aa
[Plugin] Allow custom plugins for running validation routines (#3776)
* Adds new plugin mixin for performing custom validation steps

* Adds simple test plugin for custom validation

* Run part name and IPN validators checks through loaded plugins

* Expose more validation functions to plugins:

- SalesOrder reference
- PurchaseOrder reference
- BuildOrder reference

* Remove custom validation of reference fields

- For now, this is too complex to consider given the current incrementing-reference implementation
- Might revisit this at a later stage.

* Custom validation of serial numbers:

- Replace "checkIfSerialNumberExists" method with "validate_serial_number"
- Pass serial number through to custom plugins
- General code / docstring improvements

* Update unit tests

* Update InvenTree/stock/tests.py

Co-authored-by: Matthias Mair <code@mjmair.com>

* Adds global setting to specify whether serial numbers must be unique globally

- Default is false to preserve behaviour

* Improved error message when attempting to create stock item with invalid serial numbers

* Add more detail to existing serial error message

* Add unit testing for serial number uniqueness

* Allow plugins to convert a serial number to an integer (for optimized sorting)

* Add placeholder plugin methods for incrementing and decrementing serial numbers

* Typo fix

* Add improved method for determining the "latest" serial number

* Remove calls to getLatestSerialNumber

* Update validate_serial_number method

- Add option to disable checking for duplicates
- Don't pass optional StockItem through to plugins

* Refactor serial number extraction methods

- Expose the "incrementing" portion to external plugins

* Bug fixes

* Update unit tests

* Fix for get_latest_serial_number

* Ensure custom serial integer values are clipped

* Adds a plugin for validating and generating hexadecimal serial numbers

* Update unit tests

* Add stub methods for batch code functionality

* remove "hex serials" plugin

- Was simply breaking unit tests

* Allow custom plugins to generate and validate batch codes

- Perform batch code validation when StockItem is saved
- Improve display of error message in modal forms

* Fix unit tests for stock app

* Log message if plugin has a duplicate slug

* Unit test fix

Co-authored-by: Matthias Mair <code@mjmair.com>
2022-10-18 23:50:07 +11:00
Matthias Mair 3bd64e9098
Raise error if version fails (#3792)
* add comment where feature should be

* add triggermethod for superusers

* Add plugin version warning

* switch to simple error logging

* switch import mechanism

* move import down

* reintroduce db notifications

* adding mising import

* add db entry

* Only log to DB if plugin was regsitered

* fix merge

* remove db logging
2022-10-18 09:55:56 +11:00
Matthias Mair 0bab40fe88
Plugin API refactor (#3637)
* out-of-scope: refactor plugin lookup

* lookup by settings (runtime not predefined)

* Add registry function to set state of plugin quickly

* Ensure plugin is active before assertations
2022-10-17 22:08:28 +11:00
Matthias Mair 1b76828305
Plugin framework updates (#3535)
* refactor entrypoint into helpers

* Add lookup by metadata
This is geared towards plugins packaged in pkgs that differ in name from their top module

* Make module lookup predictable in changing pkg-envs

* remove no coverage from plugin packages

* ignore coverage for production loadin

* refactor plugin collection - move assigment out

* do not cover fs errors

* test custom dir loading

* test module meta fetcher

* add a bit more safety

* do not cover sanity checkers

* add folder loading test

* ignore again for cleaner diffs for now

* ignore safety catch

* rename test

* Add test for package installs

* fix docstring name

* depreciate test for now

* Fix for out of BASE_DIR paths

* ignore catch

* remove unneeded complexity

* add testing for outside folders

* more docstrings and simpler methods

* make call simpler

* refactor import

* Add registry with all plugins

* use full registry and make simpler request

* switch path properties to methods

* Add typing to plugin

* Add a checker fnc for is_sample

* Add sample check to admin page

* Make file check a cls

* more cls methods

* Add setting for signature cheks
Fixes #3520

* make property statements simpler

* use same key in all dicts

* Use module name instead of NAME
Fixes #3534

* fix naming

* fix name

* add version checking
Fixes #3478

* fix formatting and typing

* also save reference to full array

* do not cover as we turn on all plugins

* add test for check_version

* Add version e2e test

* make test save

* refactor out assignment

* safe a db reference first

* docstring

* condense code a bit

* rename

* append logging

* rename

* also safe db reference to new object

* docstrings, refactors, typing

* fix key lookup
2022-08-16 16:10:18 +10:00
Matthias Mair 188ddc3be3
Improve plugin testing (#3517)
* refactor entrypoint into helpers

* Add lookup by metadata
This is geared towards plugins packaged in pkgs that differ in name from their top module

* Make module lookup predictable in changing pkg-envs

* remove no coverage from plugin packages

* ignore coverage for production loadin

* refactor plugin collection - move assigment out

* do not cover fs errors

* test custom dir loading

* test module meta fetcher

* add a bit more safety

* do not cover sanity checkers

* add folder loading test

* ignore again for cleaner diffs for now

* ignore safety catch

* rename test

* Add test for package installs

* fix docstring name

* depreciate test for now

* Fix for out of BASE_DIR paths

* ignore catch

* remove unneeded complexity

* add testing for outside folders

* more docstrings and simpler methods

* make call simpler
2022-08-16 13:09:48 +10:00
Matthias Mair 1010a6296f
Plugin meta detection improvements (#3516)
* refactor entrypoint into helpers

* Add lookup by metadata
This is geared towards plugins packaged in pkgs that differ in name from their top module

* Make module lookup predictable in changing pkg-envs
2022-08-11 12:02:06 +10:00
Oliver e9b0f02ecd
Update demo data hook to copy media files (#3441)
* Simplify settings.py / config.py

- get_setting function has been streamlined
- move some functions into config.py

* Spelling fix: IGNORRED is IGNORED

* Ensure yaml is installed as part of docker image

- invoke path is still mucking us around

* Fix broken migration

* Copy media files from demo dataset when installing test data

* Cleanup settings.py

* Fix for configuration file traversal

* Line fix

* Update quickstart guide for docker

* Allow plugin file and plugin dir to be specified in configuration file

* Cleanup config template file

* Allow secret_key information to be provided in configuration file

* Adjust root paths for CI tests

* resolve paths

* Revert paths for CI step

* remove dead code

* Revert configuration variables to old names

- Prevent breaking changes

* Simplify secret key generation

* Fix default timeout for background worker process

* Revert change for customization options
2022-07-31 23:16:58 +10:00
Oliver 8f9981a3b6
Convert PosixPath to string for plugin loading (#3423) 2022-07-27 21:59:23 +10:00
Matthias Mair 794e375009
Switch to pathlib (#3392)
* switch to pathlib

* more pathlib

* useconvert env to path

* fix typo

* use resolve instead of absolute

* fix gitppod

* also allow parents

* replace more os operations

* fix string replacement feature

* make sure root dirs exsist

* fix replace function

* refactor duplicate code

* reduce code

* make sure dirs exist

* fix typo

* also create parent dirs

* fix match statement

* fix statments expecting string

* return getMigrationFileNames to old behaviour

* fully resolve config file

* make sure comparison works

* use pathlib in tasks

* fix file count test

* reduce code duplication in test + add test for part

* fix test

* re-add os

* Make pathlib usage simpler
2022-07-27 10:42:34 +10:00
Oliver 7cabb78964
Docker compose relative path (#3410)
* Update development docker-compose file

- Remove external volume definition
- v2 does not allow relative path spec here
- Simplification is only required for dev version

* Remove old debug messages

* Update docker build CI step

- Check that required directories / files have been created in the correct location(s)

* Add check for pgdb directory

* Run partial docker check on pull_request

* Disable workflow running on pull requests
2022-07-27 07:50:54 +10:00
Oliver b98ae25583
Allow plugin loading from external directory (#3364)
* Load custom plugin directories

* Allow plugins to be loaded from an external directory

* Handle exception when path is not relative to base path

* Fix typo

* Use pathlib

* Move plugin directory code into registry.py

- Allows us to reload plugins without having to reload the server itself

Co-authored-by: Matthias Mair <code@mjmair.com>
2022-07-26 14:35:01 +10:00
Matthias Mair 7493c8c6cb
Use metadata in projects for plugin info (#3282)
* Use metadata in projects for plugin info

* use metadata as backup
2022-07-09 17:06:50 +10:00
Oliver 5ecba6b13c
Add error handling for case where user does not have git installed (#3179) 2022-06-11 23:11:50 +10:00
Matthias Mair f38386b13c
Add more checks to pre-commit (#3132)
* Add bandit to pre-commit checks

* fix catchall exceptions

* remove unused definitons

* remove unuseed ariables

* Add docstring

* fix B006, B008 errors

* fix B007 error

* ignore B009

* Add checks for formatting and naming
2022-06-06 08:56:52 +10:00
Matthias Mair 0c97a50e47
Docstring checks in QC checks (#3089)
* Add pre-commit to the stack

* exclude static

* Add locales to excludes

* fix style errors

* rename pipeline steps

* also wait on precommit

* make template matching simpler

* Use the same code for python setup everywhere

* use step and cache for python setup

* move regular settings up into general envs

* just use full update

* Use invoke instead of static references

* make setup actions more similar

* use python3

* refactor names to be similar

* fix runner version

* fix references

* remove incidential change

* use matrix for os

* Github can't do this right now

* ignore docstyle errors

* Add seperate docstring test

* update flake call

* do not fail on docstring

* refactor setup into workflow

* update reference

* switch to action

* resturcture

* add bash statements

* remove os from cache

* update input checks

* make code cleaner

* fix boolean

* no relative paths

* install wheel by python

* switch to install

* revert back to simple wheel

* refactor import export tests

* move setup keys back to not disturbe tests

* remove docstyle till that is fixed

* update references

* continue on error

* add docstring test

* use relativ action references

* Change step / job docstrings

* update to merge

* reformat comments 1

* fix docstrings 2

* fix docstrings 3

* fix docstrings 4

* fix docstrings 5

* fix docstrings 6

* fix docstrings 7

* fix docstrings 8

* fix docstirns 9

* fix docstrings 10

* docstring adjustments

* update the remaining docstrings

* small docstring changes

* fix function name

* update support files for docstrings

* Add missing args to docstrings

* Remove outdated function

* Add docstrings for the 'build' app

* Make API code cleaner

* add more docstrings for plugin app

* Remove dead code for plugin settings
No idea what that was even intended for

* ignore __init__ files for docstrings

* More docstrings

* Update docstrings for the 'part' directory

* Fixes for related_part functionality

* Fix removed stuff from merge 99676ee

* make more consistent

* Show statistics for docstrings

* add more docstrings

* move specific register statements to make them clearer to understant

* More docstrings for common

* and more docstrings

* and more

* simpler call

* docstrings for notifications

* docstrings for common/tests

* Add docs for common/models

* Revert "move specific register statements to make them clearer to understant"

This reverts commit ca96654622.

* use typing here

* Revert "Make API code cleaner"

This reverts commit 24fb68bd3e.

* docstring updates for the 'users' app

* Add generic Meta info to simple Meta classes

* remove unneeded unique_together statements

* More simple metas

* Remove unnecessary format specifier

* Remove extra json format specifiers

* Add docstrings for the 'plugin' app

* Docstrings for the 'label' app

* Add missing docstrings for the 'report' app

* Fix build test regression

* Fix top-level files

* docstrings for InvenTree/InvenTree

* reduce unneeded code

* add docstrings

* and more docstrings

* more docstrings

* more docstrings for stock

* more docstrings

* docstrings for order/views

* Docstrings for various files in the 'order' app

* Docstrings for order/test_api.py

* Docstrings for order/serializers.py

* Docstrings for order/admin.py

* More docstrings for the order app

* Add docstrings for the 'company' app

* Add unit tests for rebuilding the reference fields

* Prune out some more dead code

* remove more dead code

Co-authored-by: Oliver Walters <oliver.henry.walters@gmail.com>
2022-06-02 01:37:39 +10:00
Matthias Mair 9a2300d920
Add option to also reload plugin mechanisms (#3081) 2022-05-29 08:25:40 +10:00
Matthias Mair e1d22f538d resort imports 2022-05-20 17:24:51 +02:00
Oliver 80e3d0970a Adds unit tests for the samplepanel plugin 2022-05-19 11:28:18 +10:00
Oliver 11b21a9cca Allow registry.with_mixin to filter by active status 2022-05-19 11:00:31 +10:00
Matthias d9fe7ac272
general Todo - no specific task
removed therefore
2022-05-15 18:09:39 +02:00
Matthias f4fbd57e6e
errors fail whole plugins now so no saving 2022-05-15 18:09:05 +02:00