[16.0][MIG] connector + test_connector: Migration to 16.0 #442

Plus various cosmetic improvements.

The job will be retried after the given number of seconds if the exception is raised with a defined 'seconds' argument.

@bealdav

Instead of `ir.module.module`. Simplifies as we never hit the database nor have to maintain a cache. Also, it fixes an issue when we run tests directly at the installation of a module, during the tests the previous implementation was considering the module as uninstalled instead of installed. Change proposed @bealdav. Thanks! Fixes OCA#85

This context manager allows to execute a job function synchronously when it should normally have been delayed in a asynchronous job. This is really useful for writing tests that check the flow of a synchronisation without having to deal with the jobs execution mechanisms. It is used in the OCA/connector-magento project and ought to be used by other projects.

This Postgres feature is invaluable when dealing with synchronisations, especially when importing concurrently records from a system. When we export a record, we are able to acquire a lock on the exported record to prevent 2 jobs to export it at the same time. This is different when we import a record for the first time and with several jobs running in parallel, chances are high that 2 jobs will import the same record at the same moment. The Postgres advisory lock comes handy there for they allow to acquire an application lock. Usually we'll acquire the lock at the beginning of an import (beginning of ``Importer.run()``) and we'll throw a ``RetryableJobError`` if the lock cannot be acquired so the job is retried later. The lock will remain in place until the end of the transaction. Example: - Job 1 imports Partner A - Job 2 imports Partner B - Partner A has a category X which happens not to exist yet - Partner B has a category X which happens not to exist yet - Job 1 import category X as a dependency - Job 2 import category X as a dependency Since both jobs are executed concurrently, they both create a record for category X. With this lock: - Job 1 imports Partner A, it puts a lock for this partner - Job 2 imports Partner B, it puts a lock for this partner - Partner A has a category X which happens not to exist yet - Partner B has a category X which happens not to exist yet - Job 1 import category X as a dependency, it puts a lock for this category - Job 2 import category X as a dependency, try to put a lock but can't, Job 2 is retried later, and when it is retried, it sees the category X created by Job 1 See http://topopps.com/implementing-postgres-advisory-locks/ for the article where I learned about the computation of the hash for this purpose.

It is no longer a context manager, because we would expect the lock to be released at the end of the 'with' statement but it lasts until the end of the transaction.

Commit c97ebbd was a frontport from 7.0. However, openerp.sql_db.dsn was changed in 8.0 in that it now returns a tuple. Extract the db_name from the returned tuple.

The first item contains the name of the DB such as 'odoo_db' and the second item contains 'user=gbaconnier dbname=odoo_db' which is what expects psycopg2.connect.

… --no-database-list

This is a temporary fix. In version 4.0, dbfilter will be completely ignored by connector.

When PG is localized, error messages are not ascii and jobs are not postponed. Instead they are failed with a 'unicode decode error'. Note: even if PG ```lc_messages``` is set to en_us.utf8 on a localized system, the error message prefixes (such as ```DETAILS:```) are still localized.

This is a safeguard to prevent someone to write arbitrary code in jobs. Builtin types and datetime/timedelta are allowed in job arguments, and a new function 'whitelist_unpickle_global' allows to register new objects if needed.

…ctive user is also inactive Fix the test accordingly (odoo/odoo@328d34a)

* In openerp.addons.connector.session.ConnectorSession: * Remove method 'search' * Remove method 'write' * Remove method 'browse' * Remove method 'read' * Remove method 'create' * Remove method 'unlink' * In openerp.addons.connector.connector.ConnectorUnit: * Remove method 'get_connector_unit_for_model' * Remove method 'get_binder_for_model' * Remove property 'environment' * In openerp.addons.connector.connector.ConnectorEnvironment: * Remove method 'set_lang' * In openerp.addons.connector.connector.connector.MetaConnectorUnit * Remove property 'model_name' * Remove class openerp.addons.connector.connector.Environment * Remove method openerp.addons.connector.connector.intall_in_connector * In openerp.addons.connector.unit.synchronizer: * Remove class 'ExportSynchronizer' * Remove class 'ImportSynchronizer' * Remove class 'DeleteSynchronizer' * Remove module openerp.addons.connector.deprecate. *

fixes OCA#175

The default binder returns a recordset of one record since 41561c8 so backend_to_m2o should extract the Id from it.

This also runs jobs that have an ETA at a more precise time: before the runner was waking up only every minute when there was no other activity.

When we call the binder 'to_openerp' method with 'unwrap', we expect the binder to return the 'unwrapped' record of the binding, e.g. product.product instead of prestashop.product.product. When no record is found, the binder *must* return an empty recordset of the correct (unwrapped) model too.

* Switch zero assertion to type strict eval (`is`)

Now jobs are only set to Done when NothingToDoJob is raised. Closes OCA#197

Now that we have a root model 'base', we can properly override the methods to fire the events.

The reason of being for the separation was to be able to support another storage than postgres if needed. Nothing has ever been done in this way and the current backend storage (postgres) has proved to be efficient.

It has been removed from queue_job, move it back here

* Remove notion of "id", we work with recordsets. * Rename 'to_backend' to 'to_external' to avoid confusion with the notion of backend * Rename 'to_odoo' to 'to_internal' for reciprocity * Rename m2o_to_backend and backend_to_m2o in the same manner

So we can write tests using real pieces of the framework on real models instead of using mocks which is quite limited and wring things in confusing ways. * Fix the related action implementation * Move the tests of the Default Binding in this module so we have tests that reflect the reality

We probably want to use the same env that we have in the backend_record! Besides, having 2 environments bring issues when we modify one but not the other one.

The inheritance system is copied from the Odoo on, because it works well for extending addons and because it will be easier to use for developers accustomed to it.

when inheriting from multiple components, we might inadvertly override a value with None

The mapping methods are now built by the component initialization when the "aggregated" class is built.

the 'components' method had 2 different return types depending of the 'multi' argument. Now we have 'component' or 'many_components' that return a Component instance or a list of Component instances.

We keep the ComponentUnit classes though, there is some duplication of code but it will be dropped in Odoo 11.0

Proposing a new API based on components for the events

We'll now use the component_event events

Tests using odoo transactions must run post install, because during the install the registry is not ready, so the components aren't neither.

It takes a condition which skip the event when evaluated to True. A new base listener for the connector adds a default condition based on self.env.context.get('connector_no_export'). Every export event listener should be decorated with: @skip_if(lambda self, *args, **kwargs: self.no_connector_export)

It allows to open/close objects on start/end of a synchro / work session. The base method does none of that, but this is a really common use case. Example: at the beginning of the work session, I open a webservice client that is available in our WorkContext for the duration of our work. At the end, I close the client to end the connection.

It is no longer necessary for the Components and most of the connectors redefine it.

@lmignon

It the previous commit, @lmignon added the possibility to load all components of an addon in a Component Registry. This commit takes benefit of this feature to simplify the existing tests and to add a base test case for the Connector (that loads all components of 'component', 'component_event', 'connector'). It can be used in implementations using the connector.

When we want to test the implementation of a connector, it must be simple. Add 2 new test cases for TransactionCase and SavepointCase which load all the components of the dependencies addons on the tested addons. Only the components of the addons in state 'installed' are loaded, so we don't load components of addons that depend on the tested addon. The tests must call _init_global_registry and build_registry instead of _register_hook, so the registry is not set to ready. If the global registry is set to ready during tests, the event will trigger during the installation of addons which is not what we want. The existing test case is meant only to be used when one has to manipulate components in a custom registry. See discussion on guewen@447b22f#commitcomment-22851711

When we run a ComponentRegistryCase test using odoo's way to run tests (--test-enable), the current addon is in state 'to install' or 'to upgrade', thus its components not loaded (which is what we want in this test Case, as it allow us to load manually the components / load different components). When we run the same test Case from pytest or nosetest, which are not run during an odoo upgrade, the state of the addon is 'installed', thus components are loaded. To ensure the same behavior with --test-enable and external tests runners, the current addon is always excluded from the components to load.

Because it calls a name_get per row. As the information is only repeating the selection, better to hide it.

When there is no direct mapping and child elements are filtered and return no value, the resulting record to write is like {'field_xxx': 'value', child_ids': [(0, 0, {})]}

* Install odoo with pip so sphinx can import it * Find the addons which can be installed to add them in the python path

Currently translated at 100.0% (47 of 47 strings) Translation: connector-11.0/connector-11.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-11-0/connector-11-0-connector/fr/

* Sphinx crashes with a recursion error when trying to document Models fields. Remove the fields from the documentation altogether, they didn't bring much value anyway * Gone through all the API pages and removed many useless autodoc with a mix of __all__, :exclude-members: or by defining more precisely what we want with :autoclass:, :autoatttribute:, ... or simply removing them * Changed the order of a few autodocs by putting the most useful classes/functions first * Improved a few docstrings

This component locks the records for the duration of the transaction or raise a RetryableJobError if it can't acquire the lock.

Currently translated at 100,0% (47 of 47 strings) Translation: connector-11.0/connector-11.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-11-0/connector-11-0-connector/pt_BR/

Currently translated at 100.0% (70 of 70 strings) Translation: connector-12.0/connector-12.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-12-0/connector-12-0-connector/pt_BR/

Currently translated at 35.7% (25 of 70 strings) Translation: connector-12.0/connector-12.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-12-0/connector-12-0-connector/zh_CN/

Currently translated at 60.0% (42 of 70 strings) Translation: connector-12.0/connector-12.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-12-0/connector-12-0-connector/zh_CN/

Currently translated at 100.0% (70 of 70 strings) Translation: connector-12.0/connector-12.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-12-0/connector-12-0-connector/zh_CN/

Currently translated at 100.0% (73 of 73 strings) Translation: connector-13.0/connector-13.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-13-0/connector-13-0-connector/zh_CN/

Currently translated at 100.0% (74 of 74 strings) Translation: connector-13.0/connector-13.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-13-0/connector-13-0-connector/pt_BR/

closes OCA#372

Currently translated at 100.0% (74 of 74 strings) Translation: connector-13.0/connector-13.0-connector Translate-URL: https://translation.odoo-community.org/projects/connector-13-0/connector-13-0-connector/zh_CN/

Alternatives such as activities can be used.

This is used as icon in app drawer.

Since connector adds root menu item, it should have proper icon. This is most apparent in app drawer provided by web_responsive.

So we can write tests using real pieces of the framework on real models instead of using mocks which is quite limited and wring things in confusing ways. * Fix the related action implementation * Move the tests of the Default Binding in this module so we have tests that reflect the reality

We probably want to use the same env that we have in the backend_record! Besides, having 2 environments bring issues when we modify one but not the other one.

The inheritance system is copied from the Odoo on, because it works well for extending addons and because it will be easier to use for developers accustomed to it.

Tests using odoo transactions must run post install, because during the install the registry is not ready, so the components aren't neither.

It allows to open/close objects on start/end of a synchro / work session. The base method does none of that, but this is a really common use case. Example: at the beginning of the work session, I open a webservice client that is available in our WorkContext for the duration of our work. At the end, I close the client to end the connection.

When we want to test the implementation of a connector, it must be simple. Add 2 new test cases for TransactionCase and SavepointCase which load all the components of the dependencies addons on the tested addons. Only the components of the addons in state 'installed' are loaded, so we don't load components of addons that depend on the tested addon. The tests must call _init_global_registry and build_registry instead of _register_hook, so the registry is not set to ready. If the global registry is set to ready during tests, the event will trigger during the installation of addons which is not what we want. The existing test case is meant only to be used when one has to manipulate components in a custom registry. See discussion on guewen@447b22f#commitcomment-22851711

removes unknown fields from backend creation args

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[16.0][MIG] connector + test_connector: Migration to 16.0 #442

[16.0][MIG] connector + test_connector: Migration to 16.0 #442

Commits on Jan 30, 2023