Serialize with pointers : First occurence is not replace by its Reference pointers because its value is overwritten

[GhislainJ]

**Note: for support questions, please use https://nextcloud.dessia.tech/call/hr9z9bif

• I'm submitting a ...

  • bug report
  • feature request

• What is the current behavior?

The following dict is the result of a to_dict with pointers. The vectors of the to modules are equal and both should point to to the ref of the first occurence in _references. However, only the second one redirect to it, the first one stays {name : "Vector 1"} at location #/vessel/modules/0/origin or #/_references/1012d774-b552-11ee-8c66-4d72d9812f94/direction just like Module 2 whereas it should be

{
'object_class': 'dessia_common.forms.Layout',
'name': 'Layout',
'vessel': {
    '$ref': '#/_references/1012d777-b552-11ee-8c66-4d72d9812f94'
},
 '_references': {
    '1012d774-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Module',
        'name': 'Module 1',
        'origin': {'name': 'P1'},
        'direction': {'name': 'V1'}
    },
    '1012d773-b552-11ee-8c66-4d72d9812f94': {
        'name': 'V1'
    },
    '1012d776-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Module',
        'name': 'Module 2',
        'origin': {'name': 'P2'},
        'direction': {
            '$ref': '#/_references/1012d773-b552-11ee-8c66-4d72d9812f94'
        }
    },
    '1012d777-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Vessel',
        'name': 'Vessel',
        'modules': [
            {'$ref': '#/_references/1012d774-b552-11ee-8c66-4d72d9812f94'},
            {'$ref': '#/_references/1012d776-b552-11ee-8c66-4d72d9812f94'}
        ]
        }
    }
}

• If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

Object Structure

class Point:

    def __init__(self, name: str = ""):
        self.name = name

    def to_dict(self):
        return {"name": self.name}


class Vector:

    def __init__(self, name: str = ""):
        self.name = name

    def to_dict(self):
        return {"name": self.name}

    def __eq__(self, other):
        return True

    def __hash__(self):
        return 0


class Module(DessiaObject):
    _standalone_in_db = True

    def __init__(self, direction: Vector, origin: Point, name: str = ''):
        self.origin = origin
        self.direction = direction

        DessiaObject.__init__(self, name=name)


class Vessel(DessiaObject):
    _standalone_in_db = True

    def __init__(self, modules: List[Module], name: str = ''):
        self.modules = modules

        DessiaObject.__init__(self, name=name)


class Layout(DessiaObject):
    _standalone_in_db = True

    def __init__(self, vessel: Vessel, name: str = ''):
        self.vessel = vessel
        DessiaObject.__init__(self, name=name)

Script :

vectors = [Vector("V1"), Vector("V2")]
points = [Point("P1"), Point("P2")]

modules = [Module(origin=p, direction=v, name=f"Module {i + 1}") for i, (p, v) in enumerate(zip(points, vectors))]
vessel = Vessel(modules=modules, name="Vessel")
dict_ = Layout(vessel=vessel, name="Layout").to_dict()

Insert these prints in the corresponding serialization.py method :

def add_references(dict_, memo, id_memo):
    """ Add _references to a dict given the memos. """
    dict_['_references'] = id_memo

    # Rewriting dc__refs
    for refpath, serialized, id_, object_path in memo.values():
        name = serialized.get("name", "A")
        print(f"{name}{' '*(10 - len(name))}: {id_} | {object_path}")
        if not object_path.startswith('#/_references') and id_ in id_memo:
            if REF_MARKER not in serialized:
                print(f"{' '*14} Setting {REF_MARKER} : '#/_references/{id_}' @ {object_path}")
                set_in_object_from_path(dict_, object_path, {REF_MARKER: f'#/_references/{id_}'})

    for _, serialized, _, object_path in memo.values():
        print(f"{object_path} => {get_in_object_from_path(dict_, object_path)}")

This shows how Vector 1 is at first well replaced in the resulting dict, but not in its parent dict. As his parent is replaced afterwards, this first operation is cancelled

• What is the expected behavior?

{
'object_class': 'dessia_common.forms.Layout',
'name': 'Layout',
'vessel': {
    '$ref': '#/_references/1012d777-b552-11ee-8c66-4d72d9812f94'
},
 '_references': {
    '1012d774-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Module',
        'name': 'Module 1',
        'origin': {'name': 'P1'},
        'direction': {
            '$ref': '#/_references/1012d773-b552-11ee-8c66-4d72d9812f94'
        }
    },
    '1012d773-b552-11ee-8c66-4d72d9812f94': {
        'name': 'V1'
    },
    '1012d776-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Module',
        'name': 'Module 2',
        'origin': {'name': 'P2'},
        'direction': {
            '$ref': '#/_references/1012d773-b552-11ee-8c66-4d72d9812f94'
        }
    },
    '1012d777-b552-11ee-8c66-4d72d9812f94': {
        'object_class': 'dessia_common.forms.Vessel',
        'name': 'Vessel',
        'modules': [
            {'$ref': '#/_references/1012d774-b552-11ee-8c66-4d72d9812f94'},
            {'$ref': '#/_references/1012d776-b552-11ee-8c66-4d72d9812f94'}
        ]
        }
    }
}

• What is the motivation / use case for changing the behavior?

The problem here is that we lose the information that these two vectors are initially equal to each other. When dealing with dicts only there is no way to get this information back

• Possible fixes

We should probably inverse serialization order, but this seems dangerous, as the current issue occurs in a very specific usecase.
A refactor of how things are serialized, using OO, is probably a prerequisite

• Please tell us about your environment:

  • branch: dev
  • commit: latest
  • python version: 3.10