Scripted objects migration

Introduction
Scripted objects are rebuilt every time a FCStd document is opened. To do this the document keeps a reference to the module and Python class that were used to create the object, along with its properties.

Particularly focus on this part:

If the value of  or   is not found on the installed system, the object will fail to load correctly. This means that once an object is created using a particular class, the module should no longer be moved or renamed because if this is done, previously saved objects will break.

However, a valid reason for moving or renaming the module or class is to improve the structure and maintainability of the original code, for example, when restructuring an entire workbench. In this case there are various strategies to migrate old objects to using a new class. This is done in order to retain backwards compatibility, when outright breaking of old documents must be avoided.

Old object
An old object is defined in a module which is at the root of the workbench.

An object can be created using this class, and it can be saved to. If no particular viewprovider is assigned to the new object, its proxy class is simply set to a value different from, in this case, to.

Python console session with the basic properties omitted.

New object
Now we consider that the workbench is restructured, so that classes aren't just at the root directory, but instead are inside an directory. Complex workbenches that have many different types of objects should be structured in directories including objects, viewproviders, Gui Commands, task panel interfaces, etc.

This new class will refer to the same type of object, but both the module name as well as the class name have been renamed. Moreover, the properties also have changed; one property has been renamed, and a completely new property has been added.

If we create a new object with this new module we will have the following console session.

Method 1. Migration by redirecting the class
We will migrate the older object by redirecting the old class. The original class is deleted, and the name of the class is simply redirected to point to the new class.

Any document that tries to load will be redirected to load  instead.

If we open the document, and inspect the properties of the object in the Python console we will see that the older properties are conserved, but the object has a new Proxy class.

However, in this case we don't see the new properties of the new class. The reason is simply that the older object didn't have these properties. When was redirected to, only the proxy class changed, but previous information was retained.

Now, if the document is saved and opened again, it will automatically look for, and it will not require any more. The file may be removed permanently from the system as long as all older objects have been migrated to the new module. If the old module is removed but an object hasn't been migrated, the report view will show a message like this when opening a document containing such object.

If it is not realistically possible to migrate all older objects, say, because the old module was used in a workbench for many years, then must be kept as long as it's deemed necessary to give users the opportunity to migrate their objects.

Advantages and disadvantages
Advantages


 * Edit_OK.svg This is the simplest method that just requires redirecting an old class to a new class.
 * Edit_OK.svg Old properties are conserved as long as the new class doesn't override them.
 * Edit_OK.svg This is good if the old class and the new class have the same properties (handle the same type of data) but only their module or class name is different.

Disadvantages


 * Edit_Cancel.svg The new class keeps the old properties of the object, which is not always desired.
 * Edit_Cancel.svg New properties or renamed properties aren't handled, so the object will load but it may not show the correct behavior of the new class.
 * Edit_Cancel.svg The old module may have to be kept indefinitely to migrate all old objects created in the past.

Method 2. Migration when restoring the document
We will migrate the older object by modifying the old class. The majority of the original class is deleted, and instead the method is implemented. When this method exists, it will run when the document tries to restore an object that uses the class, so this is the opportunity that we have to assign a new class, manipulate the information, or print messages.

In this case, we assume that we have also defined a new viewprovider in the module. If we don't want to migrate this class, we may omit everything after the check.

A more complex example checks first that the proxy class is of the type that we are looking for, and only proceeds with the migration if it's the right type.

Assuming that we already changed the old module in this way, if we open a document with an old object, we will see the messages mentioning the use of the new classes.

Inspecting the object from the Python console we will see that the older properties are conserved, and in addition, new properties were added together with the new Proxy class.

The old properties were and ; the new properties are, , and. The migrated object retains the original two properties, and gains three new properties. However, since the new has the same name as the older property, the new property is renamed with an incremental number. Presumably this is not what we want. We can improve the situation by following the addendum 2.1 below.

Given that the classes are meant to handle the same type of object, we would like a migration in which transforms into, and  is simply assigned to the new , and there are no duplicate properties.

Advantages and disadvantages
Advantages


 * Edit_OK.svg This method allows us to check that the class that we are migrating is the right class, instead of simply redirecting to a newer class.
 * Edit_OK.svg Similar to method 1, old properties are kept as long as the new class doesn't override them.
 * Edit_OK.svg Unlike method 1, new properties are always added, however if they have the same name, they will be renamed.
 * Edit_OK.svg The migration is not immediate, we can still manipulate the information, or print messages while the object loads.

Disadvantages


 * Edit_Cancel.svg It is more verbose than method 1 because we need to implement the method to migrate the object.
 * Edit_Cancel.svg It always adds the new properties, so it may create duplicated properties in case the new properties have the same name as the old properties. This has to be handled manually.

Method 3. Migration when restoring the document, manually handling the properties
This is an extension of method 2. In the method we need to save the values of the properties that we want, and then we can remove these original properties. This is done so that when the new class is used, it will assign the new properties without risking name collisions with the older properties.

Like in method 2, if we want we can also add the piece of code that checks that the Proxy class is the right one. In this example once more we assume that we are using a custom viewprovider, with at least one custom property.

We can see that the old values are stored in an auxiliary dictionary, then the old properties are removed, then we add the new class, and finally we assign the previously saved values to the new properties. In this moment we can transform the saved values as necessary for the new class. For example, the is set to 3 times the old, and the new  simply receives the value of the old. As we know how the old and new classes are supposed to behave, we have the liberty of manipulating the data to migrate the object as we want.

We can only remove properties that were added by Python classes when we built the scripted object. Other attributes belong to the base C++ object and can't be removed.

Assuming that we already changed the old module in this way, if we open a document with an old object, we will see the messages mentioning the use of the new classes. Inspecting the object from the Python console we see that the older properties are removed, and only the new properties exist.

Since in the old class the property didn't exist, nothing was done with it. It simply was created by the new class.

Advantages and disadvantages
Advantages


 * Edit_OK.svg Similar to method 2, this method allows us to check that the class that we are migrating is the right class.
 * Edit_OK.svg We have complete control of what to do with the old properties. Typically they will be removed so that there is no name collision with new properties added. Thus we avoid duplicated properties.
 * Edit_OK.svg By saving the older values, we can manipulate the information in the restoring step as we want, and assign the corresponding values to the new properties.

Disadvantages


 * Edit_Cancel.svg This method is very verbose compared to the previous ones, because we must implement the method, and handle each of the properties individually (save value, delete property, re-assign value). This is problematic if the object that we want to migrate has many properties, or their values need to be transformed in very special ways.

Addendum A. Creating the properties only if they do not already exist
One of the disadvantages of method 2 is that it will always try to add the new properties. If the older properties have the same name as the new properties, they will be duplicated with an incremental number, so will result in, then , and so on. This makes method 2 an unrealistic option in most cases, because the new class will only use one property anyway.

To improve this method, the new class can also be modified to only add the properties if they don't already exist by the same name.

In this case, since already exists, it won't be added again;  and  don't exist, so these will be added. And just like before, will be retained because it is not explicitly removed, although possibly it is no longer used in the new class.

The same can be done for the class of the viewprovider.

By using this method 2 + A, the result is similar to method 1 in that the object will retain all previous properties, but in addition it will gain the new properties provided by the new class.

Method 3 does not need this addendum to the new class because the older properties are explicitly removed, so there won't be any conflicts when installing the new properties. Nevertheless, it is still a good practice that every class adds its required properties only if these don't already exist. This is helpful both in the case of creating new scripted objects or in migrating them.

Advantages and disadvantages
Advantages


 * Edit_OK.svg The object will retain all previous properties, but in addition it will gain new properties without repetition.

Disadvantages


 * Edit_Cancel.svg Like method 2, it still doesn't deal with renamed properties. The old properties should be manually removed.

Addendum B. Migrating different versions of the old object
Method 3 is the most complex method because the properties are handled individually. However, in this method we also have full flexibility in how we manipulate the data, and this is an advantage if we want to do complex operations.

If from the beginning we create a property that holds the version number of our object, we can use this number in the future to perform specific migration from that version to any other. We set the property to be read-only, so that we cannot overwrite it in the property editor, although it is still accessible from the Python console.

Then, when we want to migrate the object, we implement the method, and test for this version.

We don't save the value as we will set a new  number when doing the migration. As shown in the example, we can implement various functions for each corresponding version of the object that we intend to migrate. We omit the migration of the viewprovider properties but it follows the same pattern.

Advantages and disadvantages
Advantages


 * Edit_OK.svg We have complete control of what to do with the old properties, and how to perform the migration.
 * Edit_OK.svg We can implement a particular method to migrate a particular version of the old object.

Disadvantages


 * Edit_Cancel.svg This method is very verbose because we must have a clear idea on how to handle each of the properties of each "version" that we want to migrate. If our object has many different versions created over the years, we may have to prepare a long list of methods to migrate them to the newest object.

Addendum B2. Using internal class attributes instead of properties
Instead of using a property of the object to hold the version information, we can use an attribute of the class. In this way we "hide" the version information, because properties are normally public, and visible in the property editor, while class attributes can only be manipulated from the Python console. Class attributes can be saved and restored as explained in Scripted objects saving attributes.

This attribute is inspected by looking at the attribute.

Then the file is modified to migrate the object.

When we install the new class, this new class should set the new value of the version attribute, for example,.

Addendum C. Method 3 without removing old properties that are named the same
Like in Addendum A, we can write the new class to create properties only if they aren't already present. Using method 3, we save the values of the older properties, and subsequently delete the older properties. However, if the new properties are named the same as the older ones, we don't need to delete the older ones, we can just reuse the same property, as we know the property won't be duplicated. If we are using Addendum B, we have a way to query the version as well.

As we see in the example, the old property is deleted and migrated to the new  property as usual. We do not need to delete nor  because in the new class they are still used with the same name, and they won't be created again (addendum A). As we don't want to modify, this property is not touched at all; it is migrated to the new class silently. However, we do update to the new value. We omit the migration of the viewprovider properties but it follows the same pattern.

This should work like method 3, meaning that the old properties are removed and only the new properties remain in the new object. The only difference is that we omit removing and recreating the properties that are named the same. This process should work as long as the old property and the new property have the same type (for example, or ), so the old property can pass its value directly. However, if the new property has a different type than the old property, then the old property should be removed, otherwise the old property will completely overwrite the new property, which is probably not what we want because the new class will be expecting the new type and not the old type.

Advantages and disadvantages
Advantages


 * Edit_OK.svg Like method 3, this method allows us complete control of the migration of the old information.
 * Edit_OK.svg We avoid writing code that removes and recreates properties that are named the same.

Disadvantages


 * Edit_Cancel.svg Like method 3, this method is still very verbose because we have to handle the properties carefully.
 * Edit_Cancel.svg If a new property and an old property share the same name, the new property will be overwritten, which may be undesired behavior, especially if the two properties have different types. In this case, removing the old property, and migrating its value manually is still necessary.

Summary
Each of the methods has a recommended use:
 * Method 1. The module is moved or renamed but the properties are the same. Simple redirection of classes because the properties don't need to be modified at all.
 * Method 2+A. Simple migration scenarios. Display a message when the object is migrated from one class to another. The properties are of the same type and don't need to be modified at all.
 * Method 3, 3+A, or 3+B. Complex migration scenarios. Full control of the properties, deleting the old properties, and adding new properties. An identifier to know the version of the object is useful to choose the right function to perform the migration (Addendum B or B2).

Preferably avoid the following:
 * Method 2. The properties will be duplicated if the new class doesn't check for existing properties (Addendum A).
 * Method 3+C. Use only when the old properties and the new properties are of the same type. Otherwise use method 3 or 3+B to remove older properties, and handle them exactly as needed.

Links

 * Migrating and upgrading old scripted objects
 * Migrate old scripted objects