User:Suzanne.soy/XternalApps Workbench

The contents of this page are released in the public domain using the Creative Commons CC0 license, contrarily to the rest of the wiki. The contributors agree to waive all rights on their modifications to the fullest extent possible.

Rationale: this is a specification for an interoperability file format, it should not be encumbered in any way.

The XternalApps Workbench offers easy access to various applications from within FreeCAD.

An XternalApp may:


 * Be used to open objects of the application's native datatype (e.g. Inkscape being used to open, edit and save files embedded in a FreeCAD .FCStd document)
 * Be used to open objects of the native/interoperability FreeCAD type (e.g. Inkscape being used to open, edit and save native FreeCAD curves)
 * Be opened inside a FreeCAD tab, for a better look&feel.
 * Offer plug-ins and tools which can be used directly from within FreeCAD as parametric objects which transform their input (e.g. an Inkscape plug-in being applied to a FreeCAD curve, when the curve is edited the plug-in's output is automatically updated)

Other features:



Application list
With great power comes great wiring

The GammaRay™ integration aims to open GammaRay™ to inspect the FreeCAD Gui, or to open an XternalApp with GammaRay™ already attached. Embedding GammaRay™-specific files within a FreeCAD document doesn't seem very useful, so it's not a priority. Mousepad has very few tools, but it would be a good exercise on how to access the tools of a GTK application Computer Algebra System In addition to toolbars and presence as objects in the tree, it should be possible e.g. to refer to properties of other objects from within a LaTeX document, and to annotate the resulting PDF or document so that the graphs and other figures can be extracted and included in other documents Use a KIOSlave or FUSE to browse the tree structure of the document using a file browser UI; allow copying external documents in and copying objects out. When an object appears in multiple locations in the tree, beware of how to represent this as a single file appearing in multiple places… probably using symlinks or .desktop shortcuts. Lucky us, FreeCAD already embeds a web browser :) e.g. conversion from LaTeX / .docx to .html is possible  The goal is, of course, to obtain parametric families of fonts

Input parameters
The parameters of the plug-ins from Inkscape etc. are described using an XForms XML file. The XForms XML format can be used to describe forms and mainly targets web pages.

,---. `---’   `--> ,.          ,---.                                   | Generic XForms API description |  --+-->  | FreeCAD XternalApps Workbench | ,---.   ,--> `’          `---’ `---’    :                             :           ...           ....’
 * Inkscape plug-ins  +,
 * Gimp plug-ins    +’

TODO: use an XSD file instead of an and , and just link the model + instance to the XSD schema definition

Application-specific parameter extractor
A short application-specific stub is in charge of extracting the information from the plug-ins and converting it into this universal format. It is represented by a small rectangle attached to the right of the applications.

,---,-. `---'-’   `--> ,.          ,---.                                     | Generic XForms API description |  --+-->  | FreeCAD XternalApps Workbench | ,---,-.   ,--> `’          `---’ `---'-’    :                             :           ...           ....’
 * Inkscape plug-ins  | +,
 * Gimp plug-ins    | +’

For most applications, it's just a matter of iterating over the list of plug-ins available in the application's installation directory or configuration directory, and exctacting the info from an existing file format. For example, Inkscape plug-ins are described by .inx files which are XML files with contents similar to that of our file format.

Output values
The output of the tool is described using a model just like the input model of XForms; it is in a way a read-only XForms model instance which is returned by the tool.

User interface
The user interface is described in a way that leaves a lot of freedom to the implementer. The subset of XForms which we use is mainly concerned with giving names to the fields of the internal model, and grouping them. In particular, we do not include anything about hiding or disabling fields based on the contents of other fields. Indeed, a nodal interface could map these form elements to input and output pins, and once these are connected disabling or enabling them would make little sense; furthermore the data being sent to those pins could evolve with time, but the connections must persist.

A future version will likely make use of variants and records to structure data in a way which allows the description of simple combinations of allowed parameters, and can still be displayed as a pipeline of transformations (e.g. first create an instance of one of the cases of the variant, then pass it as a parameter). See [[#XForms subset and interoperability|XForms subset and interoperability] for more details on how this constitutes a rudimentary type system and why we don't want to rush to add one with the wrong features.

Another XForms form is used to display the output model.

The attribute is used to indicate that an input or output is the primary input or output. Consumers of the tool description can display the primary input and output in a special way in the user interface, for example in a nodal interface the primary input or output could be displayed at the top of a node, and it could have a different colour. There must be at most one primary input (it is optional, but if present there must be only one primary input). There must be at most one primary output (it is optional, but if present there must be only one primary output).

The attribute is used to indicate that an input or output is highly relevant. Consumers of the tool description can display the secondary inputs and outputs in a special way in the user interface, for example in a tree view of a document, the primary and secondary inputs could be displayed as children of the object. In a nodal interface, a minimised form of the node could show only the primary and secondary connections, and hide the less relevant parts of the graph. There can be any number of secondary inputs and outputs, including zero.

It is possible for a tool's input form to have a primary field and no secondary field. It is possible for a tool's input form to have some secondary fields, and no primary field. It is possible for a tool's input form to have no primary field, and no secondary field. The same applies to the tool's output, and there is no link between the relevance attributes of the input and output forms.

XForms subset, interoperability and types
Only a subset of the XForms file format is used. Since this format should be usable by various applications other than FreeCAD, keeping it simple seems wise. For example, as mentioned in User interface, a nodal interface could map these form elements to input and output pins on nodes, any unnecessary complexity in the form file format would result in a confusing user interface, or, in the case of form elements which are conditionally enabled based on the contents of other forms, would require some form of type system to check whether a pin connection should be allowed based on already-connected pins. Designing a type system adequate for inter-application communication is not a simple feat, and is a task best left for a future iteration.

,...               ...                                                                          :                                                                               :                                                                          :     ,---.                                                                          ,-->  | Other software e.g. nodal UI  | ,---,-.                                              :     `---’ `---'-’    `--> ,.    :     ,---.                                     | Generic XForms API description |  --+-->  | FreeCAD XternalApps Workbench | ,---,-.   ,--> `’          `---’ `---'-’    :                               :           ...             ....’
 * Inkscape plug-ins  | +,                                          :
 * Gimp plug-ins    | +’

Command-line
Since the standard web APIs (e.g. POST requests and form encodings) are not suitable for invocation of command-line applications, a few extra tags are used to describe the command-line API.

Future extensions could support maintaining the "connection" with a tool by piping several batches of arguments to a single process. This will be useful for opening a document in an external application, embeding its window, and sending commands to the external application.

Tool metadata
In lieu of an HTML page, a few other tags define a tooltip, icon, description etc.

Files
The XML description of a tool is split into several files:
 * The input model + form elements
 * The command-line generation, which takes an input XForms model and produces a command + arguments
 * The output extraction, which takes the output of the command and produces an output XForms model
 * The output model + read-only form elements

The and communication paths could be done by any means; usually this will happen by simply passing a pointer, but a web version could encode the input, transfer it over HTTP, and have it decoded on the server where the command-line generation happens (the reverse path would be followed for returning the output).

In other words, it does not matter how the form contents are transferred between the user interface and the service responsible for execution, as long as the model arrives intact after the transfer.

This also ensures that it is possible to e.g. easily substitute the command-line generation part with a platform-specific one.

Generic extractors
In an ideal world, all existing applications would offer some form of command-line API or scripting API to invoke each and every one of their user-facing features (buttons, tools, menu items etc.), and the list of these features would be easy to query statically or dynamically, and their options would be described in a consistent fashion, even if that format was application-specific.

The real world is far from that, our running example, Inkscape, offers just this for its plug-ins, but Inkscape's built-in tools are available through two command-line APIs (a migration is ongoing), and the command-line flag to list all available commands does not indicate the expected arguments for each command, let alone a proper description etc. Most tools don't do better, and quite a few can't even brag about having a part of their API well-documented.

Even though it would be possible to fill in the details by hand, it would take a huge amount of time; fortunately there is a quasi-universal interaction protocol which works for most applications: open the file, select the relevant portion, click a button, and save the file.

If only we could extract all the buttons of an application, it would be easy to generate XML tool descriptions doing the load-select-button-save sequence. By spoonfeeding small enough document fragments, the "select" part would be optional in a large number of cases.

To fulfill this goal, we will implement two libraries, GTKDynamite and QtBomb. The role of these libraries is to be loaded via on Linux (and equivalent mechanisms on other platforms, TBD).

Once loaded, the library will access the application's main window widget, explode the application into individual tools. It will do so by finding all toolbars and menus (those tend to follow a somewhat standard structure, application-spicific directives to guide the search should be minimal), and offering remote activation of these tools.

Additionally, the GTKDynamite and QtBomb libraries will reparent the widgets into many individual windows, attaching some X11 hints to the windows, which will allow software like the XternalApplications workbenches to grab these windows and place it at their whim. In the case of FreeCAD workbenches, the single-tool windows will be grabbed using, and moved to a toolbar which can be docked alongside FreeCAD's own toolbars.

Using these two libraries should allow us to access the tools of many applications at a small cost, regardless of the maturity of their scripting and command-line APIs. Additionally, it will allow us to hide or move the toolbars, in a way that lets the main window of the application contain the currently-opened document and nothing else; this will allow the windows of these applications to be seamlessly embedded within a tab or other widget.

,-.                                                                         ,-->  | other software  | ,---,-.                                              :     `-’ `---'-’    `--> ,.    :     ,.                                     | Generic XForms API description |  --+-->  | FreeCAD XternalApps Workbench  | ,---,-.   ,--> `’          `’ `---'-’    :                               :           ...             ....’                               : ,---,-.    : `---'-’    :                             : ,---,-.    : `---'-’
 * Inkscape plug-ins  | +,                                          :
 * Gimp plug-ins    | +’
 * GTK app + GTKDynamite | +’
 * Qt app + QtBomb   | +’

Prior art
We'll note here that the GTKDynamite + QtBomb trick is not specific to XternalApplications, such libraries already exist and are used to extract the main menu bar of applications amd move it to the top of the screen (this is called a "global menu"). This feature has been present for a long time on macos, though a more notable feature is the search bar which can search through an application's menus and settings, in a way exporting a complete list of features to the search facility.

Furthermore, some existing software (e.g. Suckless tabbed) which embeds windows into tabbed interfaces (or equivalently, draws window title bars as narrow tabs instead of full-width rectangles, and allows stacking these windows, see for example the Stack & Tile feature of Haiku's window manager.

The idea of extracting parts of applications and embedding them elsewhere isn't new either, Haiku has [https://www.haiku-os.org/docs/userguide/en/gui.html#replicants Replicants], KDE has KParts, Gnome had a few [https://web.archive.org/web/20090807072219/http://library.gnome.org/devel/api# Bonobo] components, and several office suites have the ability to include parts of a document (say, a spreadsheet) into another (through mechanisms like OLE).

Statement
It is the author's opinion that the features mentioned in Prior art (namely tabbed groups of windows, and repositioned menus) should be the responsibility of the window manager, as it is the case in these examples; and that this responsibility should also include dockable widgets like toolbars, dockable submenus, status bars and so on.

The author will further argue that loading and saving files should be the responsibility of a desktop environment; instead of letting applications load documents, provide limited sets of tools, crash and loose our data, the software ecosystem should be built from individual tools, grouped into packages merely for convenience when downloading and installing.

The dominant approach to using application pushes users towards the following workflow:


 * Open app
 * pen document
 * Edit
 * Save
 * Close app
 * Eventually restart with another app because the first one lacked a feature (and hope it is able to import/export the file without losses)

In a system based on individual tools, where the desktop environment and window manager are in charge of tabs, toolbars and dockable widgets, the workflow switches to one which FreeCAD users are accustomed to:


 * Open document
 * Choose a set of tools (workbench)
 * Edit
 * Freely switch between sets of tools while editing the same part or different parts of the document
 * Freely open multiple documents in a single interface while still editing the first one

Complete example