Should This Method Consider Removing the Translation in the Patch's AffineTransform That the CoordinateTransform Added to It

[spikeliu]

I am trying to learn more about how this amazing tool works and run into some problems understanding some of its details.
I read the code of Patch class and found that most of methods will take care the translation the corrdinate transformation added to it (by calculating the bounding box of the current ct mesh and substracting its top left corner corrdinates number from the current translation of the patch), but this one doesn't. It seems to me that the method directly computes the inversed affine and the coordinate transformation of the patch, and apply them to the world coordinate sequentially.

TrakEM2/src/main/java/ini/trakem2/display/Patch.java

Line 2432 in 7b39b73

static public final double[] toPixelCoordinate(final double world_x, final double world_y,

Is this method for some specical purpose that I overlooked?

[acardona]

Hi Chao, It is possible that this method was simply forgotten, likely because it wasn't used much. Not having looked at it for about a decade, I would need to do some speleology to find out where it is called from, both from the core code and scripts, to try to understand why it doesn't adjust for the translation induced by the coordinate transform. Missatge de Chao Liu ***@***.***> del dia dv., 24 de març 2023 a les 2:51:

…

I am trying to learn more about how this amazing tool works and run into some problems understanding some of its details. I read the code of Patch class and found that most of methods will take care the translation the corrdinate transformation added to it (by calculating the bounding box of the current ct mesh and substracting its top left corner corrdinates number from the current translation of the patch), but this one doesn't. It seems to me that the method directly computes the inversed affine and the coordinate transformation of the patch, and apply them to the world coordinate sequentially. https://github.com/trakem2/TrakEM2/blob/7b39b73c3304d9f0bc259e4d006ec740d69ace3a/src/main/java/ini/trakem2/display/Patch.java#L2432 Is this method for some specical purpose that I overlooked? — Reply to this email directly, view it on GitHub <#33>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMKGRRUXZYFKLXYRC3BNLW5UDZJANCNFSM6AAAAAAWGAEP5A> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[spikeliu]

So glad to see your reply. Actually what I wanna do is to apply the tranformations including linear and non-linear ones to some non-image data (like spatial transcrtiptomics or marker locations) that has been aglined to the image which I use as the input to trakem2. I did some test by writing jhthon scripts in Fiji, to see if I can use some of the existing APIs to accomplish this task. Basically, I found one API below very useful to my goal, then I also found toPixelCoordinate() API in my question which seems to do the opposite job.

TrakEM2/src/main/java/ini/trakem2/display/Patch.java

Line 1392 in 7b39b73

final public CoordinateTransform getFullCoordinateTransform()

Next part is what I have tested and my best guess, please forgive my English, as it is a little difficult for me to describe something this complex as a non-native english speaker.
I put a random image in trakem2, made some transfomations in different combinations, like "only linear", "only non-linear", "some linear plus non-linear with or without a final linear". Then, I input an x-y coordiante (call it a), got a new corrdinate b by appling the corrdiante transform (call it ct or a list of it, ctl) CT1 retrieved from getFullCoordinateTransform(), and then call toPixelCoordinate(), which uses the global affine transformation AFF and a modified ct CT2 on b to see if I can get a again. Most of the time, I did get a or a floating number coordinate very close to a. I also printed out transformation info of X1, AFF and X2 (the latter two actually could be found in the project XML file, call these two Y) and their relationship can be found in the code which I have already metioned in my question. Obviously, X1 and Y are different, so X1 and inversed Y can not be the invertible transformation to each other, so there must be some special design in the code.
I then tried to understand first why trakem2 needs to concatenate (this concept also costed me some time, now I know it means to do a certain tranformation before others) a translation to the current global patch's affine transformation after very coordiate transformation. I think the reason lies in mpicbg.trakem2.transform.TransformMesh as it always try to put every pixel (or data or corrdinate) of the transformed image to positive axes by draging them towards the top left corner (0, 0) of the canvas and uses the boudingbox to record how much they translate (am I right about this?). So, if trakem2 ultilize this module to record and apply the transformation (also related to the rendering), it has to take care of this hidden translation after each corrdinate transformation. Maybe the corrdinate tranformation (ct or ctl) class itself, as X1 retrieved from getFullCoordinateTransform(), doesn't do any extra trick when calling apply or applyInPlace, it will strictly follow the each ct's algorithm to do the transformation. But when it comes to applying the inversed transformation, the code may need to create a mpicbg.trakem2.transform.TransformMesh instance to make it happen as indicated in the code in toPixelCoordinate() (as shown below), so there might be no need to modify the global AFF at all. Some other details in or related to mpicbg.trakem2.transform.TransformMesh also explain why sometime, I don't get a as the final output, it could be that b is not the mesh, so TransformMesh could only estimate the closet afine tranformation it can apply to b.

TrakEM2/src/main/java/ini/trakem2/display/Patch.java

Lines 2438 to 2445 in 7b39b73

	// Inverse the coordinate transform
	if (null != ct) {
	final double[] f = new double[]{d[0], d[1]};
	final mpicbg.models.InvertibleCoordinateTransform t =
	mpicbg.models.InvertibleCoordinateTransform.class.isAssignableFrom(ct.getClass()) ?
	(mpicbg.models.InvertibleCoordinateTransform) ct
	: new mpicbg.trakem2.transform.TransformMesh(ct, meshResolution, o_width, o_height);
	try { t.applyInverseInPlace(f); } catch ( final NoninvertibleModelException e ) {}

what if the code above choose to create a mpicbg.models.InvertibleCoordinateTransform instead of a mpicbg.trakem2.transform.TransformMesh. As in my test, all the transofmation was done by using the manual transform mode in the menu, I didn't try any other algorithms from the Align menu (once again, my compliment, trakem2 and Fiji is so powerful, it integrates so many tranformationi and registration algorithms), so my transoformations are always of affine and MLS transformation type, I don't know if any other model could lead to create a different type of instance here and will cause some trouble. Or, all possible models have been organized into ct class, so the ternary operator here would always choose the second branch, which is to create a mpicbg.trakem2.transform.TransformMesh, so it will cause no bug at all.